Whisper Base Mla 24lang

Whisper-Base-MLA (24 languages) — on-device-tier MLA-Whisper, 62.5% smaller decode KV-cache

openai/whisper-base (74M) with decoder self-attention converted MHA→MLA (Whisper-MLA, arXiv:2603.00563), recovery-fine-tuned on a 24-language application set — a compact, deployable tier of the 24-language line (whisper-tiny-mla-24lang is smaller still).

from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-base-mla-24lang", trust_remote_code=True)  # transformers==4.46.x

Honest sizing note (read first)

Conversion cost GROWS as the model shrinks — measured across the 24-language family: small ≈+0.6 → base ≈+1.1 → tiny ≈+1.9 (approximate per-language medians). At the base tier you pay ≈+1.1 WER (median) for the 62.5% cache cut, and absolute quality is base-tier (much higher WER than small — that is the base model, not MLA). If quality is the priority prefer whisper-small-mla-24lang; this tier is for memory-constrained deployment where the cache cut matters most.

Results (CommonVoice-17 test — Malay from FLEURS, n≤1500/lang (ko 339, no 370, ms 749, vi 1274; rest 1500); cost = paired vs an identically-trained control; CER for th/zh/ja)

lang	WER%	CER%	conversion cost (WER pts)	sig
en	22.0	11.6	1.51	sig
de	30.6	11.6	0.83	ns
es	19.6	6.8	0.77	sig
fr	33.7	14.4	0.80	sig
it	31.9	9.4	1.39	sig
pt	29.0	10.8	0.61	ns
ru	29.8	9.1	1.71	sig
nl	27.9	10.4	1.06	sig
pl	36.0	11.0	1.60	sig
id	39.9	14.1	1.31	sig
tr	40.2	12.2	1.39	sig
hi	37.0	18.7	0.63	sig
ms	36.7	12.8	1.65	sig
sv-SE	44.3	16.8	1.10	sig
th	66.9	24.0	0.17	ns
zh-CN	43.7	25.7	0.54	ns
cs	54.1	15.8	0.85	sig
vi	44.5	22.5	1.31	sig
fi	53.6	12.2	2.61	sig
el	55.3	20.7	3.69	sig
da	56.9	23.8	0.91	ns
ja	n/a	33.9	-0.42	ns
nn-NO	69.4	27.0	2.07	sig
ko	60.6	29.1	-1.57	ns

Thin-CommonVoice languages (Korean, Norwegian, Danish, Greek, Finnish, Czech, Vietnamese) are the weakest — a DATA limit, not an MLA one. At this base tier several are near-floor (WER: nn-NO 69.4, ko 60.6, da 56.9, el 55.3) — reported for transparency, marginal-to-unusable; at that error level the paired "conversion cost" is dominated by metric noise, not MLA. So restrict the size-cost curve to the usable-WER languages. Per-language cost also depends on the recovery mix — a language's cost here differs from the same language in whisper-base-mla-cv11 (the 11-language model); expected (mix composition moves the low-rank fit), not irreproducibility. Malay was trained on FLEURS (no CV Malay exists; FLEURS is CC-BY-4.0; its eval-only license respected — audio used for training, never redistributed).

Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-base-24lang (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.

Limitations

• What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
• Runtime: requires trust_remote_code + transformers==4.46.x (no whisper.cpp/CT2/faster-whisper).
• Language coverage: covers these 24 languages; erodes Whisper's others.
• Decoding: greedy-decode evals (beam-5 adds ~1-2 WER on both arms; conversion cost unchanged — measured).
• Domain: consumer-mic read-speech.
• Training: 15k steps, warmup+cosine, encoder frozen both arms, dev-selected, bf16 (weights released as fp16).