Cohere Transcribe Arabic 07 2026 ONNX

--- license: apache-2.0 language: - ar - en pipeline_tag: automatic-speech-recognition tags: - audio - speech-recognition - transcription - onnx - directml base_model: - CohereLabs/cohere-transcribe-arabic-07-2026 --- # Cohere Transcribe Arabic 07-2026 — DirectML-optimized ONNX ONNX export of [CohereLabs/cohere-transcribe-arabic-07-2026](https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026) (2B, Arabic + English) restructured to run **fast on the ONNX Runtime DirectML EP**. Same architecture as [cohere-transcribe-03-2026](https://huggingface.co/Masterx/cohere-transcribe-03-2026-ONNX), hand-exported from the reference PyTorch weights (stock `optimum` can't export `cohere_asr`). ## What changed (`tools/onnx/cohere_decompose_attention.py` in [WinSTT](https://github.com/dahshury/WinSTT)) 1. **Loop-invariant cross-KV hoisted into the encoder** — cross-attention K/V (which depend only on the audio) are computed once per utterance in `encoder_model*.onnx`, not per decoded token. 2. **Static self-KV decode** — the self-attention KV cache is a fixed-length buffer updated by a masked write, so the DirectML EP compiles the decoder **once** per utterance instead of re-fusing its graph on every autoregressive step. Measured ~3.5 ms/token on DirectML (RTX 3080 Ti) vs ~16 ms/token on the CPU EP — **~4.6× faster**, entirely on the GPU. Two decoders per precision (both fed by the hoisted encoder, sharing the decoder sidecar): `decoder_model_merged*.onnx` (**static self-KV**, marker `winstt_static_kv`, fastest on DirectML) and `decoder_model_merged*_dyn.onnx` (**growing self-KV**, faster on the CPU EP). WinSTT picks per device. Numerically verified against the source on CPU (autoregressive logits parity: bit-exact fp32/q4). **Notes vs the multilingual export:** this torch-traced graph uses a `(B, S, nh, hd)` head layout, so the cross-attention is left dynamic (not padded to a fixed bucket) — perfectly fast for the ≤10 s segments a dictation VAD produces; only unrealistically long single clips would benefit from bucketing. The **int8** decoder ships **dynamic-only** (`decoder_model_merged_int8.onnx` is the growing-self graph): its `DynamicQuantizeLinear` scales interact badly with the fixed-KV write, so it runs the hybrid path (encoder-DML / decoder-CPU). fp32 and q4 get the full static decode. ## Files - `onnx/encoder_model[_int8|_q4].onnx` (+ ex...

Source context: 16 downloads · 2 likes · Pipeline automatic-speech-recognition · Repo Masterx/cohere-transcribe-arabic-07-2026-ONNX