Rekody Streaming En 0.6b Int8

rekody-streaming-en-0.6b-int8

An int8 ONNX conversion of NVIDIA's Nemotron speech streaming 0.6B model (the March 2026 checkpoint), packaged for on-device streaming dictation. This is the model behind Rekody, the private, local-first voice dictation tool for macOS.

• Cache-aware streaming ASR: transcribes while you speak, in 560 ms chunks, with encoder cache carried across chunks. No audio leaves the machine.
• Runs on the ONNX Runtime CPU execution provider. The files load directly with onnxruntime or parakeet-rs.
• Four files: encoder.onnx (int8, 880 MB), decoder_joint.onnx (int8, 11 MB), tokenizer.model (SentencePiece), and config.json.

Why trust this conversion

• The conversion pipeline was validated bit for bit against the prior public int8 conversion of this architecture: when fed identical inputs (the same upstream checkpoint revision), our pipeline reproduces that artifact's tensors exactly. What ships here is the same pipeline applied to NVIDIA's current (March 2026) checkpoint.
• The cost of int8 quantization was measured as statistically zero. On LibriSpeech test-clean (n = 2620), int8 vs the fp32 export it came from differs by +0.072 pp WER with a paired bootstrap 95% CI of [-0.004, +0.150], which includes zero.

Benchmarks

Word error rate (WER, %), lower is better. Measured with the Open ASR Leaderboard methodology (hf-audio/open-asr-leaderboard @ b6bdcd0b, EnglishTextNormalizer), on identical audio and with the identical normalizer for both models.

Dataset	rekody-streaming-en-0.6b-int8	openai/whisper-large-v3-turbo
AMI	12.36	13.86
Earnings22	12.27	10.81
Gigaspeech	8.64	8.34
LibriSpeech test-clean	2.09	1.60
LibriSpeech test-other	4.92	3.75
SPGISpeech	2.99	2.78
Voxpopuli	2.74	5.31
Average	6.57	6.64

Note the operating points: this model streams, producing text as audio arrives in 560 ms chunks, while whisper-large-v3-turbo is a batch model that sees each full recording before emitting anything. Matching a strong batch model's average WER under a streaming constraint is the point of this model.

License

Two licenses apply to this repository, and users must comply with both:

1. The conversion work (the int8 quantization, packaging, and benchmarking published here) is licensed under the PolyForm Shield License 1.0.0 (see LICENSE). PolyForm Shield is a noncompete license: any use is permitted except providing a product that competes with the software or with the licensor's products built on it.
2. The base model weights remain licensed by NVIDIA Corporation under the NVIDIA Open Model License (see NOTICE and the dated copy in licenses/).

File integrity (SHA-256)

File	SHA-256
encoder.onnx	83538392358e90c40592f7cf99ee65ac7dca5d144edb999ce028c372318a5753
decoder_joint.onnx	89cae615623e41e94cc6b428708cd1a89a22965606fe7ded814b1d8e20c87368
tokenizer.model	07d4e5a63840a53ab2d4d106d2874768143fb3fbdd47938b3910d2da05bfb0a9
config.json	19c6a445ddf0268a8ab3653f0cdf5740196271ba0a902eaae8d17de52f74a829

These hashes are also published in SHA256SUMS.

Provenance

• Source: nvidia/nemotron-speech-streaming-en-0.6b, revision df1f0fe9dfdf05152936192b4c8c7653d53bf557 (the March 13, 2026 checkpoint).
• Conversion recipe: NeMo 2.7.3 ONNX export, then onnxruntime quantize_dynamic QInt8 over MatMul and LSTM ops.

Contact: hi@rekody.com