Skip to content
ModelListedExternal
Download

Silent Lip Reader Model

The visual-speech-recognition (lip-reading) model weights used by the Silent Lip Reader Space. Re-hosted here so that the open-source Space is self-contained and does not break if upstream repos move.

Sourcehuggingface.co/aaahmet/silent-lip-reader-model/tree/ff1e35515594eb4562c469b99d147cf6a3d2891e
KindOtherVersionPytorch modelBgrade

Model source

Kind
Other
File size
955 MB
Version
vPytorch model
Updated
Oct 10, 2026
Source
Hugging Face

Source description

Silent Lip Reader — VSR weights

The visual-speech-recognition (lip-reading) model weights used by the Silent Lip Reader Space. Re-hosted here so that the open-source Space is self-contained and does not break if upstream repos move.

  • Architecture: Auto-AVSR — ResNet-3D + Conformer encoder, Transformer decoder, joint CTC/attention. Input: 88×88 grayscale mouth crops @ 25fps. Output: text via a 5000-unit SentencePiece (unigram5000) vocabulary. Video-only (no audio path).
  • Files: pytorch_model.pt (state dict), unigram5000.model, unigram5000_units.txt.

Credits / provenance (please read)

This checkpoint is not trained by the re-host. Honest attribution:

  • Model architecture & training: Auto-AVSR (Pingchuan Ma et al., "Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels"). All model credit to the original authors.
  • Checkpoint source: mirrored from AD1TEYA/lip-reading-model on the Hub.
  • Re-host + the surrounding system, demo, visual-VAD pipeline, evaluation and research: Ahmet Dedeler (🤗 aaahmet).

Intended use

Research and demos of silent visual speech recognition. The weights were trained on LRS3-derived data; treat as research use. Best on clear, frontal, well-articulated English. ~25–30% WER on clean speech, higher on casual speech (lip reading is inherently ambiguous — many phonemes look identical on the lips).

Usage

Used by the Silent Lip Reader Space — record a (silent) video, it crops your mouth, chunks utterances by lip motion, and decodes text. See the Space for the full pipeline and research log.

Built / curated by Ahmet Dedeler — https://ahmetdedeler.com. A credit/link back is appreciated if you use this. License MIT (follows the upstream Space).

Sources

1 source
Hugging FacePrimaryVersion ff1e35515594eb4562c469b99d147cf6a3d2891e
huggingface.co/aaahmet/silent-lip-reader-model/tree/ff1e35515594eb4562c469b99d147cf6a3d2891e

Verified Oct 10

Model artifacts

1 artifact

pytorch_model.pt

pt · 955 MB · SHA-256 fbf7cd70ff1c…8ddb · Hugging Face

Download file

Source excerpts

2 excerpts
ReadmeAug 2huggingface.co/aaahmet/silent-lip-reader-model

--- license: mit tags: - lip-reading - visual-speech-recognition - vsr - silent-speech - auto-avsr library_name: pytorch pipeline_tag: video-classification --- # Silent Lip Reader — VSR weights The visual-speech-recognition (lip-reading) model weights used by the [**Silent Lip Reader** Space](https://huggingface.co/spaces/aaahmet/silent-lip-reader). Re-hosted here so that the open-source Space is **self-contained** and does not break if upstream repos move. - **Architecture:** Auto-AVSR — ResNet-3D + Conformer encoder, Transformer decoder, joint CTC/attention. Input: 88×88 grayscale mouth crops @ 25fps. Output: text via a 5000-unit SentencePiece (`unigram5000`) vocabulary. **Video-only** (no audio path). - **Files:** `pytorch_model.pt` (state dict), `unigram5000.model`, `unigram5000_units.txt`. ## Credits / provenance (please read) This checkpoint is **not** trained by the re-host. Honest attribution: - **Model architecture & training:** **Auto-AVSR** (Pingchuan Ma et al., *"Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels"*). All model credit to the original authors. - **Checkpoint source:** mirrored from `AD1TEYA/lip-reading-model` on the Hub. - **Re-host + the surrounding system, demo, visual-VAD pipeline, evaluation and research:** **[Ahmet Dedeler](https://ahmetdedeler.com)** ([🤗 aaahmet](https://huggingface.co/aaahmet)). ## Intended use Research and demos of silent visual speech recognition. The weights were trained on LRS3-derived data; treat as **research use**. Best on clear, frontal, well-articulated English. ~25–30% WER on clean speech, higher on casual speech (lip reading is inherently ambiguous — many phonemes look identical on the lips). ## Usage Used by the Silent Lip Reader Space — record a (silent) video, it crops your mouth, chunks utterances by lip motion, and decodes text. See the [Space](https://huggingface.co/spaces/aaahmet/silent-lip-reader) for the full pipeline and research log. --- Built / curated by **Ahmet Dedeler** — https://ahmetdedeler.com. A credit/link back is appreciated if you use this. License MIT (follows the upstream Space).

NoteAug 2huggingface.co/aaahmet/silent-lip-reader-model

Source context: 0 downloads · 4 likes · Pipeline video-classification · Library pytorch · Repo aaahmet/silent-lip-reader-model