Arabic Htr Fanar 7b LoRA

arabic-htr-fanar-7b-lora

Arabic handwritten text recognition (HTR). A QLoRA adapter for QCRI/Fanar-2-Oryx-IVU that turns a handwritten Arabic line image into Arabic Unicode text.

On the held-out KHATT v1.0 test split (1038 real handwritten lines) it reaches 3.77% CER / 16.50% WER — 36% fewer character errors than the best other model measured here (Arabic-English-handwritten-OCR-v3, 5.86% CER, zero-shot). Fine-tuning took the same base model from 50.98% → 3.77% CER (93% error reduction).

Read this before comparing: this adapter is fine-tuned on the KHATT training split and evaluated on the KHATT test split — an in-domain result. The other models in the table below are evaluated zero-shot; they were not trained on KHATT. The table therefore shows "how well does each model read KHATT handwriting today", not "which model is better in general". A model fine-tuned on their data would score differently. Every number was measured under one identical protocol (same images, prompt, greedy decoding, normalization) — see Evaluation.

Results

#	Model	Params	CER ↓	WER ↓	Exact ↑	Lines ≤5% CER	Notes
1	Ours — Fanar + LoRA (generalist mix)	8.29B	3.77%	16.50%	29.4%	77.6%	KHATT + Muharaf + wiki-synthetic (built for generality)
2	Ours — Fanar-2-Oryx-IVU + LoRA	8.29B	4.11%	15.96%	31.7%	79.2%	fine-tuned on KHATT train (in-domain)
3	Arabic-English-handwritten-OCR-v3	3.75B	5.86%	18.30%	28.8%	69.1%	community Arabic/English handwriting model
4	Arabic-handwritten-OCR-4bit-Qwen2.5-VL-3B-v3	3.75B	5.96%	18.88%	28.9%	70.5%	community model, trained on Muharaf — never saw KHATT
5	Ours — Qwen2.5-VL-3B + LoRA	3.75B	6.12%	18.43%	29.3%	74.1%	fine-tuned on KHATT train (in-domain)
6	Qwen2.5-VL-7B-Instruct	8.29B	50.85%	82.56%	0.1%	0.5%	general-purpose VLM
7	Fanar-2-Oryx-IVU (base, zero-shot)	8.29B	50.98%	82.58%	0.1%	0.5%	our base model, before fine-tuning
8	Qwen2.5-VL-3B-Instruct	3.75B	73.35%	93.92%	0.1%	0.1%	general-purpose VLM
9	Qari-OCR-v0.3 (Arabic OCR)	2.21B	235.55%	126.10%	0.0%	0.0%	Arabic OCR specialist — built for PRINTED text

CER = character error rate (verbatim: diacritics and punctuation count). Lower is better. Full raw numbers: benchmark.json in this repo.

CER comparison

Corpus-level CER hides the shape of the errors, so here is the quality profile — what share of lines land under a given error threshold:

Quality profile

Accuracy vs size

Quality profile of this model: median line CER 1.89%, with 77.6% of lines at ≤5% CER, 92.5% at ≤10%, and 29.4% transcribed exactly.

Sample predictions

Ground truth	Prediction	CER
ذهب نوح مظفر ضرغام بصحبه رؤوف بن لؤي رايق ظافر	ذهب نوح مظفر ضرغام بصحبه رؤوف بن لؤي رايق ظافر	0.0%	good
عطعوط وهلال خازن عفيف للحج . بدأت قوافل	عطعوط وهلال خازن عفيف للحج . بدأت قوافل	0.0%	good
أصحابنا ظ ع ك ، ث خ ض، ب س ش، ص غ هـ أننا في الحج هل تعلم فائدة الكلمات التالية	أصحابنا ظ ع ك ، ث خ ض، ب س ش، ص غ هـ أننا في الحج هل تعلم فائدة الكلمات التالية	0.0%	good
كان جاري في الخيمه بيكلم وهو نائم بكلمات لا أفهمها مثل انقض بغـلس له الضابط	كان جاري في الخيمه يتكلم وهو نائم بكلمات لا أفهمها مثل انقض بغـلس له الضابط	2.7%	typical (median)
نس .	لكل .	75.0%	failure case

Usage

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
from PIL import Image

BASE = "QCRI/Fanar-2-Oryx-IVU"
ADAPTER = "mabdulaziz499/arabic-htr-fanar-7b-lora"

processor = AutoProcessor.from_pretrained(BASE, min_pixels=200704, max_pixels=802816)
model = AutoModelForImageTextToText.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

PROMPT = "Transcribe the handwritten Arabic text in this image exactly as written, including any diacritics. Output only the transcription, nothing else."
image = Image.open("line.png").convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": PROMPT}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=160, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0].strip())

Input expectations: one single line of handwritten Arabic per image (as in KHATT). For a paragraph or a form, segment it into lines first and call the model per line — see Limitations.

Output: Arabic text in logical Unicode order (NFC). Use arabic-reshaper + python-bidi only for terminal display, never for storage or scoring.

Training

Base model	QCRI/Fanar-2-Oryx-IVU
Method	QLoRA — 4-bit NF4 base, LoRA r=16, α=32, dropout=0.05
Adapted modules	attention + MLP projections of the language model (vision tower frozen)
Trainable params	40.4M (LoRA only)
Data	KHATT v1.0 train+validation (16882 lines after leakage filtering) + font-rendered synthetic augmentation
Schedule	3000 steps ≈ 2.8 epochs, effective batch 16, lr 0.0001, cosine, warmup 0.03
Compute	2.6 h on a single 96 GB GPU
Final train loss	0.2956

Data hygiene: KHATT contains fixed paragraphs copied verbatim by many writers, so the same sentence appears in both train and test in different handwriting. Every training line whose exact text occurs in the test split was removed before training (~1,700 lines). Without that filter a model can memorize the test text and report a flattering CER.

Evaluation

All models in the table were scored by the same script under one protocol:

• Data: local_khatt:test — 1038 real handwritten lines, held out.
• Prompt: identical for every model (above).
• Decoding: greedy (do_sample=False), max_new_tokens=160.
• Image budget: identical (min_pixels=200704, max_pixels=802816).
• Metric normalization (NFC, logical order, whitespace-collapsed, diacritics KEPT): references and hypotheses pass through the same normalizer; diacritics are kept and count as errors (verbatim scoring). A diacritic-insensitive secondary score is in benchmark.json.
• Text order: KHATT mirrors store ground truth in visual (reversed) order; it is restored to logical order by grapheme-cluster reversal before scoring. Getting this wrong silently invalidates CER.

Limitations

• Single-line recognizer. Multi-line images are out of distribution: the model may fall back to a memorized KHATT sentence. Segment into lines first.
• Domain. Trained on KHATT: mostly young Saudi writers, clean scans, black ink on white paper, MSA prose. Photos, ruled/form paper, stamps, cheque-style amounts, dialectal or heavily abbreviated writing are all further from the training distribution — expect degradation.
• In-domain result. 3.77% CER is a KHATT number. Your handwriting is not KHATT; measure on your own data before trusting it.
• Not printed-text OCR. For printed Arabic use a print OCR model.
• Numbers/dates are rare in KHATT and correspondingly weak.
• No diacritic supervision. KHATT labels are largely undiacritized, so the model outputs bare text.

License & data terms

The adapter is released under Apache-2.0, matching the base model QCRI/Fanar-2-Oryx-IVU (Apache-2.0). Using it requires downloading that base model under its own license. It was trained on KHATT v1.0, distributed for research; check the dataset's terms before commercial use and cite the KHATT authors in publications.

Citation

@misc{arabic_htr_fanar_7b_lora,
  title  = {arabic-htr-fanar-7b-lora: a QLoRA adapter for handwritten Arabic line recognition},
  author = {Abdelaziz, Mahmoud},
  year   = {2026},
  url    = {https://huggingface.co/mabdulaziz499/arabic-htr-fanar-7b-lora}
}

Please also cite KHATT (Mahmoud et al., Pattern Recognition, 2014) and the base model Fanar-2-Oryx-IVU.