A newer model does not automatically mean a better model on every task. Three months ago Dharma-AI published the DharmaOCR paper and open-sourced one of the models (Dharma-OCR-LITE, a 4B Image-Text-to-Text model) with a single target: Brazilian Portuguese OCR. In the months since, two newer and better-resourced entrants arrived — Mistral OCR4 and Unlimited-OCR. On July 16, Dharma-AI published a comparison putting all three on the same Portuguese document set. Let’s state the evidence’s provenance up front: the benchmark was designed by the Dharma team, covers Portuguese only, and is a vendor self-evaluation, not independent third-party testing. But the question it answers matters to anyone choosing an OCR model: is release date or domain training the dominant variable?
Three Months Later: No Free Victory
The scores first: DharmaOCR takes 0.925, the highest extraction quality score on this benchmark, with the lowest degeneration rate. Mistral OCR4 scores 0.798; Unlimited-OCR scores 0.7587 — roughly 13 and more than 16 points behind respectively. Note the timeline: both competitors shipped after DharmaOCR. Newer did not buy a win in this language domain.
| Model | Quality score | Gap |
|---|---|---|
| DharmaOCR | 0.925 | — (benchmark best, lowest degeneration) |
| Mistral OCR4 | 0.798 | ~13 points |
| Unlimited-OCR | 0.7587 | >16 points |
Two-Stage Training: Build the Domain, Then the Stability
DharmaOCR’s pipeline has two stages, each solving a different problem. The first is supervised fine-tuning on Portuguese documents that vary in source, format, and complexity. The effect is to concentrate the model’s representational capacity on the target language instead of spreading it across a multilingual space: the same parameter budget, spent entirely on Portuguese vocabulary, syntax, and document structure.
The second stage applies Direct Preference Optimization (DPO), with training signal drawn from preference data between competing outputs. What matters is which problem it solves: not accuracy, but stability. DPO suppresses the failure modes behind repetitive or incoherent output, and the side effects are lower inference time and cost plus materially better production reliability. SFT builds in-domain capability; DPO makes that capability hold under the conditions where models tend to fail. Neither stage is optional.
Why Token-Level Prediction Drifts: The DPO Mechanism
To see why DPO works, look first at SFT’s structural weakness. SFT optimizes individual token predictions: once an early token drifts from the source under ambiguous visual signal, every subsequent prediction is conditioned on the drifted state, errors reinforce themselves, and the model falls into repetition loops. This is not random noise — it is a drift path determined by the training objective itself.
DPO’s signal is different. It trains against the coherence of complete outputs rather than single-token accuracy, so the model learns to compare quality at the level of the whole extraction. On visually complex documents, it is less likely to commit to the one-way drift path. The two stages map onto two levels: token-level accuracy, and output-level stability.
Chico Buarque Becomes “Chico Barque”: The Shape of Errors
The most persuasive evidence comes from ENEM (Brazil’s national high school exam) handwritten essays. Mistral OCR4 transcribed Chico Buarque — one of Brazil’s most recognized musicians and poets — as “Chico Barque.” Unlimited-OCR rendered the same name as “chico bique.” Same name, two models, two corruptions — and the name is not obscure; it is nationally known. The corruption goes beyond names: confronted with the Chico Buarque line “O Brasil não exclui, assimila” (“Brazil does not exclude, it assimilates”), Unlimited-OCR returned “a dose de chico bique, ‘o Brasil no exclu, eliminila.’” — losing the quoted sentence’s structure entirely.
The errors are not random: multilingual models fail at precisely the vocabulary and proper nouns that distinguish Brazilian Portuguese from a broader multilingual corpus. Where errors concentrate is itself a diagnostic signal.
Text Degeneration Is the Production Problem
An ordinary transcription error and text degeneration are different categories of failure. The former keeps a correspondence with the source — it can be found and fixed. Degeneration happens when the visual signal turns ambiguous: small fonts, degraded scans, dense handwriting. The generative OCR model continues generating from prior learned patterns, producing content with no connection to the page. The article shows Mistral OCR4 emitting output on a small-font document with no relation to what was written. That is not a low-quality transcription; it is a different class of failure.
For production the distinction is structural: degenerated output has no correctable object. Fed into document classification, information extraction, or compliance workflows, it is structurally unusable data — not noise a pipeline can scrub out.
The Structural Logic of Specialization, and the Authors’ Own Caveat
Dharma-AI’s core claim is not “we stay ahead forever” but a structural argument: allocating finite parameters across many domains dilutes the commitment to any single domain; neuron superposition allows parameters to be reused but does not reverse that trade-off. The authors even state that future, newer models will likely outperform the current DharmaOCR even on Brazilian Portuguese — an expectation, not a risk. The advantage of specialization is not holding a leaderboard position; it is the leverage of concentrated resources. The source states plainly that this structural dynamic does not reverse. When to redirect that leverage toward a new domain is this article’s editorial judgment, not the source’s claim.
A Builder’s Checklist
When choosing an OCR model, don’t stop at average scores. Four things belong in your own evaluation: first, build a test set from your own documents, covering the layouts and quality distribution you actually face; second, log degeneration — whether low-quality scans produce source-less text; third, check where proper-noun errors concentrate, because that points at the model’s out-of-domain weakness; fourth, confirm the failure mode — does it refuse output or generate unusable content? If a general-purpose model already clears your bar on your documents, don’t specialize for specialization’s sake. But if errors concentrate on specific languages, layouts, or terminology, the two-stage recipe — in-domain SFT followed by DPO — is the genuinely transferable methodology in this case.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.