Summary: Dharma-AI published a July 16 comparison claiming its Portuguese-specialized DharmaOCR scored 0.925, ahead of newer Mistral OCR4 at 0.798 and Unlimited-OCR at 0.7587. The vendor-designed benchmark is not independent, but it illustrates how domain-specific SFT followed by DPO can improve extraction quality and reduce degeneration. The authors themselves expect future newer models to surpass DharmaOCR, so specialization is presented as a structural trade-off rather than permanent leadership.
A model getting newer is no guarantee it gets better at your particular task. Three months back, Dharma-AI put out the DharmaOCR paper and open-sourced one of its models (Dharma-OCR-LITE, a 4B Image-Text-to-Text model) aimed at exactly one job: Brazilian Portuguese OCR. Since then, two newer and better-funded rivals have taken the stage — Mistral OCR4 and Unlimited-OCR. On July 16, Dharma-AI published a comparison running all three over the same Portuguese document set. Provenance first: the benchmark is the Dharma team’s own design, Portuguese-only, a vendor self-evaluation rather than independent third-party testing. Yet the question it tackles faces everyone picking an OCR model — is release date or domain training the variable that actually dominates?
Three Months Later: No Free Victory
Scores on the table: DharmaOCR posts 0.925, the top extraction quality score on this benchmark, alongside the lowest degeneration rate. Mistral OCR4 lands at 0.798 and Unlimited-OCR at 0.7587 — gaps of roughly 13 and more than 16 points. And mind the timeline: both competitors shipped after DharmaOCR did. Newer bought no victory in this language domain.
| Model | Quality score | Gap |
|---|---|---|
| DharmaOCR | 0.925 | — (benchmark best, lowest degeneration) |
| Mistral OCR4 | 0.798 | ~13 points |
| Unlimited-OCR | 0.7587 | >16 points |
Two-Stage Training: Build the Domain, Then the Stability
DharmaOCR’s pipeline runs in two stages, each treating a different ailment. Stage one is supervised fine-tuning over Portuguese documents spanning varied sources, formats, and complexity levels. The point is to gather the model’s representational capacity onto the target language rather than smearing it across a multilingual space: one parameter budget, spent wholly on Portuguese vocabulary, syntax, and document structure.
Stage two brings in Direct Preference Optimization (DPO), with training signal drawn from preference data between competing outputs. What counts is the problem it treats: not accuracy but stability. DPO suppresses the failure modes behind repetitive, incoherent output, and the side effects arrive as gifts — lower inference time and cost, and materially better production reliability. SFT erects the in-domain capability; DPO keeps that capability standing under precisely the conditions where models tend to buckle. Neither stage is negotiable.
Why Token-Level Prediction Drifts: The DPO Mechanism
To grasp why DPO works, start with SFT’s structural weak point. SFT optimizes token-by-token prediction: the moment an early token strays from the source under ambiguous visual signal, every later prediction is conditioned on that drifted state, errors compound, and the model weaves itself into repetition loops. That is not random noise — it is a drift path laid down by the training objective itself.
DPO’s signal runs on different rails. It trains against the coherence of complete outputs instead of single-token accuracy, so the model learns to judge quality at the level of the whole extraction. Faced with visually dense documents, it is far less likely to step onto that one-way drift path. The two stages thus split the labor across two levels: token-level accuracy, and output-level stability.
Chico Buarque Becomes “Chico Barque”: The Shape of Errors
The most convincing exhibit comes from ENEM (Brazil’s national high school exam) handwritten essays. Mistral OCR4 transcribed Chico Buarque — among Brazil’s most recognized musicians and poets — as “Chico Barque.” Unlimited-OCR rendered the very same name as “chico bique.” One name, two models, two corruptions — and hardly an obscure string, since Chico Buarque is a national figure. The damage extends past names: shown the Chico Buarque line “O Brasil não exclui, assimila” (“Brazil does not exclude, it assimilates”), Unlimited-OCR returned “a dose de chico bique, ‘o Brasil no exclu, eliminila.’” — shedding the quoted sentence’s structure altogether.
These errors are not scattered at random: multilingual models stumble precisely on the vocabulary and proper nouns that set Brazilian Portuguese apart from a broader multilingual corpus. Where the errors cluster is itself a diagnostic signal.
Text Degeneration Is the Production Problem
An ordinary transcription error and text degeneration belong to different species of failure. The former keeps some correspondence with the source — findable, fixable. Degeneration strikes when the visual signal goes ambiguous: small fonts, degraded scans, dense handwriting. The generative OCR model keeps generating from prior learned patterns and emits content with no tether to the page. The article shows Mistral OCR4 producing output on a small-font document bearing no relation to what was written — not a low-quality transcription but a different class of failure altogether.
For production, the distinction is structural: degenerated output offers nothing to correct. Once it flows into document classification, information extraction, or compliance workflows, it is structurally unusable data — not noise a pipeline can scrub clean.
The Structural Logic of Specialization, and the Authors’ Own Caveat
Dharma-AI’s central claim is not “we stay ahead forever” but a structural argument: spreading finite parameters across many domains dilutes the commitment to any single one; neuron superposition lets parameters do double duty yet never reverses that trade-off. The authors say outright that future, newer models will likely outperform today’s DharmaOCR even on Brazilian Portuguese — an expectation, not a risk. Specialization’s advantage lies not in defending a leaderboard slot but in the leverage of concentrated resources; the source states plainly that this structural dynamic does not reverse. As for when that leverage should swing toward a new domain, that is this article’s editorial judgment, not the source’s position.
A Builder’s Checklist
When choosing an OCR model, refuse to stop at average scores. Four items belong in your own evaluation: first, assemble a test set from your own documents, covering the layouts and quality spread you actually face; second, log degeneration — whether low-quality scans spawn source-less text; third, examine where proper-noun errors cluster, because that arrow points at the model’s out-of-domain weakness; fourth, pin down the failure mode — does the model decline to answer, or generate unusable content? If a general-purpose model already clears your bar on your documents, don’t specialize for specialization’s sake. But where errors concentrate on specific languages, layouts, or terminology, the two-stage recipe — in-domain SFT followed by DPO — is the genuinely transferable methodology this case hands over.