OvisOCR3 Brings Five Document Tasks into a 0.8B OCR Model

Introduction

OCR systems are increasingly expected to do more than recognize characters. Invoices, contracts, research papers, and business reports combine paragraphs, fields, figures, and tables, sometimes across several pages. A useful system must preserve these relationships rather than return an unstructured text stream. The OvisOCR3 Technical Report, featured by Hugging Face Daily Papers, presents a 0.8B model built around that broader view of document intelligence.

Key points

• Five tasks in one model. OvisOCR3 unifies document parsing, general OCR, key information extraction (KIE), document figure captioning, and cross-page table reconstruction. The target output therefore includes text, structure, extracted fields, and links between visual elements.
• Training is organized around task balance. The authors use both real and synthetic data in continued pretraining. During mid-training, they apply domain balancing to reduce the effect of different task volumes and response lengths, helping prevent one high-volume task from dominating the shared model.
• Knowledge is transferred from several specialists. Five domain-specific 4B teachers provide on-policy distillation signals. The process is followed by mixed-task reinforcement learning with domain-specific rewards, allowing the compact model to refine behavior for different OCR-related objectives.
• Evaluation spans multiple document settings. The report compares OvisOCR3 with general-purpose vision-language models and specialized OCR systems. Its reported results include a 96.74 overall score on OmniDocBench v1.6, leading overall results on several other document benchmarks, and first-place results on the English and Chinese splits of OCRBench v2. The summary also reports strong performance for figure localization and cross-page table relations.

Why it matters

The main contribution is not simply the parameter count. OvisOCR3 illustrates a practical recipe for building a unified document model: balance supervision across domains, use specialized teachers to transfer expertise, and apply task-aware rewards to correct the shared model’s outputs. If the reported behavior holds in real deployments, one compact model could reduce the need to maintain separate OCR, extraction, captioning, and table-processing pipelines. Its size may also make local or resource-constrained inference more feasible than larger general-purpose vision-language models.

There are important qualifications. Benchmark leadership does not guarantee consistent performance on every scan quality, layout, language, or industry vocabulary. Cross-page table reconstruction in particular can be sensitive to page breaks, missing regions, and ambiguous cell relationships. The supplied summary also does not provide complete details about latency, memory use, training-data scale, or task-by-task ablations. Those factors should be checked against the released weights, inference examples, and representative business documents. OvisOCR3 is best understood as a compact experiment in unified document intelligence, rather than as proof that one OCR model eliminates all specialized systems.

Source: Hugging Face Daily Papers