OvisOCR3 Technical Report

OvisOCR3 Technical Report. By Yu Xia and fourteen others, from Alibaba Group. Published on the twentieth of October.

Optical character recognition is not one task but a family of them. Parsing whole documents, reading text in general images, pulling out key information, writing captions for figures inside documents, and rebuilding tables that break across pages. Each of these has different amounts of training data and asks for responses of very different lengths. Serving them all well from a single small model means finding a way to keep the large tasks from crowding out the small ones.

OvisOCR3 is a model of zero point eight billion parameters that takes on all five of those task families at once. Document parsing, general OCR, key information extraction, document figure captioning and cross-page table reconstruction are unified in one model rather than split across specialists.

Training proceeds in stages. The model is first trained on real and synthetic data through continued pretraining. Domain-balanced mid-training follows, to balance supervision across tasks whose data volumes and response lengths differ. Then comes multi-teacher on-policy distillation: five domain-specific teachers, each of four billion parameters, transfer their expertise into the one shared model. The last stage is mixed-task reinforcement learning, with rewards specific to each domain.

OvisOCR3 is compared with general-purpose vision-language models and with specialized OCR models, on public and in-house benchmarks covering all five task families. On OmniDocBench version one point six it reaches an overall score of ninety-six point seven four. It posts the highest overall scores on PureDocBench, on Real5-OmniDocBench, and on MDPBench.

On OCRBench version two it ranks first, scoring sixty-five point nine on the English split and sixty-one point three on the Chinese split. It also beats the baselines on OCRBench and on all four key information extraction benchmarks. On DocFigCapBench it has the highest localization F1, with a caption score close to that of Qwen3.8-Max. For tables that cross pages, it leads in Relation F1 on CrossTabBench, and in truncated-table TEDS on MPDocBench-Parse.

The authors read these results as showing that a compact model of zero point eight billion parameters can perform strongly across diverse OCR-related tasks. The route there is a sequence of balancing acts: balance in mid-training, many teachers distilled into one student, and rewards shaped per domain. The model is released on Hugging Face.

OvisOCR3 Technical Report. By Yu Xia and fourteen others, from Alibaba Group. Published on the twentieth of October. This is the long version: the whole paper rather than the abstract.

Reading the characters on a page is now the easy part of OCR. Users want the structure of the whole page, fields pulled out to a schema, a description of a figure, and a table that runs across a page break recovered as one table. With Ovis OCR two, this team showed a model of zero point eight billion parameters could lead at page parsing. Widening it, they argue, is not a matter of adding tasks to the mix. A full page can run to thousands of tokens, a field extraction to a few, and what counts as correct differs by task. The goal is specialist performance on five task families from one compact model.

The five families are document parsing, general OCR, key information extraction, document figure captioning, and cross-page table reconstruction. All share one set of weights, and the instruction alone selects the task. Cross-page tables use the same model twice. First it parses each page on its own. Then it takes two adjacent page images with their parses, and returns either the merged table or an explicit statement that nothing continues. For longer tables it works backward from the last pair, writing each merged table into the earlier page before pairing it with the page before.

The imbalance the paper is built around shows up in the data. Even with examples balanced by count, general OCR supplies twenty-two point eight percent of the examples but only five point two percent of the supervised tokens. Key information extraction supplies sixteen point one percent of the examples and two point five percent of the tokens.

Training runs in four stages, starting from Qwen three point five at zero point eight billion parameters. Continued pretraining updates the whole model on roughly seventy-two million sampled examples, mostly parsing and general OCR. Mid-training then rebalances to about eight point eight million examples, and here the answer to the imbalance appears: two branches trained from the same start. One averages the loss over all tokens, which favours long answers. The other averages within each example first, so every example counts equally. The example-level version did better on the short-answer tasks. Rather than pick one, the authors merge the two branches' weights and carry the result forward.

Next comes distillation from five teachers. Each is a four billion parameter Qwen model, fine-tuned and then trained with reinforcement learning on one domain. The student generates its own answer, and the teacher for that example's domain grades every token, given the same input and the student's own prefix. Two ways of turning that into a loss were compared. A policy-gradient version uses the log-probability gap on each sampled token as a reward. A top-K version matches the two distributions over the student's top candidate tokens, minimising a reverse divergence capped at each position. Top-K won, mostly on parsing and general OCR. The last stage is reinforcement learning across all five domains at once, with group-relative advantages so that very different rewards can drive one policy. Extraction is penalised for filling fields that should be empty, dropping fields that are not, or putting values under the wrong key. Cross-page tables use a relation-first rule: merge quality is scored only when every candidate in the group gets the page pairing right. Otherwise the reward reflects pairing alone. The teachers are used only in training.

The comparison that most directly tests the thesis is against Ovis OCR two, the same size but trained for parsing alone. If adding four other task families diluted the model, parsing would slip. It does not. On OmniDocBench version one point six the score rises from ninety-six point five eight to ninety-six point seven four. That is not first place: Xiaomi OCR zero scores ninety-six point eight three, which the abstract does not mention. The larger gains are in hard conditions. On PureDocBench the drop from clean renders to recaptured pages is seven point eight four points, against fifteen point zero four for the predecessor. On MDPBench the gap between Latin and non-Latin scripts narrows from twenty-three point seven points to one point five.

Outside parsing, the English split of OCRBench version two puts it eleven point eight points above the four billion parameter Qwen. The two places it trails Qwen three point eight Max are telling. Its figure caption score, judged out of ten, is six point nine three against seven point three. And on CrossTabBench it wins on detecting continuations, a relation score of ninety-seven point zero seven against ninety-four point four seven, but loses on the quality of the merged tables, eighty-nine point six six against ninety-one point seven one.

There is no limitations section, and the conclusion names no weaknesses. What the evidence leaves open can still be stated. The key design choices are reported without numbers: that example-level loss helps short tasks, that top-K beats policy gradients, that merging the branches pays. No table isolates what each stage adds. Three benchmarks are the authors' own, including DocFigCapBench and CrossTabBench. The caption score is assigned by Qwen three point eight Max, which is also the strongest model being compared. Many baseline scores are copied from leaderboards rather than rerun, and every result is a single number with no spread across runs.

The idea to carry away is that unifying tasks is a balancing problem at every layer. Balance examples and the tokens still tilt toward long answers, so two normalisations are trained and fused. Each example is distilled from the teacher for its own domain, and rewarded by that domain's notion of correct. What comes out is still one model under a billion parameters, with nothing task-specific left at inference.