Sketchpad: certified exact arithmetic with a small vision-language model

Sketchpad. Certified exact arithmetic with a small vision-language model. By Mostafa 'Neo' Mohsenvand, from the MIT Media Lab. Published on the third of November.

Language models get most long-arithmetic problems right. But that accuracy is an estimate, measured on samples, and it carries no guarantee for an input nobody has tried yet. The usual escape is tool use, which is exact, but it achieves exactness by moving the computation out of the network altogether. The model hands the sum to a calculator and is no longer the thing doing the arithmetic. So there are two options on the table: a network that is usually right, or a correct answer the network did not compute. Neither gives you a network you can certify.

The paper's claim is that a third option exists. Exactness at every input length becomes a finite, checkable property of a network, provided each of its steps reads only a bounded view of an external working memory. If a step can only ever see a small window, then there are only finitely many windows it could be shown, and checking the network on all of them is a job with an end.

Sketchpad puts that into practice with Qwen3.5-0.8B and a low-rank adapter. The model executes arithmetic procedures on a rendered sheet. Each forward pass reads a three-cell view and emits one step: a digit, a carry or borrow, and the next operation. A fixed runtime moves the view around the sheet, and it writes no digit of any result. Every digit comes from the model.

The guarantee is proved in Lean, in several parts. The model is exact on all inputs of any length if and only if it is exact on the finite set of views the runtime presents. When steps are also allowed to read earlier steps, no finite test suffices, which is why the bounded view matters. And the guarantee tolerates bounded numerical perturbation, and extends to compositions of procedures.

The check itself was run where the model is deployed: a laptop GPU running the model inside a web browser. All two thousand four hundred and forty-six views pass. The certified model performs signed addition and subtraction, long multiplication and division, square roots and greatest common divisors, at arbitrary length. In the browser it solved two hundred of two hundred pre-registered problems.

A second test used sums of up to one thousand digits and products of up to twenty digits. Sketchpad solved all two hundred and twenty. Ten other models were compared on the same set, and the strongest of them, GPT-6.1 Sol, solved two hundred and seventeen. Only one of those two results rests on a certificate.

Two further results extend the reach. A symbolic runtime built on these procedures solves, expands and factors polynomials, with every number in the process computed by the model. And a multiplicative gate on the adapter recovers the base model exactly whenever the sheet is not in use, so the arithmetic skill does not cost the model what it could do before.

What changes here is the kind of statement one can make about a network. Instead of an accuracy rate over the problems someone happened to sample, there is a proof that covers every length, resting on a check that has finitely many cases and was actually completed. The price is architectural discipline: each step sees only a small view, and the runtime that moves it never touches the answer.