Apertus 1.5: From Open Text Models to Open Multimodal Reasoners

Apertus 1.5. From Open Text Models to Open Multimodal Reasoners. By Project Apertus and one hundred and thirty-one others, from ETH Zurich, EPFL, and six other institutions. Published on the eleventh of October. No code repository is linked to it.

Apertus one point five extends the fully open Apertus family from text to multimodal reasoning. The new models understand images and audio as well as text, and their context window grows fourfold, to two hundred sixty-two thousand one hundred and forty-four tokens. Thinking is not a separate model but a flag in the chat template, and switching it on lifts mathematical reasoning by up to twenty points on the smaller model.

Two sizes are released, one with eight billion parameters and one with seventy billion. Multimodality enters through the tokenizer. The language-model head still produces only text, and the architecture is unchanged. What is released goes beyond the weights: data preparation and training recipes and the evaluation suites, all under permissive licenses, along with the Apertus charter that guided the alignment stages.

Both are continued pretraining of the Apertus one point zero models, on four trillion additional tokens for the smaller one and two trillion for the larger. Supervised finetuning, long-context finetuning, reinforcement learning with verifiable rewards, and preference alignment then complete them. As before, training uses only openly available data, respects crawler opt-outs retroactively, and excludes restrictively licensed material.

The motivation is about infrastructure. Language models are becoming infrastructure, the authors argue, and infrastructure built in the open, by public institutions, on documented data, is the alternative to depending on a few providers. For anyone weighing the models themselves, they improve over their predecessors on nearly every benchmark. Among fully open models of its class, the smaller model leads on knowledge, instruction following, and cultural benchmarks, and its instruction following surpasses every compared model, open-weight ones included.

Image and audio understanding are a first for the family. There, the models are competitive with fully open multimodal models, strongest on document understanding and on long-form and lower-resource speech recognition. Specialized vision and audio models remain ahead on perception, and on transcription of English and other high-resource languages.

Openness here reaches past data and code to values: the charter that steered alignment is published next to the models. The release is offered as evidence for a claim about the whole stack, that documented, public, fully open development is viable at scale.