dNaViT: Discrete Codes are General-Purpose Visual Representations

dNaViT. Discrete Codes are General-Purpose Visual Representations. By Xiaotong Li and eleven others, from Meituan's LongCat Team. Published on the ninth of November.

Discrete autoregressive modeling, next-token prediction above all, is a cornerstone of modern language models. Carrying the same paradigm over to vision would let visual and language models converge on one framework, with shared methods and shared infrastructure. That raises the question the work sets itself: how far can a finite, discrete vocabulary go in representing the richness of the visual world?

The proposal is dNaViT, a Discrete Native-resolution Vision Transformer. It produces discrete visual primitives meant to play the part that tokens play in language, so that an image becomes a sequence of entries drawn from a fixed vocabulary. If that works, the machinery language models already use for next-token prediction has something visual to predict, and the two kinds of model can share it. The name carries the other part of the design: native resolution.

It does not start from scratch. dNaViT is built on a frozen pretrained encoder, and applies a lightweight residual modulation trained with a dual-branch objective, which injects reconstruction-aware cues into the encoder's semantic features. Capacity comes from grouped product quantization, which lets the discrete vocabulary grow without inflating the length of the spatial sequence. A balanced training recipe then keeps both kinds of information alive at once: the semantic, and the pixel-level. Each choice answers a pressure. The frozen encoder brings semantics. The modulation adds what reconstruction needs. The grouped quantization buys a richer vocabulary without longer sequences, and the balanced recipe stops either side from crowding out the other.

The result is meant as a versatile discrete visual tokenizer. The same codes serve unified encoding across downstream tasks, and can also be decoded back into high-fidelity images.

In extensive evaluations, dNaViT matches or exceeds task-specific baselines across twelve visual understanding benchmarks, four generation benchmarks, and an image editing benchmark. One tokenizer is set against specialists on each of those fronts.

Behind the paper sits a bet on convergence. If a compact discrete vocabulary can carry enough of the visual world for understanding, generation and editing alike, then images can be handled the way text already is. The authors call such a vocabulary a universal visual currency, and see it as the way to unlock fully unified, native multimodal methods.