V-CoLA: Vision Token Compression with Linear Attention

V-CoLA. Vision Token Compression with Linear Attention. By Hao Jiang and eleven others, from Alibaba Group. Published on the eighth of October.

Vision-language models are capable, but they carry a heavy computational cost, and most of it comes from one place. Vision tokens dominate the input sequence. That has made vision token compression a key direction for easing the burden: keep the tokens that matter, drop or fold the rest. The difficulty is that the architectures underneath are changing. Hybrid designs that incorporate linear attention, with Qwen3.5 given as the example, are now appearing, and the compression methods built for softmax attention struggle to generalize to them. The authors' analysis finds that both families of prior approach, those based on attention and those based on similarity, suffer notable performance degradation in this setting. Their conclusion is that linear attention needs compression methods designed for it.

V-CoLA is their answer: an efficient, training-free token compression framework built specifically for linear attention. Nothing is retrained. The framework decides which vision tokens are critical and compresses the remainder, and it does so in a way meant to fit how linear attention actually runs.

Two components do the work. The first is a uniqueness-aware importance criterion, which is how V-CoLA identifies the vision tokens that should be kept. The second is an adaptive token merging strategy, which carries out the compression itself. Around both sits an engineering commitment. Every component is optimized at the implementation level to stay compatible with the chunk-wise parallelism of linear attention. That is the part aimed at practical value: a compression scheme that broke the parallel structure of the model it was meant to accelerate would give back what it saved.

Experiments span multiple benchmarks. With only fifty percent of the vision tokens, V-CoLA keeps ninety-nine point five percent of the original performance. Pushed much further, down to twelve point five percent of the tokens, it still retains more than eighty-eight percent.

On speed, the prefill stage runs faster by a factor that ranges from one point eight six times at the low end to six point one five times at the high end.

The argument runs in a straight line. A new attention regime arrived, the old compression tricks stopped transferring, and V-CoLA rebuilds token selection and merging for that regime without any training. Its claim is that close to full accuracy survives at half the vision tokens, and that the savings show up as real prefill time because the method respects chunk-wise parallelism.