Region-Level Policy Optimization for Fine-grained MLLM Perception

Region-Level Policy Optimization for Fine-grained MLLM Perception. By Yuheng Shi and three others, from the University of Sydney and the City University of Hong Kong. Published on the seventeenth of September. Its code repository is worked out: about six hundred and fifty source files, with a readme and a licence.

Fine-grained visual perception in multimodal language models is usually improved by raising the resolution. The extra visual tokens inflate the cost of vision encoding and of language-model prefilling. The authors show that the two operations underneath fine-grained perception have different needs. Localizing the region of interest and recognizing what is in it do not require the same resolution. In a controlled diagnostic, localization tolerates roughly three to four times stronger token compression than recognition. Finding where to look needs less detail than reading what is there.

That suggests a plan: localize from a coarse view, then concentrate resolution on the evidence selected. Two ways of localizing each fall short. Decoding coordinates with the model itself can be trained end to end from answers, but costs a full model pass per query and depends on grounding ability. A lightweight proposal network, distilled from the model's attention, is fast, but inherits the noise in the attention it learned from. And because its region reaches the answer only through a discrete choice, how faithful that region is to the answer cannot directly supervise the network. The network picks a region, the model answers, and nothing in between tells the network whether its pick was the one the answer depended on.

Vision-RL2 optimizes the proposal network with reinforcement learning at the level of regions. Coherent regions are treated as actions. A frozen model, acting as a reader, scores each region by how much removing it changes the likelihood of the answer. A region that matters is one whose absence the reader notices. The reader stays frozen throughout. It judges regions; it is not trained.

Two complementary objectives, one subtractive and one additive, suppress distracting proposals and recover missing evidence. Only the predictor is updated, with no region annotations, no response sampling, and no reasoning trajectories. One objective deals with what the proposal includes and should not, the other with what it leaves out and should not. The refined proposal also enables a sparse encoding that magnifies the evidence and leaves background tokens out. Magnifying the evidence gives recognition the resolution it needs, and leaving out the background is where tokens are saved.

Across six fine-grained benchmarks and four model backbones, Vision-RL2 improves accuracy over the base model at every token budget. It also surpasses the base model's accuracy at its largest budget while using about four times fewer visual tokens. The comparison is against the base model at its most expensive setting, and Vision-RL2 beats it with about a quarter of the visual tokens. Code is available.

The design follows from the diagnostic. Since finding the region needs less resolution than reading it, the expensive detail is spent only where the answer lives, and the region finder is trained on the one signal that actually says where that is: whether the answer changes without it.