Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

Selection-Based Structured Reasoning. Toward Efficient Multimodal Search Agents. By Feiyu Gavin Zhu and eleven others, from Apple and Carnegie Mellon University. Published on the first of October. Its code repository is public and real: eighteen source files.

Multimodal agents usually write out free-form reasoning before each action they take. For a large model that habit may pay for itself. For a small model it often does not. Limited capacity can produce long stretches of reasoning that give the action step very little useful guidance, and every one of those tokens still costs inference time. The agent ends up paying heavily, turn after turn, for text that barely steers what it does next.

Selection-based Structured Reasoning, or SSR, changes the form of the reasoning rather than its quantity. Instead of generating reasoning openly, the model selects it. The recurring high-level reasoning an agent tends to need is written down ahead of time as a set of reusable natural-language candidates. At each turn, the model picks among them according to how likely each candidate is given the current context. No auxiliary task head is added to make that choice. The model's own likelihoods do the choosing.

Because the candidates are fixed in advance, they can be scored in parallel. Teacher-forced prefilling computes the token likelihoods of every candidate at once, both within a candidate and across candidates, and all of them share one key-value cache built from the context. Scoring a menu of options this way replaces the slow, step-by-step generation of a fresh passage of reasoning on every turn.

SSR is evaluated on seven multimodal search benchmarks, with models of two billion and four billion parameters. It is trained under several reinforcement learning objectives and under supervised fine-tuning, and across those settings the efficiency gains arrive without a loss in task performance. Its average success rate is competitive with leading search agents of the same scale. Per-turn reasoning latency falls by over ninety percent. Total model inference latency per question falls by between twenty-eight and fifty-four percent.

Keep the two latency figures apart when you hear them. One measures the reasoning step alone, the other the model's whole inference for a question, and the abstract reports them separately. What SSR asks a small agent to give up is the freedom to compose its reasoning from scratch, and on these benchmarks the agent kept its success rate without that freedom.