Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents

Omni-Decision. Evidence-Ledger Planning for Omni-Modal Agents. By Ming Ma and eight others, from the Institute of Neuroscience, Chinese Academy of Sciences, the University of Chinese Academy of Sciences, and three other institutions. Published on the twenty-fourth of September. No code repository is linked to it.

Omni-modal agents answer questions by gathering evidence from many places: video, audio, web pages, and computation. The authors locate their main bottleneck in planning rather than in perception. Noisy multimodal observations pile up in the conversation history and disrupt later decisions, and multimodal models have limited capacity for planning over many steps in the first place. A controlled experiment backs the diagnosis. Swapping out the planner costs far more performance than swapping out the perception backend. If the agent is going to improve, the planner is where the leverage sits.

Omni-Decision is built on what the authors call evidence-ledger planning. The growing dialogue history is thrown out and replaced by an explicit ledger, and the ledger records three things: what evidence is still missing, what has already been confirmed, and where records conflict. Instead of rereading everything it has ever seen, the planner consults a running account of what it knows, what it still needs, and what does not add up.

Two pieces keep that ledger clean and the planner improving. First, a critic reads each noisy observation and passes only the usable content into the ledger, discarding the rest. That keeps the planner working from a compact context from the start of a task to its end. Second, every run is recorded step by step, as a state, an action, and a verdict. Those recorded trajectories become training data. Supervised fine-tuning, and then reinforcement learning at the level of decisions, are both applied to them to make the planner better.

On OmniGAIA, Omni-Decision reaches a state-of-the-art accuracy of eighty-one point four percent, and it does so at roughly forty-three percent of the cost per question of Gemini three point one Pro. On WorldSense long-video understanding it scores sixty-five percent, level with the strongest end-to-end model.

Picture an investigator who stops rereading the whole case file and keeps a single sheet instead: open questions, settled facts, contradictions. That is the move here. The diagnosis said planning was the weak link, and the response was to change what the planner reads, not how the agent sees. The critic guards what goes onto that sheet, and the training on recorded runs sharpens the one who reads it.