ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs

ANTE. Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs. By Yi Ge and eight others, from the University of Maryland, College Park and AWS AI Labs. Published on the eleventh of October. No code repository is linked to it.

Large language models have made software engineering agents capable of complex coding tasks. Inside a containerized environment, such an agent receives tool responses or execution results, and chooses its next actions based on what came back. What most of these agents never do is look ahead. Their policies do not anticipate the consequences of an action before taking it.

Two costs follow. The terminal reward barely attributes a failure to any particular earlier step, so the agent learns little about which move went wrong. And an ill-judged edit can corrupt the repository state irreversibly before any signal arrives at all. Recent code world models aim at these problems, but they come in two forms that the authors find limiting. Either they are trained and then frozen before reinforcement learning begins, or they are reduced to an auxiliary next-token loss on the environment's outputs.

ANTE, short for Anticipate, Then Execute, is an agentic reinforcement learning framework built on a simple loop. Before each action, the policy states an internal belief: a prediction of what the action will do. Real execution then settles whether that prediction was right. Those verified predictions become two things: per-step credit for the agent's actions, and a basis for deciding when to trust its own forecast and when to abstain.

The training side is Belief-Anchored Policy Optimization, or BAPO. With a belief stated before every action, BAPO takes the settled predictions and turns them into per-step advantages, anchoring the credit on the policy's own forecasts. Instead of one reward at the end of a long trajectory, each step gets a signal tied to what the agent expected and what actually happened.

The inference side is ANTE-TTO, for test-time optimization. Its job is to turn the trained forecasts into better actions, which is what makes effective test-time scaling possible. It selects among candidate actions without any external scorer. The world model the policy has learned is the thing doing the judging.

On SWE-bench Verified, ANTE raises Qwen3-32B from twenty-five percent to fifty-seven point two percent pass at one. That is six point two percent above a matched GRPO baseline trained without the belief state, which isolates the contribution of the beliefs themselves.

The world model improves along the way. Its accuracy climbs from forty-four percent to sixty-nine percent during reinforcement learning. And the learned foresight is reported to transfer, both across harnesses and to out-of-distribution domains.

The design asks the agent to commit to an expectation before it acts, and then holds it to account. A prediction that execution can check is a source of credit that a sparse terminal reward cannot provide, and the same habit of forecasting becomes a scorer when choosing among actions at inference. The authors frame the goal as a code world model that knows what has happened, what holds now, and what the next action will achieve.