From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents

From Failures to Learnable Worlds. Policy-Conditioned Environment Generation for Navigation Agents. By Yue Zhang and five others, from UNC Chapel Hill. Published on the fourteenth of October. Its code repository is public but slight: seven source files.

Simulated environments let training data for embodied agents be generated at scale. But more data is not necessarily the data a particular agent needs. Existing pipelines for generating environments mostly optimize generic properties: realism, diversity, coverage of tasks. Meanwhile, different pretrained policies can fail for different environmental reasons. A pipeline that serves every agent the same broad mix ignores the reasons this agent, in particular, is failing.

WEDGE is a framework for weakness-guided navigation learning. Its central move is to let the failure modes of a pretrained agent decide what training environments get generated, and to do that again at every round of adaptation. The environments are conditioned on the policy, not on generic goals. Two policies that fail for different reasons would, under WEDGE, be handed different worlds to train in. And because the diagnosis is repeated each round, the worlds can change as the policy does.

The process starts from a pretrained policy and its rollouts, some successful and some failed. An LLM contrasts the two and proposes weakness features that are executable. Those proposals are automatically validated on a diagnostic set, and WEDGE selects the strongest validated weakness, judged by the gap in success rate it reveals.

Each validated weakness is then mapped to the environmental factors that control it. WEDGE varies those factors to build environments in the difficulty regimes where the policy struggles. The output is targeted training experience, used to adapt the policy. The chain, then, runs from rollouts, to a proposed weakness, to a validated one, to the factors behind it, to new environments. Each link turns a vague sense that the agent fails into something the generator can act on.

The framework is tested with three navigation agents, each with a different adaptation mechanism. The first finding is that the policies have distinct weakness profiles and benefit from different training environments, which is the premise of the method showing up in the data.

On ProcTHOR, weakness-conditioned adaptation consistently outperforms matched alternatives that use random examples or hard examples. For Uni-NaVid and Gemini, the gains from adapting on ProcTHOR also carry over to HM3D. Repeating the diagnosis shows something further: as a policy improves, its dominant weakness can persist, or it can shift to something else, and the new diagnosis gives updated guidance for building the next environments.

The reframing is in the title. A failure is usually read as a score, something that tells you how far a policy has to go. WEDGE reads it as an instruction: a description of the world the policy should practise in next. Because the weakness can move as the policy learns, the instruction is rewritten round by round, so the training environments keep aiming at whatever the policy currently gets wrong.