CoDA: Co-Evolution of Data Engines and Action Policies

CoDA. Co-Evolution of Data Engines and Action Policies. By Tianjiao Yu and seven others, from the University of Illinois Urbana-Champaign, the Chinese University of Hong Kong, Shenzhen, and one other institution. Published on the fifteenth of October. No code repository is linked to it.

Generative simulation can automate the construction of robot environments. A physically valid environment, though, is not necessarily a useful one for training. An environment can obey physics perfectly and still teach a policy nothing it needs. The challenge, as the authors put it, has two halves. One is to identify environments that address a particular policy's failures. The other is to find supervision that helps the policy overcome them once it is placed there.

CoDA lets a simulation data engine and an action policy co-evolve. The two share a heterogeneous memory that records environment construction, policy failures and learning progress. Validated diagnoses from a vision-language critic are compiled by that memory into two kinds of program: generation programs that recreate the difficulties, and training programs that help the policy get past them. A failure, in other words, is turned into two things at once: a place to practise, and a way to be taught there.

Supervision gets a component of its own. Text2potential converts those diagnoses into staged potentials for action-chunk value learning. That gives dense supervision for policy updates, and, under standard shaping assumptions, it preserves the task's optimal policy. Measured learning progress then guides further generation and the refinement of programs, which closes the loop.

Across four manipulation task families, CoDA raises out-of-distribution success from five point three percent after supervised fine-tuning to twenty-seven point six percent. That exceeds the strongest closed-loop baseline by nine point five percentage points, with the same generation budget.

On LIBERO-Pro, fifteen rounds of generation and training raise success from twenty-five percent to eighty-four percent under position perturbations. Under instruction perturbations, success climbs from one percent to seventy-six percent.

Environments here are not generated for their own sake. The engine builds what the policy is currently failing at, and the same diagnosis that shapes an environment also shapes the dense signal used to learn inside it. Progress is measured, and the measurement decides what gets built next. Data and policy improve in step, each one reading the other's progress, and the instruction perturbation result, from one percent to seventy-six, shows how far that loop can move a policy over fifteen rounds.