Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation

Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation. By Wenxuan Wang and four others, from the Harbin Institute of Technology, the Shanghai AI Laboratory, and three other institutions. Published on the fifth of October. Its code repository is worked out: about one hundred and ten source files, with a readme.

Wrapping an image generator in an agentic harness is an effective way to improve text-to-image results. The harness can use memory, skills, workflow orchestration, result verification, and iterative refinement to keep constructing and revising prompts, and better prompts draw out better images. The catch is where those gains live. They stay outside the diffusion model, and they are only realized while the full harness is running. Take the harness away and the model is exactly as it was.

Diffusion On-Policy Context Distillation, or D-OPCD, aims to move some of that benefit inside. It treats the prompt the agent improved as privileged context, and distills the knowledge encoded in the harness into the weights of the diffusion model. The goal is a model that keeps part of the harness's benefit when it is conditioned on nothing but the user's original query.

The harness used for this is a text-to-image agent equipped with a component the authors also propose, the Auto Skill Evolver, or ASE. Two prompts are in play. The original query is all the model will see at deployment. The agent-improved prompt carries what the harness knows, and it serves as the privileged context during distillation. The aim is that the model, given only the first, retains part of what the second provided.

D-OPCD internalizes harness capabilities into the generator's weights. Across four benchmarks, the average direct-generation score rises from sixty point five two to sixty-five point zero nine. That is the model on its own, without the harness in the loop.

A second result follows from the first. Once that knowledge is absorbed into the weights, the harness can shed skills that have become saturated and resume evolving. A second round of ASE on the updated generator improves on a skill-free harness by a further one point eight three points.

The second result is what turns a one-off transfer into a cycle. Distillation empties the harness of what the model has now learned, which frees the harness to discover something new, which can in turn be distilled. The authors read this as pointing toward text-to-image systems where harness and model keep improving each other through continual co-evolution.