LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators

LLM-to-Flow Distillation. Teaching Natural Language Goals to Scientific Generators. By Riccardo De Santi and eight others, from ETH Zurich, the ETH AI Center, and four other institutions. Published on the twelfth of October. Its code repository is worked out: about two hundred source files, with a readme.

Generative discovery in science needs two capabilities joined together: high-level scientific reasoning, and the ability to generate samples that respect the structure of a design space. Today they remain largely separated. Foundation language models encode broad chemical and biological knowledge. Domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries. One could align such a generator to a goal through an explicit proxy reward, but in experimental sciences that reward is often unavailable, or poorly captured by learned predictors. The knowledge of what a good peptide or molecule might be sits in one kind of model. The ability to produce something structurally valid sits in another. The question is how to connect them when there is no score to optimize in between.

LLM-to-Flow Distillation, or LFD, aligns a scientific generator to a high-level natural-language discovery goal without requiring a proxy reward. A frozen language model acts as an adaptive teacher for the domain-specific generator. The teacher is never trained. Its role is to judge, and to decide what to ask for next, while all of the learning happens in the generator.

Take a goal such as generating peptides with promising anticancer activity. The teacher does not demand that outright. It constructs a curriculum of intermediate sub-goals that is aware of what the flow can currently do, and progressively directs those sub-goals toward the original discovery goal. In each round, the language model judges sampled candidates under the current objective.

Those judgments induce preferences, and the preferences are iteratively distilled into the generator through DPO. The curriculum is what keeps the signal alive. Even when the initial model places negligible probability on designs that satisfy the final goal, the procedure still provides dense learning signals, which lets the generator adapt toward regions that are poorly covered and out of distribution.

Experiments cover two domains: therapeutic peptides, and small-molecule structure design. The authors present the outcome as a new form of generative control: aligning arbitrary domain-specific scientific generators to discovery tasks expressed as high-level natural-language goals.

The design is a division of labour. The language model contributes judgment and a sense of direction. The flow keeps its learned priors over what a valid design looks like. Preference distillation is the channel between them, and the curriculum is what lets that channel carry signal from the very start, when the final goal is still almost out of reach.