Training Reasoning Models to Generate Diverse Scientific Ideas

Training Reasoning Models to Generate Diverse Scientific Ideas. By Manya Wadhwa and five others, from New York University, Stony Brook University, and one other institution. Published on the fifteenth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Generating ideas is one of the first steps in a research pipeline, and large language models are increasingly used to automate it. Most of that work optimizes the quality of each idea on its own and overlooks diversity. A model that offers many excellent variations of one idea has not given a researcher many options. The authors study document-grounded ideation. Given prior papers, a model must propose a set of ideas that are each high quality, clearly distinct from one another, and grounded in the input documents.

The proposal is to train with a reinforcement learning objective that scores the set rather than the single idea. In one generation, the model writes K structured idea cards grounded in the source papers. The set is then scored jointly, with each idea's quality weighted by its distance from the ideas written before it. An idea that sits close to an earlier one earns little for its quality; one that is both good and different earns more.

That weighting is the teaching signal. Because the credit for a new idea depends on how far it lies from its predecessors, the model learns to steer each new idea away from what it has already written. Diversity is not a separate objective bolted on beside quality. It is folded into the quality score itself, at the level of the set.

The trained model has eight billion parameters. On the authors' contamination-free benchmark, it achieves higher creative utility, a measure combining quality and diversity, than Claude Opus four point eight and GPT six Astra. In a pilot human study with four experts, its sets were judged as promising as those of Claude Opus four point eight, and contained marginally more unique ideas.

A further test asks whether the ideas can be carried out at all. A coding agent ran proof-of-concept experiments on them, and the model's success at k was comparable to that of Claude Opus four point eight. On that evidence, the added diversity did not cost any executability.

The design choice to notice is where diversity lives. Instead of asking for variety and hoping, the objective makes every idea's reward depend on the ones before it, so the model learns to spread out across a set while each member still has to be good. On the authors' results, an eight billion parameter model trained this way leads on creative utility, matches its comparison on expert judgment, and gives up nothing on whether the ideas run.