Reinforcing Agentic Creativity in Scientific Ideation with Night Science

Reinforcing Agentic Creativity in Scientific Ideation with Night Science. By Priyanka Kargupta and six others, from the University of Illinois Urbana-Champaign, Microsoft, and one other institution. Published on the twenty-eighth of September. Its code repository is public and real: eleven source files.

Large language models do well on structured, verifiable tasks. For open-ended scientific ideation, their low-entropy bias works against them: outputs tend to be homogeneous and predictable. Discovery covers a wider creative spectrum than that. At one end is structured day science. At the other is loosely structured, serendipitous night science, which reaches ideas beyond the ones usually considered. Models that stay at the predictable end miss the second kind.

AI Night-Scientist is an agentic framework that uses reinforcement learning to teach models when and how to depart from predictable reasoning. Drawing on cognitive science, it models creativity along three axes. Action is what to do and how creatively to do it. Process is when to explore and when to exploit. Outcome is the novelty and usefulness of the idea that results.

Those three axes become the basis for training. Models are trained with GRPO and exposed, over the course of training, to varying degrees and forms of creativity. The point is not simply more randomness. It is creativity of particular kinds, at particular moments, judged by particular results. The action axis concerns the individual moves a model makes while reasoning. The process axis concerns timing: whether, at a given point, to keep pursuing a promising line or to go looking for another. And the outcome axis concerns the idea at the end, which is supposed to be both new and of some use. Night science, in this framing, is not the abandonment of structure but a deliberate, learned departure from it.

The trained models produce substantially more diverse scientific proposals. Compared with the base model, the range of research directions widens by twenty-seven point eight percent and the range of contribution types by fourteen point nine percent. Predicted citation impact improves by up to thirty-two percentage points, and originality by sixty-six point two points.

A natural objection is that the same diversity could be had by turning up the decoding temperature. The authors test it, and the gains cannot be reproduced that way. What matters instead is semantic guidance that specifies what kind of creativity to pursue. Randomness alone spreads outputs around. Telling the model which sort of creative move to make is what turns that spread into proposals that are more original and are predicted to be cited more.

Temperature is a single dial, and it failed. The authors' conclusion is that creativity in these models is not a dial at all but a learnable ability that operates at several levels, and that it can be shaped to help researchers reach ideas beyond those language models typically explore.