Blind Variation, Selective Retention: Improving Creative Idea Generation in AI Scientists

Blind Variation, Selective Retention. Improving Creative Idea Generation in AI Scientists. By Guowei Xu and five others, from Stanford University. Published on the ninth of November.

Discovery needs ideas that are original, useful, and not obvious. AI scientists, though, are usually handed as much task information as possible at the moment they generate ideas. The paper's concern is that all that context anchors generation toward familiar approaches. Telling the proposer everything about the task can pull its proposals back toward the familiar. The generous briefing, meant to help, ends up working against two of the three qualities discovery asks for: originality, and being non-obvious.

The answer borrows from a cognitive framework called Blind Variation and Selective Retention, or BVSR. Information plays two roles in discovery, and the framework separates them. During variation, proposals are generated under deliberately limited task information, to broaden exploration. During retention, the system learns from proposals previously verified on the same task, together with their outcomes, and uses that to keep the promising candidates for costly empirical evaluation.

Put plainly, the blindfold goes on for the generating and comes off for the choosing. The proposer is starved of task detail so it wanders further. The selector is fed the task's accumulated empirical record so it can tell which of those wanderings deserve an expensive test. Information is not thrown away in this design. It is moved. What the proposer is denied, the selector receives, in the form of proposals already verified on this task and the outcomes they produced. The costly step, empirical evaluation, is then spent only on the candidates that survive retention.

BVSR was evaluated across five proposer models and twelve scientific tasks. Limiting task information substantially raised proposal originality and non-obviousness, and produced a larger share of highly creative, high-utility proposals than existing state-of-the-art prompting-based baselines. Learning from earlier outcomes on the task then improved selection: half of the candidates could be discarded while most of the top-performing proposals were kept.

Dropped into existing AI-scientist workflows, BVSR achieved stronger discoveries across five domains: gene expression analysis, drug response prediction, anomaly detection, scaling law discovery, and brain activity prediction.

More information is not uniformly better for an AI scientist. That is the lesson the authors draw. Which information helps depends on the stage. While exploring, task-specific detail should be constrained. While selecting, the accumulated experience on that task should be exploited. One system, two stages, and opposite policies toward what it is allowed to know. The name of the framework already says it: variation that is blind on purpose, and retention that is selective because it has seen what worked.