Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station

Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station. By Wenyu Du and one other, from DualverseAI, the University of Hong Kong, and one other institution. Published on the sixth of October.

AI systems have moved quickly in scientific discovery when someone hands them a well-defined metric to push on. Whether they can take on open-ended discovery by themselves, with no such metric, has remained unclear. The claim here is that they can make meaningful progress, given a suitable environment. That environment is Station, an open world in which multiple agents simulate a scientific ecosystem. The authors augment it with two mechanisms aimed at problems specific to open-ended work: a Supervisor mechanism, and periodic Meta Reflection. Both are meant to keep exploration going even when there are no intermediate metrics to show progress.

The test is built from three recent oral papers presented at ICLR. Agents receive the main research question each paper studied, but the paper's results are withheld and web access is disabled. The original findings are partitioned into individual criteria, and the measure is how many of those criteria the agents rediscover on their own.

On average, Station rediscovers sixty-two point seven percent of the criteria. Codex Multiagent-v2 reaches fifteen point four percent, and AI Scientist-v2 lands between fourteen point four and twenty point six percent. Ablation and behavioral analyses point to the two added mechanisms: used together, they improve both the coverage of the research and its continuity. A second test drops the oracle papers entirely. On two open-ended tasks with no published answer to compare against, some of the agents' discoveries closely match discoveries that researchers reported after the knowledge cutoff date.

The boundaries matter here. The rediscovery evaluation rests on three papers from one venue, and success is counted as recovering criteria drawn from findings that already exist, with the results hidden and the web switched off. The tasks without oracle papers number two, and the evidence there is that some discoveries closely match later reports, not that all of them do. The conclusion is also conditional by its own wording: it is about what agents can do inside a suitable environment, with these two mechanisms in place, rather than about agents in general.

Read together, the results move the question from whether agents can discover at all toward what surrounds them while they try. In Station, a supervisor and a habit of periodic reflection are there to keep the agents exploring when nothing tells them they are getting warmer, and the ablations credit the pair, working together, with broader and more continuous research.