JumpStart Your Policy Learning with Lessons from 160,000 Training Runs

JumpStart Your Policy Learning with Lessons from 160,000 Training Runs. By Nabil Omi and four others, from the University of Washington and Microsoft. Published on the twenty-fifth of September. Its code repository is public and real: about thirty-five source files.

Reliable progress in offline policy learning depends on three habits: careful reporting, well-tuned baselines, and evaluation across diverse conditions. Earlier work had already shown that results can be sensitive to reporting choices, to hyperparameter tuning, and to the properties of a dataset. What had not been done was to investigate those sources of variability together, at a scale large enough to see how they shape the conclusions a field draws. The central claim that comes out of doing it is blunt. At this scale, no algorithm dominates. Among the strongest methods, aggregate performance is often close, but which method leads differs substantially from one environment to the next.

The study covers offline reinforcement learning and imitation learning together, training more than one hundred and sixty thousand policies across one hundred and fourteen datasets. Two further findings sit beside the main one. Proper hyperparameter tuning frequently reshuffles the rankings people perceive between algorithms, so an ordering read off an untuned comparison may not survive tuning. And the composition of a benchmark can itself produce conflicting conclusions: choose a different set of environments, and a different method can look best.

The authors also study how sensitive methods are to their hyperparameters, and how well those settings transfer across environments. From that analysis they identify a simple strategy for deriving strong default configurations. The findings are then turned into a dataset-conditioned recommender, which gives practitioners task-specific suggestions about which algorithm to use, rather than a single universal answer.

The claims are about offline policy learning, meaning offline reinforcement and imitation learning, and about the datasets and environments the study covers. Within that territory the message is not that one method is best, but that a ranking depends on tuning and on which benchmarks are chosen. The recommender is task-specific by design, which is itself a statement of scope: it answers for a given dataset, not for the field in general.

Alongside the analysis comes a release called JumpStart. It holds every trained policy, the per-model scores and hyperparameters, strong baselines across all environments, the training and evaluation code, and an extensible website for retrieving, analysing, and contributing results. A conclusion that shifts with tuning and benchmark choice is hard to check from a table in a paper. Putting every run in the open is how the authors hope to make that checking routine, and to support work beyond what this study itself covers.