AdaPRA: Process Reward Agents with Adaptive Rubrics for Data Science

AdaPRA. Process Reward Agents with Adaptive Rubrics for Data Science. By Atharva Naik and seven others, from Carnegie Mellon University and Microsoft. Published on the twenty-third of October.

Data science agents can already automate machine learning workflows from end to end. On human leaderboards, though, they remain far from competitive. The authors locate the shortfall in the process rather than the final answer. Agents explore too little. They validate unsoundly. They write coding bugs. And they spend a fixed budget poorly, re-running the same exploratory data analysis and the same baseline models on every attempt, as if each attempt were the first.

AdaPRA answers both halves of that list. The first part is a persistent coding agent harness that accumulates knowledge across attempts on the same task, so a later attempt does not start from nothing. The second part is rubric-guided process supervision, which identifies errors in the agent's work and corrects them.

The supervision is carried out by a process reward agent. It audits the working agent's trajectory and its workspace by actually executing checks, not only by reading. Its rubrics are not fixed either: they adapt to each task. One finding shapes how the rubrics are built. Rubrics seeded from easier tasks generalize to harder ones, and adapting them per task improves performance further.

With five attempts per task, AdaPRA raises the leaderboard percentile of an eighty billion parameter Qwen model by twenty-four point five points on DSPredict-Easy. On DSPredict-Hard the gain is fourteen point eight points. Process errors go down at the same time. Rubrics mined from one base model's runs also improve a different model, DeepSeek-V3.2, when used as supervision. And AdaPRA beats DataPRM, which the authors describe as a competitive process reward model for data science.

What carries over here is the rubric. Knowledge gathered while supervising one model, or seeded on easy tasks, still helps on another model and on harder tasks. Memory across attempts and an auditor that runs its own checks together target the process errors that kept these agents off the top of the leaderboard.