Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy

Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy. By Zekai Li and ten others, from Galbot. Published on the nineteenth of October. No code repository is linked to it.

The subject is physical recursive self-improvement, physical RSI: how quickly, and how far, an agent acting on the physical world can improve from its own feedback. The authors set out to investigate its mechanisms, its scaling behavior, and its generalization systematically. A single score does not answer those questions. Public benchmark averages can sit close together for systems that improve very differently, and a high score need not come with early or efficient gains.

PhysRSI-eval quantifies physical RSI along five axes: first-attempt performance, efficiency, success, budget, and improvement from failure. Together they describe not only where an agent ends up, but where it started and how it got there. Two agents with the same final success can differ on every other axis: one strong from its first attempt, the other weak at first and then learning a great deal from what went wrong. A single average would treat them as the same system.

Different VLMs act as policies on eighteen manipulation tasks, all under a fixed interaction budget. Observational scaling is then studied from two perspectives: predicting the relative performance of VLM agents from their public capabilities, and forecasting later progress from early physical feedback. Between them, these two perspectives ask how quickly and how far an agent can improve. A final set of workspace interventions analyzes what improvement leaves behind once it has happened.

The analysis motivates what the authors call the Physical Barrel Hypothesis: different capabilities may constrain different parts of improvement. The five-axis metrics also tell apart systems with close public benchmark averages. The interventions show three things. Removing history reduces success in the late rounds. Frozen workspaces remain useful on unseen layouts. And in three cross-task cases, success arrives earlier.

Those interventions give a partial answer to the question in the title. Improvement here is partly tied to the scene and partly portable: history matters within a run, while what accumulates in the workspace carries over to new layouts, and sometimes to new tasks. The authors read this as evidence that scene-specific adaptation and reusable experience can coexist.