AnyStep-WAM: Budget-Aligned Distillation and Adaptive Inference for World Action Models

AnyStep-WAM. Budget-Aligned Distillation and Adaptive Inference for World Action Models. By Rui Wang and six others. Published on the twenty-seventh of September. Its code repository is a placeholder: a readme, and nothing to run yet.

World-action models couple predictive visual modeling with action generation, and they typically generate by iterative denoising over a fixed number of steps. Manipulation tasks are not uniform, though. They are made of action chunks that differ in how sensitive they are to generation errors. Critical actions require precision. Less sensitive ones would allow faster generation, with fewer denoising steps. A fixed step count spends the same effort on both.

The AnyStep World Action Model is a general framework for two things: prediction under a budget that can be tuned, and allocation of computation that depends on the scene.

The first ingredient is budget-aligned teacher-trajectory distillation. It trains interval-conditioned flow maps using explicit transitions from a frozen teacher, together with shared low-rank adapters. The result supports action generation anywhere from one-step prediction to refinement over many steps.

The second ingredient builds on that range. A lightweight risk-benefit scheduler looks at a single one-step preview and predicts two things: how difficult the moment is, based on the curvature of the teacher's trajectory, and how faithfully the student would track the teacher at each budget. It then selects the smallest budget predicted to satisfy fidelity requirements that adapt to the risk.

The framework is applied to three widely used world-action models, Motus, FastWAM, and LingBotVA, on RoboTwin two point zero. Average denoising steps fall by sixty point two percent, forty-nine point eight percent, and eighty-five point two eight percent respectively, while baseline task success rates are maintained.

Under a one-step denoising budget, the AnyStep training on its own substantially improves performance. Task success rises by seven point zero seven percent on Motus, twelve point zero eight percent on FastWAM, and eight point nine four percent on LingBotVA. Six real-world manipulation tasks provide further validation.

The underlying observation is easy to state: not every moment of a manipulation needs the same amount of computation. AnyStep turns the step count into a decision taken per scene, from a one-step glance, instead of a constant fixed in advance.