DRAP: Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI

DRAP. Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI. By Oswaldo Ludwig. Published on the third of November.

Continual learning for a physical agent is harder than the conventional kind, because errors compound. A small policy error at one step can push the agent outside its training distribution. Out there, the next prediction is worse, the one after that worse again, and the trajectory can end in closed-loop task failure. Forgetting a previous skill is therefore not uniformly costly. Forgetting how to act at a point where small deviations get amplified is far more damaging than forgetting how to act where they do not.

Dynamics-Risk Adaptive Plasticity, or DRAP, builds that asymmetry into how the model is protected. It is replay-free: no old data is revisited. Instead it estimates how important each adaptable model tensor was to past behavior, using sample-level action gradients weighted by the local risk of trajectory amplification. Components that mattered at dynamically sensitive points are the ones it guards.

The risk estimate comes from the expert data, not from the policy. A ridge-regularized local linear model is fitted to expert state-action transitions, and the largest singular value of its state-transition matrix gives a Lyapunov-inspired estimate of how much a one-step perturbation is amplified. Because it is fitted to expert transitions, the estimate is policy-independent.

After each task, risk-weighted squared gradients are added into a tensor-level importance memory. On the tasks that follow, DRAP normalizes this cumulative importance and uses it to scale the effective AdamW learning rate of each tensor. The plasticity of components that supported earlier behavior at sensitive trajectory points is reduced, and reduced immediately.

The method is instantiated on a fixed KappaTune-selected subnetwork of OpenVLA, and evaluated on replay-free sequential adaptation across the LIBERO Spatial, Object, and Goal suites. Against standard AdamW fine-tuning and against EWC, the results show substantially better retention of previously learned behaviors, while acquisition of each new suite stays strong.

DRAP's premise is that in physical AI, where in a trajectory a skill was used matters as much as how often. Weighting importance by local amplification risk turns that premise into one number per tensor: a brake on its learning rate.