Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models

Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models. By Mingyue Cui and four others, from the University of Notre Dame. Published on the thirtieth of September. Its code repository is worked out: about forty source files, with a readme and a licence.

Vision-language-action models, or VLA models, generalize broadly across robotic manipulation tasks. Complex environments ask more of them, though: a policy has to balance task success against unintended contact. One existing answer is a runtime shield, which corrects individual actions as they happen. The catch is that a shield leaves the underlying policy unchanged. The policy keeps proposing what it proposed before, the shield keeps overruling it, and those repeated disagreements can harden into a persistent mismatch between policy and shield, one that blocks progress on the task.

FailBank is a four-stage, self-evolving framework built to convert runtime feedback into persistent policy improvement. Instead of letting each correction vanish once the action is taken, it turns the corrections into training signal, so the feedback ends up changing the policy itself rather than only constraining it in the moment.

During collection, a fixed CBF-based safety module acts as an observe-only teacher. It produces counterfactual corrections, while the policy itself remains in control of what the robot does. Outcome-aware admission comes next. It converts the useful proposals into corrective targets, and it also retains successful actions that were never corrected, keeping them as quiet anchors. Both kinds of example then feed guarded LoRA updates to the policy: targets that pull it toward the corrections, anchors that hold in place what was already working.

Evaluation runs on the VLA-Arena benchmark, across two difficulty levels and two VLA backbones. Against the base policies, FailBank improves the joint operating point of success and cost. Across the two backbones, task success rate rises by eight point five and six point nine percentage points. Over the same pair, policy-induced cumulative cost falls by thirty-five point six percent and by twenty-three point eight percent, respectively.

Against runtime shielding the comparison changes character. FailBank raises task success rate by twenty-five point four percentage points on one backbone and nine point five on the other, while keeping policy-induced cumulative cost comparable. Read together, the two comparisons say that FailBank holds roughly the cost level a shield achieves while completing noticeably more of its tasks.

The argument is about where feedback goes. A shield's corrections are temporary constraints on single actions, and then they are gone. FailBank treats the same signal as supervision, keeping it so that it shapes the policy from then on. The results are offered as evidence that runtime feedback can do that second job, and not only the first.