SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving

SafeDriveVLA. Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving. By Shaoyuan Xie and eight others, from UC Irvine, UC San Diego, and two other institutions. Published on the fourteenth of October. Its code repository is worked out: about one hundred and seventy source files, with a readme and a licence.

Vision-Language-Action models, or VLAs, are drawing attention in autonomous driving. They inherit commonsense priors from large language models, and they widen the navigation interface from a fixed set of commands to free-form instructions in natural language. That open-ended interface is also a new vulnerability surface. An inattentive driver may give an instruction that is unsafe in the current scene. A malicious user may give a dangerous one on purpose.

To measure this, the authors present what they describe as the first benchmark suite that evaluates navigation following and conflict awareness together, along with adversarial robustness. Existing VLAs fall short on both counts. SimLingo completes no language-instructed lane change in closed loop, and every model degrades sharply when signals conflict. A driving model has to follow instructions well, and also know when not to.

SafeDriveVLA, presented as the first driving VLA of its kind, makes that judgment explicit. It predicts a navigation-compliance mode with three levels. That prediction separates the reasoning about conflict from the generation of actions, so deciding whether to comply is its own step rather than something buried inside the driving output.

What informs the compliance prediction is navigation-conditioned world model dreaming. The contrast with earlier world models is the point. Those roll out expert actions or planned actions. This one dreams of strictly following the navigation signal, whatever it says, and so shows the model when following is safe. The imagined consequence of obeying becomes the evidence for whether to obey.

On Bench2Drive, SafeDriveVLA reaches a Driving Score of eighty-eight point one six and a Success Rate of sixty-eight point six four percent. Compared with the best SimLingo variant, it lifts closed-loop language following by forty-nine point three percent.

Under unsafe instructions, the same comparison shows collisions falling from sixty to thirty-eight. Video demonstrations are on the project page.

An open instruction channel is only as safe as the model's sense of when to comply. SafeDriveVLA builds that sense out of imagination: it pictures what would happen if the instruction were followed to the letter, and then decides, in a step kept apart from driving itself, how far to go along with it.