Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving
Xin Zhou, Zongchuang Zhao, Zhibo Yang, Mingsheng Li, Humen Zhong, Shuai Bai, Du Chu, Ruizhe Chen and 8 others
Qwen Team · Huazhong University of Science and Technology
Published August 31, 2026 · on the feed September 2, 2026 · ▲ 368 · arXiv 2609.00111
The abstract
We present Qwen-Drive-1.0, an initial step towards a vision-language foundation model for autonomous driving. Qwen-Drive-1.0 retains the architecture of the pretrained vision-language model (VLM) and integrates 3D perception, visual question answering, and motion planning within a unified framework. An external bird's-eye-view (BEV) perception head jointly performs 3D object detection, semantic occupancy prediction, and BEV map segmentation. It serves as a probe of the 3D information accessible from the shared representations and provides an explicit, inspectable interface to 3D scene structure. A Planning Expert conditions on shared VLM representations to generate future ego trajectories. A staged training recipe combines driving supervision with general-purpose vision-language data to acquire driving-specific competence while helping preserve broad visual understanding and instruction-following capabilities. Experiments demonstrate strong 3D perception and driving scene understanding while largely preserving general vision-language capability. Comprehensive evaluations across open-loop, pseudo-closed-loop, and closed-loop settings further show highly competitive motion-planning performance.
The Brief
2:08A short podcast summary of the paper. Written and voiced by AI.
Transcript
Qwen-Drive-1.0. An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving. By Xin Zhou and fifteen others, from the Qwen Team and Huazhong University of Science and Technology. Published on the thirty-first of August. Its code repository is worked out: about forty source files, with a readme and a licence.
A driving model has to do several jobs at once. It has to perceive the scene in three dimensions, answer questions about what it sees, and plan where the car goes next. A general vision-language model brings broad visual understanding and the ability to follow instructions, but driving competence has to be added to it, and adding it risks wearing away the general abilities that made the model a sensible place to start.
Qwen-Drive one point zero, which the authors call an initial step towards a vision-language foundation model for autonomous driving, keeps the architecture of the pretrained vision-language model as it is. Around it, a single unified framework brings together 3D perception, visual question answering and motion planning, all drawing on the same shared representations.
Two components hang off that shared core. An external bird's-eye-view perception head jointly performs 3D object detection, semantic occupancy prediction and map segmentation. It also serves as a probe: it shows how much 3D information the shared representations actually hold, and gives an explicit, inspectable interface to the structure of the scene. The second component, a Planning Expert, conditions on the same representations to generate future trajectories for the ego vehicle. Training is staged, mixing driving supervision with general-purpose vision-language data, so the model acquires driving-specific skill while its broader understanding is helped to survive.
Experiments report strong 3D perception and driving scene understanding, with general vision-language capability largely preserved. Motion planning is evaluated in three settings: open-loop, pseudo-closed-loop and closed-loop. Across all three, the reported performance is highly competitive.
The design choice running through it is restraint. Nothing in the base model's architecture is replaced. Perception and planning attach from outside, and the perception head doubles as a window onto what the shared representation knows about the 3D world.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A general model can talk about a road. Driving on it is another matter.
You want one model that sees a driving scene in 3D, answers questions about it and plans where the car goes. A general vision-language model is a sensible start.
The road ahead
- 1 · The problem ~7 min A general vision-language model is a good start for driving, but teaching it driving through text alone leaves gaps in 3D and erodes its general knowledge.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Leave the pretrained model unchanged, hang a 3D perception head and a planner off its shared features, and mix general data into training.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Keep the pretrained model as it is, attach perception and planning modules to its shared features, then train in stages with mixed data.
- 6 · The experiment ~6 min You’ll guess: After all its driving training, what happened to the model's general vision-language capability?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.