VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
Yu-Cheng Chou, Shihao Wang, Jin Huang, Chi-Pin Huang, Chiao-An Yang, Xianzhe Fan, Jing Wang, Zhiqi Li and 11 others
NVIDIA · Johns Hopkins University · Hong Kong Polytechnic University · Purdue University · The University of Hong Kong
Published November 5, 2026 · on the feed November 6, 2026 · ▲ 111 · arXiv 2611.08449
The abstract
Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.
The Brief
2:22A short podcast summary of the paper. Written and voiced by AI.
Transcript
VFlash. Fast, Accurate Visual Planning via Flash Primitive Decoding. By Yu-Cheng Chou and eighteen others, from NVIDIA, Johns Hopkins University, and three other institutions. Published on the fifth of November.
Models built for physical tasks like robotics and autonomous driving do not have to think only in words. They can plan visually, with points, bounding boxes and trajectories that mark spatial goals and anticipate motion. The catch lies in how autoregressive vision-language models produce those plans: one coordinate token at a time. A richer plan means more tokens, and every extra token adds cost. A single box is cheap. A long trajectory, spelled out coordinate by coordinate, is not, and the better a planner wants its plans to be, the more it pays for them in decoding time.
VFlash, through what it calls Flash Primitive Decoding, starts from the fact that visual primitives have a known structure. A box is a box; a trajectory is a sequence of points. Rather than spelling each coordinate out in turn, VFlash predicts a primitive's coordinates jointly. A lightweight drafter generates an entire primitive in a single forward pass, with a layout that adapts to the primitive's type, dimensionality and length. That covers both compact boxes and long trajectories of varying length.
One further step matters. Spatial accuracy, the authors argue, does not depend on reproducing the exact token sequence an autoregressive model would have emitted. So VFlash trains on annotated primitives directly, and it accepts any prediction that is structurally valid, without requiring agreement with target tokens. The drafter's output is judged as a plan, not as a transcript. Two predictions can differ token by token and still describe the same useful path; VFlash does not have to pick the one an autoregressive model would have written.
On the evaluated robot trajectory tasks, in both 2D and 3D, VFlash consistently produces lower prediction error than autoregressive decoding. Preliminary single-rollout measurements put generation at roughly twenty-five times faster for 3D point motion forecasting. Downstream, a world action model conditioned on VFlash plans averages seventy point two percent success across four MolmoSpaces task suites. The baseline reaches sixty-two point six percent. So the plans are not only quicker to produce; a world action model conditioned on them outperforms the baseline.
Fast and accurate, as the title puts it, and both come from the same move. Coordinates that belong together are predicted together, and a plan is accepted for being structurally valid rather than for matching tokens one by one.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here it is a planner that spells out every coordinate, one slow token at a time.
You are running a vision-language model as the planner for a robot. It plans visually, with points, boxes and trajectories, writing each one out a coordinate token at a time.
The road ahead
- 1 · The problem ~6 min A robot's visual planner writes boxes and trajectories one coordinate token at a time, so richer plans slow down how often it can update.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Because a box or trajectory has a known layout, a small drafter can predict all its coordinates at once and the result is accepted if structurally valid.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Train a small drafter on labelled primitives, set each block's layout from the request, fill it in one pass, accept valid results, and feed plans to an action model.
- 6 · The experiment ~6 min You’ll guess: In preliminary single-rollout measurements, how much faster was VFlash than autoregressive decoding at generating 3D point motion forecasts?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.