HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model
Rongxing Ding, Hongyu Qu, Kehong Wu, Xudong Wang, Fan Zhang, Xinguang Xiang, Xiangbo Shu
Nanjing University of Science and Technology · National University of Singapore · Shenyang Institute of Automation, Chinese Academy of Sciences · The Chinese University of Hong Kong
Published October 29, 2026 · on the feed November 2, 2026 · ▲ 24 · arXiv 2610.40373
The abstract
Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. However, existing methods often model short-term dynamics and long-term task objectives separately, while directly incorporating predicted futures into action policies without sufficiently considering their relevance to current action requirements. To address these limitations, we propose HiFOR, a hierarchical foresight framework with action-guided world retrieval. Specifically, Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones, and final goal states through a Hierarchical Dependency Mask (HDM), establishing dependencies across temporal horizons. Building upon these predictions, Structure-Aware Latent World Modeling organizes future latent representations along the horizon, view, and spatial dimensions to facilitate fine-grained retrieval. Furthermore, Action-Guided Factorized World Retrieval adaptively retrieves relevant future information from the structured latent world based on evolving action representations during denoising, providing targeted guidance for action generation. Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks demonstrate the effectiveness of our approach.
The Brief
2:21A short podcast summary of the paper. Written and voiced by AI.
Transcript
HiFOR. Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model. By Rongxing Ding and six others, from Nanjing University of Science and Technology, the National University of Singapore, and two other institutions. Published on the twenty-ninth of October.
Vision-language-action models have shown promise in robotic manipulation by bringing three things into one system: visual perception, language understanding and action generation. More recent work has added future prediction, to better understand environmental dynamics and task progress. Two weaknesses remain. Existing methods tend to model short-term dynamics and long-term task objectives separately, as though the next moment and the final goal were unrelated questions. And they feed predicted futures straight into the action policy, without asking enough whether a given piece of that future is relevant to what the action requires right now.
HiFOR is built to answer both. It is a hierarchical foresight framework, which here means predicting the future at several horizons together rather than one at a time, and it pairs that foresight with action-guided world retrieval. The policy is not simply handed everything that was predicted. It retrieves from it.
There are three parts. Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones along the way, and final goal states. A Hierarchical Dependency Mask, or HDM, ties these together by establishing dependencies across the temporal horizons. On top of those predictions, Structure-Aware Latent World Modeling arranges the future latent representations along three dimensions: horizon, view and space. That arrangement is what makes fine-grained retrieval possible.
The third part is where action enters. Action-Guided Factorized World Retrieval pulls relevant future information out of the structured latent world, and it does so adaptively, based on the action representations as they evolve during denoising. The result is targeted guidance for action generation, with what gets retrieved shifting as the action takes shape.
Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks are reported to demonstrate its effectiveness. No figures accompany that claim, so the evidence on offer is breadth of testing rather than the size of a margin.
Picture two planners. One is handed a pile of forecasts. The other can put a question to them and get back only what bears on its next move. HiFOR is designed to be the second kind, with the action under construction deciding what to look up.
Paper Quest · 7 chapters · about 36 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A robot that can see the future still fumbles. Here is why.
You build a robot arm policy that reads camera views and an instruction such as pick up the red mug and place it on the plate. Predicting the future should help it act.
The road ahead
- 1 · The problem ~6 min Robot policies that predict the future often look either near or far, and many feed every prediction in whether it matters now or not.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Predict near, milestone and goal futures together, store them organized by horizon, view and space, and let the forming action retrieve only what it needs.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Predict three linked horizons through the HDM, arrange them as a structured latent world, then retrieve from it at every denoising step.
- 6 · Field test ~3 min
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.