X-Planner: Event-Structured Task Planning for Embodied Intelligence
Howard Lu, Shalfun Li, Porter Pan, Cris, Lumen, Cyril, Eric Hu, Lily Li and 24 others
X Square Robot
Published September 21, 2026 · on the feed September 24, 2026 · ▲ 2 · arXiv 2609.25187
The abstract
Task planning bridges high-level instructions and executable behavior in long-horizon manipulation, yet modern Vision-Language-Action (VLA) systems often leave this intermediate structure implicit. Existing chain-of-thought (CoT) planners also tend to rely on coarse task-level annotations or serialize long reasoning traces token by token. We present X-Planner, a planning front-end that addresses both the supervision and representation of embodied reasoning. Our planning data combine Ego, UMI, and teleoperation under a hierarchy granularity with source-dependent annotation depth. Takeover-time annotations and human-designed failures supervise ongoing error recognition. On the model side, a shared VLM backbone exposes two event-structured plan forms: a discrete interface that emits interpretable event states and a latent interface that relays continuous CoT states across staggered Transformer depths through Staircase Decoding. A frozen latent-to-text reconstruction objective provides a semantic anchor for the latent representation. Offline two-step planning evaluation places X-Planner second among four evaluated models on both BERTScore-F1 and a judge-based Overall score. In real-robot experiments, respectively, outperforming the evaluated baselines. These results characterize planning-text quality and downstream execution.
The Brief
2:18A short podcast summary of the paper. Written and voiced by AI.
Transcript
X-Planner. Event-Structured Task Planning for Embodied Intelligence. By Howard Lu and thirty-one others, from X Square Robot. Published on the twenty-first of September. Its code repository is worked out: about two hundred and fifty source files, with a readme and a licence.
Task planning is the bridge between a high-level instruction and executable behavior in long-horizon manipulation. Modern Vision-Language-Action systems often leave that intermediate structure implicit. Existing chain-of-thought planners have problems of their own. They tend to rely on coarse, task-level annotations, or they serialize long reasoning traces one token at a time. The first gives the planner too little to learn from. The second makes every step of reasoning pay the cost of being written out in full.
X-Planner is a planning front-end that addresses both sides of embodied reasoning: how it is supervised, and how it is represented. The plan becomes an explicit, event-structured object rather than something left inside the action model.
On the supervision side, the planning data combine three sources, Ego, UMI and teleoperation, under a hierarchy of granularity, with annotation depth that depends on the source. Takeover-time annotations and failures designed by humans supervise the recognition of errors while a task is under way. On the model side, a shared vision-language backbone exposes two event-structured plan forms. One is discrete, and emits interpretable event states. The other is latent. It relays continuous chain-of-thought states across staggered Transformer depths, through what the authors call Staircase Decoding. A frozen latent-to-text reconstruction objective gives that latent representation a semantic anchor. The latent form avoids writing every thought out as tokens, and the anchor keeps it tied to something that could still be read back as text.
In offline evaluation of two-step planning, X-Planner places second among four evaluated models, both on BERTScore-F1 and on an overall score from a judge. Real-robot experiments are also reported, with X-Planner outperforming the evaluated baselines there.
Taken together, the results are meant to characterize two things: the quality of the planning text, and how execution goes downstream. Underneath is one design choice. The plan is made explicit, either as readable events or as a latent stream anchored to text, where earlier systems had left it implicit.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Long tasks need a plan. Most robot models keep theirs hidden.
You run a robot that maps camera views and an instruction straight to actions. It handles short skills, but a long task needs choosing and ordering subgoals first, and that structure stays hidden inside the action model.
The road ahead
- 1 · The problem ~7 min Robots that turn views and words straight into actions handle short skills, but long tasks need an ordered set of subgoals they never make explicit.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Pull the plan out into a front-end that plans in behavior-sized events, learns to spot errors, and can hand over readable text or compact latent states.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Build layered data, add error labels, ground everything in one vision-language backbone, then emit events or relayed latent states.
- 6 · The experiment ~6 min You’ll guess: In offline two-step planning, where did X-Planner rank among the four evaluated models on BERTScore-F1?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.