CausGround: Causal Interaction Grounding for Adaptive Embodied Planning

CausGround. Causal Interaction Grounding for Adaptive Embodied Planning. By Yijun Shen and eight others, from East China Normal University, the Beijing Academy of Artificial Intelligence, and three other institutions. Published on the twenty-first of October.

Multimodal large language models show promising reasoning. Reliable physical interaction asks for something more specific: planning that accounts for when an action can be executed and how it changes the world. Those preconditions and effects depend on the environment, and multimodal models often fail to capture them, which makes adapting to a new setting hard. Current approaches fall into two camps. One needs costly training for each environment. The other leans on interaction experience that is unstructured and difficult to reuse. The first is expensive to repeat for every new setting, and the second leaves lessons learned in one episode hard to apply in the next.

CausGround takes a third route. It is a non-parametric framework for causal interaction grounding: it distills interaction experience into causal knowledge centered on actions, in a form that can be reused. Nothing about the underlying model is retrained. What accumulates is structured knowledge about what each action requires and what it does.

The framework identifies the preconditions and effects of actions and organizes them in a factorized representation of the world state, one that captures the attributes of objects and the relations between them. At planning time, the causal dependencies relevant to the task are grounded in the current observations. That supports two directions of reasoning. Backward reasoning over preconditions works out what must be true before an action can run. Forward reasoning over effects anticipates what the action will change. Both feed into choosing the next action and into planning over many steps. Put simply, the planner can ask both what it needs to get done first and what will follow once it acts.

Evaluation spans simulated high-level planning, interaction reasoning, and manipulation with a real robot. On adapted versions of the VirtualHome and RoboCasa benchmarks, CausGround improves vision-based high-level planning across several multimodal backbones. On VirtualHome, average task success rises from twenty-four point nine percent to fifty-three point three percent. On RoboCasa, closed-loop planning success goes from forty-three percent to fifty-seven percent. On VirtualHome, that is more than a doubling of success.

On SWITCH, average interaction reasoning accuracy climbs from twenty-five point eight percent to thirty-one point nine percent, which the authors read as better action selection, recovery and verification. In the real-world experiments, CausGround reasons backward from goals interpreted by the multimodal model to derive executable sequences of high-level actions for sequential manipulation. Here the backward direction does the work, starting from the goal and tracing what has to happen first.

Across simulation, interaction reasoning and a physical robot, the common thread is that the gains arrive without any parameter updates. The authors' conclusion is that knowing what an action needs and what it causes, stored in reusable form, is enough to support long-horizon embodied planning across diverse environments.