TARSphere: A Robot-Oriented Full-Stack Framework for Long-Horizon Embodied Agents in Evolving Physical and Social Environments

TARSphere. A Robot-Oriented Full-Stack Framework for Long-Horizon Embodied Agents in Evolving Physical and Social Environments. By Yicheng Xu and seven others, from Zhejiang University and Fudan University. Published on the twenty-first of October. Its code repository is worked out: about two thousand source files, with a readme and a licence.

A robot has to act under physical constraints and social ones at the same time, and adapt as its surroundings and the people around it change what they need. Existing embodied-agent systems tend either to focus on goals specified in advance, or to abstract execution away into symbolic actions. Both limit how well an agent can be evaluated in a world that keeps evolving.

TARSphere is a full-stack framework aimed at that kind of evaluation: long tasks, grounded both physically and socially. Its target is a frozen multimodal large language model acting as the robot's brain. The questions are how well it perceives robustly, adapts to disruption, handles a task that gets updated partway through, coordinates with humans, and understands their intent.

There are three layers. TARSphere-Infra runs robot policies in a physics simulator populated with embodied NPCs and constrained dialogue. An event-driven finite-state machine tracks completed actions and delivered information to update task states and judge outcomes. TARSphere-Harness lets the frozen model act through executable skills at multiple levels, using observations, hierarchical memory and execution feedback to revise its decisions.

On top of those, agent-in-the-loop curation aligns task meaning, causal dependencies and scene geometry to build TARSphere-Bench. It holds three hundred and sixty episodes across thirty indoor scenes. Five paired conditions probe the abilities listed earlier, on two kinds of task: navigate and report, and fetch and deliver.

Eight MLLMs were evaluated. General-purpose models can outperform embodied models. Task updates and human interaction remain difficult. And the agents collide and fall often, even when every model shares the same action interface and the same decision budget. That last result is the reason the authors argue execution safety should be evaluated alongside task success.

What the framework adds is a world that moves while the agent works in it. Goals change, people intervene, and the robot's own body can fail by falling over. Under those conditions, the evaluation finds general-purpose models holding up better than embodied ones, and finds that finishing the task is not the only thing a robot benchmark should be scoring.