SuperNav: An Agentic Navigation System for Any Task in Any Scene
Jinkai Zhang, Jingyi Xu, Yuanhong Yu, Jiarui Guo, Ruizhen Hu, Hujun Bao, Xiaowei Zhou, Sida Peng
Zhejiang University · Shenzhen University · Causa Robotics
Published October 8, 2026 · on the feed October 9, 2026 · ▲ 75 · arXiv 2610.12126
The abstract
General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general-purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation-specific fine-tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent-oriented Tools for physical interaction, and task-progress and context management. A unified visual-point interface connects decision-making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance-level, multi-object, and demand-driven tasks. Category-level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/
The Brief
2:31A short podcast summary of the paper. Written and voiced by AI.
Transcript
SuperNav. An Agentic Navigation System for Any Task in Any Scene. By Jinkai Zhang and seven others, from Zhejiang University, Shenzhen University, and one other institution. Published on the eighth of October.
A general-purpose service robot needs to navigate in response to all kinds of human requests, in places it has never seen. That asks for two kinds of generality together: across tasks and across scenes. Some existing methods fine-tune multimodal large language models to predict navigation actions directly. Their behavior then depends on how much the navigation training data covers, which may limit how well they generalize to new requests and new environments. A request phrased in a way the training data never covered, or a building laid out unlike any seen before, is exactly where such a model is least certain. The general knowledge the model started with can also be pushed aside by the fine-tuning meant to make it a navigator.
The key insight is a division of labor. Let the multimodal model do what it is good at: interpreting requests, understanding scenes, and making decisions, while keeping its general-purpose abilities intact. Hand the actual motion to navigation tools. SuperNav realizes this by equipping a pretrained model with a specialized agent harness, with no navigation-specific fine-tuning of the model.
The harness supports the model's decisions in three ways. It provides Navigation Skills. It provides agent-oriented Tools for physical interaction. And it manages task progress and context. Joining decision to motion is a unified visual-point interface. Through it, the model specifies destinations directly in images, and revises its decisions based on feedback from execution. Pointing at a spot in an image is a natural output for a model that already understands images, and it hands the tools a clear target to move toward. When execution does not go as planned, the feedback lets the model change its mind instead of pressing on. Together these components are meant to sustain navigation across different task requirements and different environments.
SuperNav outperforms four evaluated baselines on instance-level tasks, multi-object tasks, and demand-driven tasks. A category-level evaluation on HM3D, together with deployment on a real quadruped robot, further supports its applicability across environments.
The bet is that generality comes from not touching the model. Fine-tuning for navigation ties a model to the data it was tuned on. Leaving the model general, and giving it a harness of skills, tools and a way to point at where it wants to go, lets the same system take on new requests and new places, up to a physical robot.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A robot that can do any task, anywhere, is still a wall most navigation systems hit.
You are building a service robot that must handle any request, from finding a specific object to finding a place to rest, in buildings it has never seen.
The road ahead
- 1 · The problem ~7 min A service robot must handle any kind of request in buildings it has never seen, and models trained on navigation data struggle beyond that data.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Keep a pretrained multimodal model for understanding and deciding, let it point at destinations in images, and let navigation tools move the robot.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min SuperNav runs a loop: an unmodified model reads the request and scene, points at a destination, a tool moves, and feedback informs the next decision.
- 6 · The experiment ~6 min You’ll guess: With no navigation-specific fine-tuning, how did SuperNav compare with four evaluated baselines on instance-level, multi-object and demand-driven tasks?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.