NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime

NavGPT-3. Harnessing Context in a Hierarchical Navigation Runtime. By Gengze Zhou and eleven others, from Adelaide University, Metacognition, and six other institutions. Published on the seventh of October. Its code repository is worked out: about forty-five source files, with a readme and a licence.

Language models trained with long-horizon agentic reinforcement learning have three useful abilities. They can generalize knowledge through reasoning. They can express precise actions. And they can pursue a goal across many steps. Together, that raises the ceiling on what an embodied agent can understand and decide. Physical interaction, though, still belongs to action policies, which supply dense control at low latency. So there are two kinds of model, each strong at a different layer: one thinks slowly and broadly, the other acts quickly and locally. The question is how to connect them.

NavGPT-3 is a harness that connects the two, and on top of the harness sits a runtime that behaves like an operating system. Reasoning, acting and monitoring each run as a thread. Every thread has its own context, its own tools and its own permissions. The runtime schedules those threads, and it decides which one controls the robot's motion at any moment. That arrangement is what lets the robot react to sudden events in the real world: it can interrupt one thread and switch to another, the way an operating system handles an urgent request.

Underneath the harness sits the action policy, NavGPT VLA, trained on nineteen point two eight million examples. It allocates its visual tokens using codec allocation, in proportion to how much the scene changes, so a scene that changes more gets more tokens. On its own, its eight billion parameter model reaches a success rate of seventy-four point five one on R2R-CE. On RxR-CE it leads, with a success rate of seventy-eight point one nine.

With the complete harness, NavGPT-3 sets the state of the art on R2R-CE, at a success rate of eighty-one point five one. On RxR-CE, the authors report that it brings an autonomous agent to human level for the first time. Its success rate there is ninety point four three, against ninety point four for human followers. Path fidelity, measured by nDTW, is seventy-eight point four seven, against seventy-seven point seven for humans.

It also gets there faster than a person. Each episode takes one minute and twenty-two seconds, where a human needs roughly three minutes. The authors ablate the harness design comprehensively, along with the interplay between the two models, to show how the tools and the action policy shape the path from language-model reasoning to physical control.

One result from those ablations stands out. When NavGPT VLA executes the route, the reasoning loop gets shorter. The system's minimum reaction time falls sharply. With the language model deciding, each decision takes between three and nineteen seconds. With the action policy stepping, each step takes between half a second and one second, a rate of one to two hertz.

The authors draw the lesson that the embodied interface itself is central. Frontier language-model intelligence and low-level physical control each run at their own speed. What sits between them, the threads, the scheduling, the ability to interrupt, decides how well the two combine. They plan to release all models, code and evaluation records.