Heartbeat Thinking: Speculative Reasoning and Tool Execution While Listening in Realtime Voice Agents

Heartbeat Thinking. Speculative Reasoning and Tool Execution While Listening in Realtime Voice Agents. By Zhenyu Zhang and six others, from Ant International. Published on the nineteenth of October. No code repository is linked to it.

Tool-using voice agents often leave users waiting after their speech ends, even when the actionable information arrived earlier in the request. Speculative approaches try to close that wait by starting work while the user is still speaking. The trouble is how they handle new speech. Chunk-bounded generation, or generation that is interrupted whenever the input updates, can cut a tool call off before its arguments are complete. That ties computation time to the rhythm of speech updates, and limits how much useful work gets done in advance.

Heartbeat Thinking changes what happens when input updates. Instead of interrupting work in progress, it handles the updated input through new Thinker invocations, while ongoing invocations, and any tools already dispatched, are allowed to finish. New speech no longer cancels old work. It simply prompts more work, running alongside whatever is already under way, so a tool call that started on the first half of a sentence can still complete with its arguments intact.

The invocations are periodic, which is where the heartbeat comes in. Each one reads two things: the latest partial transcript, and a shared record of reasoning already completed and tool results already returned. Those results are what carry the work forward. They help resolve missing arguments and kick off dependent calls, so preparation keeps progressing across speech updates instead of restarting at each one.

The evaluation covers SpeechFCEval, FDBench v3, and AudioCRAG, against a SHANKS-style windowed baseline. Heartbeat improves aggregate speculative tool coverage or retrieval reuse over that baseline, while its effects on final task quality depend on the benchmark. On FDBench v3 specifically, it raises tool coverage by sixteen percentage points, and cuts mean final-response latency by thirty-six point four percent. The gains in preparation are the consistent part of the picture. Whether that preparation also lifts the quality of the final answer varies from one benchmark to the next.

The principle on test is preservation. When computation that has already started is allowed to run to completion during speech, the time a user spends talking stops being idle time, and becomes reusable preparation for a faster and more capable response once they are done.