From Evidence to Action: How Tool-Using Agents Fail

From Evidence to Action. How Tool-Using Agents Fail. By Hongzhan Lin and five others, from the National University of Singapore, Hong Kong Baptist University, and two other institutions. Published on the sixth of October. Its code repository is worked out: about two hundred and fifty source files, with a readme and a licence.

Tool-using agents change external state, and those changes have consequences. A correct outcome, though, does not guarantee that the agent's actions were supported by evidence it had established beforehand. This study traces where the chain from evidence to action breaks, as agents move from deciding whether to act, to executing a single action, to running dependent workflows. The central claim is that failures arise not only from missing information, but from how agents use the evidence they have already established when they decide and act.

Across ten configurations of model and harness, strong judgment about actions in a static setting can sit alongside much weaker execution in an interactive one. Failures often begin before anything is executed. Agents stop with their investigation incomplete, or act before the evidence they needed is in place. Once that evidence is obtained, single actions are usually carried out reliably. Workflows with several actions add their own failures: prerequisites left unresolved, and execution left incomplete.

The instrument for this is SafeActBench: six hundred and fifty-six cases across six operational domains, under five protocols that progress from static judgment and investigated non-action through to single-action and multi-action workflows. A provenance-bound Evidence Ledger, together with a deterministic trajectory evaluator, tracks what information was established, when each action happened, and whether downstream dependencies were satisfied.

The conclusions rest on ten model-harness configurations and on the domains and protocols the benchmark defines. The pattern described is that strong static assessment can coexist with weak execution, not that it always does, and that single-action execution is usually reliable once evidence is in hand, not always.

The useful shift is in where to look. Grading an agent by its final outcome hides whether it acted on solid ground. Following the evidence step by step shows the weak point sits early, in investigation and in the decision to act, more than in carrying the action out.