Why Tool-Using Agents Fail Between Evidence and Action

Introduction

For an AI agent that can query databases, update records, send requests, or trigger external processes, reaching the correct final state is only part of the safety story. An agent may produce the right outcome by chance while never establishing the evidence that should have supported its decision. In operational settings, that is still a serious failure: the process is not auditable, reproducible, or reliably safe.

A paper featured by Hugging Face Daily Papers examines this evidence-to-action chain in tool-using agents. Rather than asking only whether an action is ultimately correct, the authors ask where the chain breaks: during judgment, investigation, or execution.

Key findings

• Static judgment does not guarantee execution. For three configurations reevaluated on the same V1 cases, static accuracy was at least 95%, while interactive success was no higher than 52%. Knowing that an action is appropriate in an abstract setting does not mean an agent will investigate first and execute it correctly in an environment.
• Many failures begin before the action itself. In V0 episodes, agents stopped before completing the required investigation in 21.7%–62.9% of cases. In V1, 37.0%–66.9% of action attempts occurred before the required evidence had been established.
• Missing evidence does not reliably trigger restraint. In a controlled intervention covering 43 V1 cases, one decisive record was withheld. Even then, agents still acted in 46.5%–53.5% of completed episodes.
• Multi-step tasks add a separate class of problems. Once the required evidence was available, single-action execution was usually dependable. Dependency-constrained workflows, however, exposed unresolved prerequisites and downstream steps that were left incomplete.

Benchmark and evaluation method

SafeActBench contains 656 cases across six operational domains and five protocols. The protocols progress from static action assessment and investigated non-action to single-action and multi-action workflows. This progression makes it possible to distinguish a failure to decide from a failure to investigate or execute.

The researchers also introduce a provenance-bound Evidence Ledger. It records what information was established, when it became available, and when an action occurred. A deterministic trajectory evaluator then checks whether each consequential action was supported by evidence available beforehand and whether downstream dependencies had been satisfied. Because the evaluator does not rely on an LLM judge, the assessment is tied to the recorded trajectory rather than a subjective interpretation of the final answer.

Why it matters

The results suggest that agent evaluation should not stop at final-state accuracy or task completion. A robust test should verify that investigation was complete, evidence was traceable, actions followed evidence in time, and every prerequisite in a multi-step workflow was actually met.

For system builders, this points toward practical safeguards such as evidence ledgers, explicit confirmation gates, and dependency-aware executors. More broadly, the paper shifts attention from whether an agent knows what action is correct to whether it uses established evidence to take that action. As agents gain the ability to change external state, that process-level distinction will become central to trustworthy deployment.

Source: Hugging Face Daily Papers