Look Before Acting: Pre-Action Checks for Safer LLM Agents

Introduction

As large language models move from generating text to operating terminals, editing repositories, and calling external tools, the most serious failures are not necessarily the ones that produce exceptions. A command may execute successfully, or an edit may be applied without a conflict, while the resulting state is still wrong. These silent failures are difficult to detect because the executor reports apparent success.

The arXiv paper Look Before You Leap: Pre-Action Verification for LLM Agents proposes a straightforward safeguard: inspect an action before it takes effect. The verifier is intended to be cheap and deterministic. It does not have to guess when evidence is weak; it can abstain and let the agent revise or retry the action.

Key findings

• Command validation can be layered. The researchers evaluate a static verifier on 9,930 commands covering 482 tools. It catches 95.8% of invalid commands at a 10.0% false-positive rate. Syntax and binary checks are oracle-exact, producing zero false positives while catching half of all errors. Every reported false positive comes from flag validation, whose limits are tied to help-text coverage.
• Edit formats shape failure behavior. Content-anchored formats, such as search-and-replace and diffs, tend to fail explicitly when their target content no longer matches. Location-anchored approaches are less reliable. After a one-line shift, line-number edits corrupt 99.1% of files in the test; function-name edits target the wrong function 12.7% of the time.
• Abstention makes failures recoverable. Under a refuse-when-unsure policy, selective grounding reaches 0.958 recall at a 7.0% false-positive rate. An anchor-and-verify applier records one silent misapplication in 8,320 trials, or 0.01%.

Why it matters

The broader lesson is about where agent oversight should happen. A common workflow is model generation, execution, and then observation of the result. But observing a successful process does not necessarily establish that the intended object was changed or that the action matched the user’s goal. Pre-action verification instead checks whether the conditions for the intended effect are still present before execution begins.

For agent builders, tool APIs should expose more than a binary success signal. They should make targets, preconditions, anchors, and refusal states explicit. Code-editing systems can benefit from content anchors, patch validation, and context checks rather than relying solely on line numbers. Shell tooling can use syntax and executable checks as a low-cost first layer, while treating argument validation as dependent on the available documentation.

This approach is not a complete proof of user intent. A verifier can establish that an action is structurally valid or still points to the expected target, but it may not understand the task’s full purpose. Its practical contribution is more modest and more useful: convert invisible mistakes into visible refusals that can be repaired, instead of allowing plausible but incorrect actions to propagate.

Source: arXiv