LEASH Uses Uncertainty to Route and Monitor Computer-Use Agents

Introduction

Computer-use agents are moving beyond short interactions toward long-horizon tasks that require many decisions across graphical interfaces and applications. Using a frontier model at every step can improve reliability, but it also makes deployment expensive. Relying only on a small open-weight model has the opposite problem: the agent may be efficient, yet become brittle when the task enters an unfamiliar state or requires a sequence of precise actions.

LEASH proposes a middle path. A weak agent remains in control by default, while a stronger agent is activated when the system detects that the current task or state is becoming risky. The framework also addresses a problem that is often overlooked in escalation designs: after the handoff, the stronger agent may still make a wrong turn, and its API may expose no internal signals for direct inspection.

How the framework works

LEASH organizes uncertainty-aware control into three layers:

• Initial task uncertainty: Before execution begins, the system estimates whether the task is likely to exceed the weak agent’s capabilities. This supports early routing to the stronger model when necessary.
• Step-level uncertainty: During weak-agent execution, LEASH checks the risk of the current state at each step. Its estimator reuses activations already generated while producing an action, so it does not require an additional forward pass or a separately trained monitor.
• Post-handoff oversight: Once the strong agent takes over, the weak model becomes a shadow observer. Because the strong agent is commonly accessed through a black-box API, the observer relies on externally visible behavior and environment changes to detect signs that the agent is losing track of the task.

The same latent representation is used as the basis for all three layers. LEASH compares the weak model’s current hidden state with states collected from previous successful and failed trajectories. A representation that resembles failure cases more closely is treated as a higher-risk signal. This turns the model’s existing internal activations into a lightweight uncertainty estimate instead of adding a dedicated uncertainty network.

Reported results

The authors evaluate LEASH on OSWorld with four contemporary model backbones. According to the paper’s abstract, LEASH improves over random escalation by 6.55 to 11.94 percentage points when the escalation rate is held constant, and it consistently outperforms standalone weak agents. In one configuration using Qwen3.5-35B-A3B, LEASH reaches a 73.66% task success rate at $0.4117 per task. The reported success rate is close to always-on Claude Opus 5 at 72.90%, while inference cost is reduced by 19.2%.

The broader lesson is that model selection can be treated as a runtime risk-management problem rather than a fixed deployment choice. A small model handles the high-volume, lower-risk portion of work; the stronger model is reserved for difficult states. Crucially, escalation is not treated as the end of supervision. The continued observer acknowledges that a capable model can also drift after taking control.

Why it matters

For agent developers, LEASH offers a practical architecture for balancing cost and reliability without requiring a new training pipeline. Its three-stage design also separates three operational questions: whether to start with a strong model, when to escalate during execution, and how to detect trouble after escalation.

There are still important caveats. Similarity between hidden states and historical trajectories may depend on the quality and coverage of the stored successes and failures. The signal may also vary across task distributions. Moreover, a shadow observer watching a black-box model can only infer risk from external behavior; it cannot guarantee that every hidden mistake will be detected. LEASH should therefore be viewed as a cost-aware control and oversight mechanism, not as a complete proof of agent correctness.

Source: Hugging Face Daily Papers