One Success Is Not Reliability: Testing Agents with Thinkingbox

Introduction

An agent can produce a plausible answer, call a tool with valid syntax, and still fail to complete the business task. In operational workflows, the agent may need to ask for missing information across several turns, apply domain-specific policies, coordinate dependent tools, and leave a backend system in exactly the required state. A missed step or an unnecessary side effect can invalidate an otherwise convincing run.

Thinkingbox, introduced by the authors of the paper, is designed around this distinction. It is a sandbox and benchmark for tool-agent-user interaction that evaluates the outcome of an entire workflow, not just the quality of the final text or the legality of individual tool calls.

Key points

• Isolated execution. Thinkingbox offers MCP-compatible tool sessions in isolated environments, helping keep runs separate and making workflow experiments more controlled.
• Full interaction traces. The system records the interaction among the user, agent, and tools. This makes it possible to inspect information gathering, intermediate decisions, and execution errors rather than looking only at the final answer.
• Executable outcome checks. Each task has checks tailored to its requirements. They accept valid trajectories while rejecting wrong, missing, or extra effects in the terminal backend state. Selected tasks also impose properties on the final response.
• Business-oriented coverage. Thinkingbox-bench contains 507 policy-conditioned workflows across retail, hospitality, auto insurance, neobank internal IT, and consulting IT/HR support.
• A reliability gap. Across proprietary and open-weight models, the strongest reported result is 65.36% pass@1, while pass^20 is only 25.25%. In other words, finding a successful trajectory once is much easier than reproducing success reliably.

Why it matters

Many existing agent evaluations treat a valid tool call, a useful intermediate action, or clean termination as evidence of progress. Thinkingbox shows why those signals are insufficient. A number of failed trials still terminate cleanly and perform legitimate state-changing actions. They nevertheless fail the task because the complete set of required effects was not achieved, or because an extra effect altered the backend state.

This distinction matters for enterprise automation. A production system cannot be judged by whether it occasionally finds a workable path. It must follow policies, preserve constraints, avoid collateral changes, and deliver the intended persistent outcome consistently. Thinkingbox places these requirements in one evaluation loop, providing a more operationally meaningful way to compare agents.

The benchmark is still a sandbox abstraction, so its scenarios cannot represent every business environment. Its contribution is therefore less about declaring a final ranking than about sharpening the evaluation question. Instead of asking whether an agent said something reasonable or invoked a tool correctly, we should ask whether it changed the underlying system into the intended state, with no missing or unauthorized effects. That is the difference between a successful demonstration and a dependable workflow agent.

Source: Hugging Face Daily Papers