Back to articles

System 1 Agents replace free-form tool calls with choices

3 min read

Why rethink tool calls?

Clinical agents may need to route requests to databases and analysis tools, then supply the exact arguments those tools require. Sending patient data to a cloud language model raises privacy concerns, while running a model locally does not by itself prevent hallucinated values. The paper “System 1 agents: a method for local, reliable and fast agentic AI” proposes changing the task: instead of asking a model to write a tool call, have it choose from structured options.

From schemas to decisions

A deterministic compiler combines each tool’s schema with the user’s request and turns them into a series of multiple-choice questions. Candidate answers are drawn from the request and the tool definition. A one-pass decision model then selects a tool and its argument values, or chooses an option indicating that no candidate fits. Since the model selects among supplied values rather than composing arbitrary argument text, the design aims to reduce invented content in tool calls.

The authors also describe what happens when the correct answer is absent from the candidates. Tool selection includes a “none of these” option, while each argument question includes “not specified.” If a required argument is missing, the agent leaves it empty and asks the user through a fixed prompt derived from the schema. For an optional argument, it retains the tool’s default. If no tool is suitable, the agent declines. These escape routes matter: constrained choice is useful only if the system can avoid forcing a bad match.

What the evaluation found

The study tested 1,952 requests involving 157 biomedical tools. The best System 1 agent selected the correct tool or correctly declined for 97.0% of requests. That was higher than the best result, 95.2%, among the 18 local LLM-agent settings evaluated. For 6,056 argument values explicitly stated in requests, System 1 filled 97.0% correctly. The paper also reports that none of its System 1 calls contained invented content, compared with invented content in 5.5% to 19.7% of the LLM calls.

These are results on the study’s test tasks, not proof of performance across clinical practice. The reported figures do not establish that every request can be handled, or that every tool schema will produce suitable candidate options. The system’s behavior still depends on how requests and tool definitions are represented, and on whether the options capture the information needed for a decision.

Why it matters—and what remains open

The contribution is not simply local inference. It is the decision to narrow tool use from open-ended text generation to a sequence of structured, inspectable choices. In sensitive workflows, a tool argument grounded in a request or schema may be preferable to a fluent but unsupported guess. Explicitly asking for missing required information, preserving optional defaults, and declining unsuitable requests are also safer than silently inventing a value.

The method is a promising design for reducing one specific source of agent failure: fabricated tool-call content. Its benchmark results should nevertheless be followed by evaluation in broader and more realistic workflows before drawing conclusions about clinical readiness. The System 1 Agents framework is available as open-source software.

Source: Hugging Face Daily Papers

Comments

Checking sign-in status...

Loading comments...

Related articles