Introduction
Calling a large language model at every step of an agent workflow can add substantial latency and expense—even when a step is simply choosing among a few known options. AnyS1 proposes a fast path for those routine decisions: let a smaller model make the choice, and call a stronger System-2 model only when confidence is insufficient. Its central idea is to derive decision points from existing procedural instructions rather than writing custom routing hooks for every environment.
How the framework works
- Mine choices from skills. AnyS1 scans more than 1,000 public, human-authored skills to find grounded decisions with a finite set of options. The aim is to make these instructions a shared source of decision points across different workflows.
- Ground training in execution. The researchers run workflows in isolated containers and collect states tied to actual execution. These are used to build Any-DP, the dataset for training a general-purpose 4B pointer System-1 model. The model is trained with calibration and label smoothing to support both candidate selection and confidence-based routing.
- Compile the routine path, retain a fallback. Once a workflow is compiled, System-1 can resolve routine choices in one forward pass. If its confidence falls below a calibrated threshold, the decision is handed to System-2. This design seeks to reduce expensive calls without requiring the small model to handle every situation.
What the results suggest
Across three unseen benchmarks and different System-2 model families, the paper reports that AnyS1 maintains task reliability while reducing episode cost by 75–91% and model-call time by 63–92% compared with an uncompiled agent that relies on System-2. It also reports that performance holds under zero-shot transfer of the confidence threshold. These results suggest that some workflow decisions can be compiled into a fast selection step, but they are experimental findings, not a guarantee of equal savings in every deployment.
The practical boundary is candidate coverage. A skill document may describe the expected options, while a live tool can return a state that was not anticipated. In such cases, the system may need to fall back; if a wrong choice does not trigger a fallback, it could affect the rest of the task. Production evaluations should therefore measure how often real trajectory decisions are genuinely enumerable, what happens after an incorrect pick, and how much latency fallback adds. AnyS1 offers a promising general approach to fast/slow model collaboration; whether it is dependable in practice will hinge on how these edge cases are handled.
Source: Hugging Face Daily Papers
Comments
Checking sign-in status...
Loading comments...