CAST Trains Tool-Calling Agents to Critique Actions Before They Fail

Introduction

Tool-using language-model agents are increasingly asked to complete tasks that unfold over many steps. A customer-service agent may need to inspect several purchases before issuing a refund, while a healthcare-oriented agent may have to follow domain-specific policies across a sequence of interactions. In these settings, a single incorrect tool call can be irreversible. The problem is made harder by the fact that the same failure may not appear in every run, making repeated-trial reliability more important than success on one trajectory.

The paper CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents proposes a training framework designed to address this issue. Its central idea is to turn critique into reusable supervision rather than leaving it as a prompt-time inspection step.

How CAST works

CAST starts with trajectories and relatively sparse task outcomes, then seeks to recover supervision at the level of individual actions:

• Trajectory analysis: The framework examines tool calls in the context of the evolving environment and the final task result.
• Structured rationales: It synthesizes explanations of why an action is valid, risky, or inconsistent with the relevant policy, including in partially observable settings.
• Critique-model training: Instead of learning only from a success or failure label for the entire run, a model learns to verify individual actions.
• Policy optimization: The critique model is then used to construct critique-aware training data for improving the policy model.

This design targets a gap between two common approaches. Prompt-based critics can produce useful explanations, but those explanations are not necessarily stable training data. Optimization-based methods, meanwhile, can make use of outcomes without providing a systematic source of detailed verification rationales. CAST attempts to connect the two by making action-level critique part of the learning pipeline.

Reported results

The authors fine-tuned models from the Qwen3 family on dynamic tool-calling benchmarks. According to the paper summary, CAST exceeded GPT-OSS-120B by more than 10 percentage points on the pass^4 measure for Retail tasks. In an out-of-domain Telehealth setting, it delivered an additional 9 percentage-point improvement. These results are presented as evidence that critique-aware training can improve robustness across repeated executions and changing states, rather than merely boosting isolated response quality.

Why it matters—and what remains unclear

The broader implication is that a final task label is often too coarse for training an agent. A failed run does not by itself reveal which action caused the failure, what information was overlooked, or which policy constraint was misunderstood. Action-level rationales could help shift agent training from post-hoc rejection toward pre-execution verification.

The available material does not establish how expensive rationale generation is, how often the critique model makes its own mistakes, or how well the method transfers across larger sets of tools and policies. Those questions will matter in deployment, where critique adds both computation and a possible new source of error.

Source: Hugging Face Daily Papers