SKT trains AI agents to use skills with verified synthetic data

Introduction

Language-model agents are increasingly expected to do more than produce text. They call tools, follow procedures, reuse prior routines, and coordinate multiple steps across an execution environment. In this setting, “skills” have become a practical way to package reusable procedural knowledge. A skill may encode a tool-use routine, a task-specific workflow, or a repeatable pattern for solving a class of problems.

But a skill library alone does not make an agent competent. The model still has to decide which skill is relevant, when to invoke it, how to combine it with other skills, and how to carry an execution trajectory to completion. The paper “SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation,” submitted to Hugging Face Daily Papers, addresses exactly this gap.

Key points

• The focus is skill use, not skill storage. SKT starts from the observation that current models may fail even when the necessary skills are available. The missing capability is often selection, application, and coordination.
• The pipeline builds skill-grounded tasks. It selects suitable single-skill and multi-skill configurations from a large skill collection, then synthesizes tasks that require those skills to be used meaningfully.
• Verification is central. Generated tasks and trajectories are checked through rule-based and agent-based verification. When problems are found, feedback-guided repair is applied. Only successful trajectories that substantially use every required skill are retained.
• The dataset is sizable. With 2,000 public skills, SKT produces 4,000 task packages and 27,164 verified trajectories.
• SkillEval provides a held-out testbed. Using the same pipeline but a disjoint test pool, the authors construct SkillEval, an executable benchmark for measuring skill-use capability.

Why it matters

The paper reframes a core challenge in agent development. Many systems emphasize expanding the catalog of tools, plugins, and skills. Yet real failures often happen after those resources are already present: the agent chooses the wrong procedure, skips a necessary skill, uses skills in the wrong order, or fails to coordinate them in a multi-step task.

SKT treats this as a data problem. Instead of relying only on hand-written demonstrations, it uses a verified synthesis process to generate supervision at scale. The experiments reported in the abstract show that supervised fine-tuning on SKT trajectories consistently improves skill-use performance across different models, benchmarks, and agent harnesses. Ablations further suggest that the gains depend on high-quality verification, while cross-harness evaluation indicates that the improvements are not tied to a single interface. Scaling experiments also show that broader skill coverage leads to stronger gains.

The broader implication is that agent progress may depend as much on executable, validated experience as on larger skill libraries. For teams building practical agents, SKT points to a more disciplined route: generate tasks grounded in real skills, verify that the trajectories actually use those skills, and train models on the successful execution traces. That approach does not eliminate the need for good skills or robust environments, but it offers a scalable way to teach agents how to put available capabilities to work.

Source: Hugging Face Daily Papers