Back to articles

AutoMaster: Teaching a Model to Be a Better Teacher

3 min read

The problem: success is not the same as a good demonstration

Agent post-training often treats successful task completion as a signal that a trajectory is worth copying. But a trace can reach the right outcome through unsupported assumptions, unproductive repetition, or superficial verification. If training data is selected only by whether the task succeeded, these habits may be reinforced along with the useful behavior.

AutoMaster asks whether the teacher itself can be made more helpful without changing the underlying model. Its answer is Skill-FT, or Skill-Guided Fine-Tuning: natural-language skills steer a teacher’s rollouts toward desired behaviors, suppressing undesirable habits and drawing out effective behaviors that appear less often. A target model is then fine-tuned on the resulting demonstrations.

An automated improvement loop

AutoMaster uses an LLM agent to examine trajectories, identify behavioral patterns, design and refine skills, and select demonstrations. Feedback from further teacher rollouts and evaluation of the target model informs the next iteration. The process therefore goes beyond filtering for successful examples: it tries to shape how those examples are produced.

The distinction matters. A task-level success label says little about whether the model checked its assumptions, avoided needless repetition, or made its reasoning actionable. Skill guidance offers a way to express preferences about such behavior in natural language and use them to influence the teacher’s demonstrations.

Results and a notable training detail

The reported experiments cover Terminal, Search, and software engineering tasks, using 4B and 9B target models. Compared with Vanilla SFT on successful but unguided teacher trajectories, AutoMaster raises mean performance by 1.42–8.99 percentage points. The paper holds the teacher, task pool, training-set size, and SFT recipe fixed, focusing the comparison on the effect of shaping demonstrations.

The authors also compare whether skill instructions remain in the fine-tuning examples. Keeping them improves results over removing them in all six settings, even though evaluation does not include the instructions and the supervised outputs are identical. The reported advantage is 2.15–4.01 points. This suggests that context surrounding a demonstration may contribute to learning its behavioral pattern, rather than serving only as a temporary prompt for generating the trace.

Why it matters—and what remains open

AutoMaster shifts attention from collecting more successful trajectories to specifying what a good trajectory should look like. That is a useful perspective for multi-step agents, where reliable checking and efficient action can matter as much as the final answer. It also presents a practical route to improving supervision while keeping the teacher and the main training recipe fixed.

The findings are bounded by the tasks, model sizes, and settings reported in the paper; they do not establish that skill guidance will help every agent or that learned behaviors will persist in all deployment conditions. The broader takeaway is narrower but important: success alone is an incomplete measure of demonstration quality, and post-training can benefit from explicitly shaping the behavior that teachers show.

Source: Hugging Face Daily Papers

Comments

Checking sign-in status...

Loading comments...

Related articles