AutoSFT Turns SFT Data Design into a Multi-Agent Research Loop

Introduction

In post-training, the training algorithm and compute budget are only part of the equation. The composition and quality of supervised fine-tuning data can determine which capabilities improve and which regress. Yet many automated approaches give an agent only a narrow choice: select rows from a fixed pool or ask a generator for additional examples. Human data engineers operate on a much wider space. They adjust category proportions, remove weak records, repair errors, replace unsuitable samples, and synthesize data for missing capabilities.

AutoSFT asks whether an automated system can perform this broader form of data research.

How the system works

The paper frames SFT as an iterative search over data recipes. The base model, trainer, and budget remain fixed within each comparison, while a multi-agent system proposes and evaluates new recipes.

• A meta-agent coordinates the loop. It separates experimental execution from interpretation and combines proposals from different agents.
• Execution is separated from analysis. One component runs data operations and training, while analysis agents inspect outcomes and suggest changes. This prevents execution logs from consuming the entire reasoning context.
• Independent analysts explore different directions. Rather than relying on one chain of reasoning, the system collects proposals from multiple perspectives.
• A synthesis agent expands the search space. If the existing pool cannot address a capability gap, the system can create new supervised examples instead of merely rearranging available data.
• Every round starts from the base model. This design makes it easier to attribute an observed change to a particular system component rather than to accumulated model state.

The initial single-agent loop exposed two problems. The agent mostly settled on changing category proportions, while execution output filled its context and useful information was discarded during the search. AutoSFT is designed to address both limitations through explicit role separation and coordination.

Reported findings

Adding the components incrementally produced measurable improvements. The executor cut context loss roughly in half. Analysis subagents raised the best IFEval result by 6.5 points, while synthesis added 3.4 points on IFEval and 2.9 points on IFBench. The same loop was also transferred to code tasks: analysis improved MBPP+ by 3.7 points over a single-agent baseline that did not improve.

The broader lesson is that automated post-training is not simply a matter of asking one model to edit a dataset repeatedly. Effective data research requires a loop that can run experiments, interpret evidence, explore alternatives, and fill gaps with new data.

Implications and open questions

AutoSFT points toward automated data engineering as a distinct layer of post-training. Its action space is closer to what human practitioners actually do, and its modular architecture makes the roles of execution, analysis, and synthesis explicit. The transfer to code suggests that the research loop is not limited to one instruction-following benchmark.

At the same time, the available material does not specify the full model configuration, dataset scale, operating cost, detailed action definitions, or broader generalization tests. The findings should therefore be read as evidence for the value of structured multi-agent data research, not as proof of a universally reliable post-training system. Further work will need to test stability across models, tasks, and budgets, as well as the quality and evaluation risks of synthesized data.

Source: Hugging Face Daily Papers