From One to Many and Back: Category-Aware Training for SWE Agents

Why category awareness matters

Repository-level software engineering is not one homogeneous capability. Fixing a bug, adding a feature, changing tests, or coordinating edits across files can require different reasoning patterns and tool-use behaviors. When these tasks are pooled into one agentic reinforcement-learning process, the overall success rate may improve while particular categories stagnate or regress. The paper One to More, More to One treats this pattern as a category “see-saw” and proposes a training pipeline that first separates capabilities and then brings them back together.

The proposed pipeline

• Organize tasks with executable labels. The authors use executable task construction and SWE Labeler, an evidence-grounded, multi-axis labeling system. This creates more structured training pools and makes it possible to inspect progress beyond a single aggregate score.
• Develop category experts. Category-specific reinforcement learning is used to produce experts. The initial results improve average training success, but do not guarantee uniform improvement at the instance level. This motivates a more explicit mechanism for consolidating successful behavior and selecting tasks according to the current policy.
• Iterate with RRE. Refresh–Repair–Expand alternates long-horizon Agentic-miniRL with three complementary operations. The updated policy refreshes its mastery of previously seen instances, reuses its own verified successful trajectories for Repair SFT, and reselects tasks for further reinforcement learning. In effect, the loop turns success verification into a source of corrective supervision while continuing to explore new or insufficiently mastered cases.
• Distill experts into one student. Label-routed multi-teacher on-policy distillation, or MOPD, routes tasks to relevant teachers and consolidates their behavior into a single deployable model. A ReLU-gated reward extrapolation mechanism keeps only the direction in which a teacher improves over the reference policy, rather than indiscriminately combining all teacher signals.

Reported results and implications

The evaluation covers pooled RL, balanced RL, expert development, and single-model integration. It examines both aggregate and per-category behavior, which is important because a higher average can otherwise conceal the loss of a specific capability. The paper reports 58.04% on Pro-618 and 59.00% on SWE-bench Multilingual for the resulting model.

The work is notable not simply for these benchmark figures, but for its training perspective. Expert training and policy integration do not require an external model to generate solution trajectories or action targets. Instead, the system relies on executable environments, verified outcomes, and successful trajectories produced by the policy itself. That design may reduce dependence on expensive teacher demonstrations, while making the learning loop more closely tied to the agent’s actual execution behavior.

There are also practical questions left for further study. The reliability of category labels, the stability of task routing, and the transfer of category experts across repositories and benchmarks will affect how broadly the approach can be applied. The public release of the model, dataset, SWE Labeler repository, and referenced container images should make such questions easier to investigate. More broadly, the paper suggests that SWE agents should be evaluated not only by how much their average score rises, but also by whether improvement is distributed across the capabilities that matter.

Source: Hugging Face Daily Papers