NeoHorse-1 Turns Agent Routing into a Path to Recursive Self-Improvement

Introduction

Recursive self-improvement sounds straightforward in theory: a model observes where it performs poorly and uses that evidence to become better. In practice, the difficult part is building a reliable bridge between capability observations and the next training mixture. NeoHorse-1 explores that bridge through an agent-oriented post-training pipeline, in which routing is not merely an inference optimization but also a source of learning signals.

How the system works

• A heterogeneous model pool handles different demands. For each user turn, the system predicts the capabilities required and selects a service tier. It records the prediction, the routing decision, and the subsequent interaction rather than retaining only the final answer.
• Agent context remains part of the training example. The collected records preserve interleaved reasoning, tool calls, and harness context. Before admission, examples pass structural validation, six-dimensional semantic evaluation, and subscene-level labeling.
• Routing becomes a curriculum signal. Supervised fine-tuning is arranged in three stages. The same progression is extended to routing-guided on-policy distillation, where a teacher supervises responses generated by the student under the selected capability trajectory.
• Evaluation affects what comes next. Capability-guided allocation translates evaluation feedback into the next training mixture. In this loop, the skills the system has learned influence which examples it prioritizes in the following update.

Results and limitations

The evaluation covers 11 benchmarks spanning harness-based agents, tool use, coding, and instruction following. According to the paper, post-training raises the macro-average from 58.94 to 64.87 for the 4B model, and from 65.60 to 69.04 for the 9B model. The resulting 4B model also narrows the aggregate gap with the 9B base model. These numbers support the value of routing-aware post-training, but they do not establish fully autonomous or open-ended self-improvement.

Why it matters

NeoHorse-1 reframes evaluation as an active component of training. Agent performance depends not only on language generation, but also on choosing tools, following a harness, and sustaining multi-step interactions. Capturing those decisions can produce training data that is closer to real agent behavior than isolated question-and-answer pairs.

The approach also exposes important dependencies. Routing errors, weak semantic grading, or noisy subscene labels could send the next training mixture in the wrong direction. Teacher supervision may likewise preserve or amplify existing biases. For now, NeoHorse-1 is best understood as an initial prototype for harness-mediated recursive improvement: a concrete feedback architecture, not a final demonstration of general self-evolving intelligence.

Source: Hugging Face Daily Papers