Miles v0.1 Turns Frontier Post-Training into Deployable Infrastructure

Introduction

The difficult part of reinforcement-learning post-training is rarely just the optimization algorithm. A practical system must generate rollouts reliably, feed them into training, move updated weights between services, and continue operating across different hardware topologies. RadixArk’s Miles v0.1 addresses this broader engineering problem with an open-source, full-stack system for frontier-scale post-training.

Rather than presenting post-training as a collection of research scripts, Miles organizes the loop around a set of components that are intended to be clean, verifiable, and customizable. That design goal is important because large-scale RL systems often fail at the boundaries between rollout generation, optimization, and deployment.

Key points

• An end-to-end loop. Miles covers the main stages of the post-training pipeline. Its rollout engines are built on SGLang, while the trainer can use either NVIDIA Megatron-LM or PyTorch FSDP.
• Multiple synchronization choices. The project provides three weight-synchronization transports for different deployment topologies. This gives users room to adapt the system to their cluster layout instead of relying on one fixed communication pattern.
• Several post-training modes. In addition to full-parameter RL, Miles supports LoRA RL, on-policy distillation, supervised fine-tuning, and a true on-policy configuration intended to keep rollout and training behavior aligned.
• Beyond language models. The same architectural approach is reported to extend to diffusion models, suggesting that the project is targeting reusable post-training infrastructure rather than one narrow algorithmic recipe.

The reported case study

The report describes a fully asynchronous agentic-RL run on terminal-use coding tasks. The system trained a GLM-5.2 744B-A40B model on 64 NVIDIA GB300 GPUs, with a median step time of 263 seconds across the first 30 measured steps. The most notable aspect of this example is its systems perspective: rollout generation and model training can proceed asynchronously while supporting a very large model and tool-using tasks.

The number should be read in context. The supplied material does not include a comparison against another framework, a cost analysis, or a measured improvement in coding-task quality. Therefore, the case study demonstrates an operating configuration, but it does not by itself establish that Miles is universally faster or more effective than competing approaches.

Why it matters

Miles makes several production concerns explicit: whether each component can be verified, whether interfaces remain clean as the system grows, and whether core backends and synchronization methods can be replaced. For researchers, that modularity may reduce the amount of infrastructure work required when an experiment changes. For enterprises, it could provide a more direct path from a post-training prototype to a continuously operated training service.

The open-source release will still need to be judged through its documentation, tests, hardware compatibility, and community activity. Based on the available material, Miles is best understood as a promising infrastructure option for frontier post-training, not as a conclusively validated universal solution.

Hugging Face Daily Papers