CodeMidas Turns Source Code into Reinforcement Learning Environments for Coding Agents

Introduction

Reinforcement learning can help coding agents become more capable, but its effectiveness depends heavily on the supply of diverse tasks and dependable verifiers. Existing task-construction pipelines often start from issues, commits, or other development artifacts. These sources are useful, yet they cover only a fraction of the behavior already implemented in a software project.

CodeMidas, introduced by the Xiaomi MiMo team, takes a different starting point: the source code itself. The system turns functionality already present in open-source repositories into executable reinforcement-learning environments. In principle, this gives coding-agent training access to a much broader pool of software behaviors.

How the pipeline works

CodeMidas assigns agentic computation to several stages of environment construction rather than treating task generation as a one-shot prompt-writing exercise:

• Functionality exploration: agents inspect and execute repository code to understand what it does and where its behavioral boundaries lie.
• Behavioral specification: the observed behavior is converted into a task description with concrete expectations for inputs, outputs, and side effects.
• Test construction: candidate tests are grounded in execution of the original implementation instead of being generated entirely without runtime evidence.
• Validation and filtering: candidates undergo execution checks and repeated solution rollouts to assess whether they are solvable and whether their tests can reliably distinguish successful implementations.

The central idea is a closed loop between task generation and verification. A task is more useful for RL when it can run in a realistic environment, provides a meaningful success signal, and does not pass merely because of an accidental shortcut.

Scale and reported results

The resulting collection contains 5,545 training tasks from 3,185 open-source codebases. It covers 23 programming languages and 15 technical domains. The researchers used the tasks to train MiMo-V2.5 with GRPO and reported improvements on five diverse benchmarks. The gains include 11.7% on issue repair in DeepSWE, 17% on whole-program construction in ProgramBench, and 8.5% on terminal work in Terminal-Bench v2.1.

Ablation results indicate that increasing the number of high-quality tasks improves performance. Trajectory analysis also suggests a behavioral shift: after RL training, the agent explores codebases more extensively and uses a wider range of self-verification strategies.

Why it matters

CodeMidas frames a repository as more than an object to be patched. Its existing functions can also serve as raw material for constructing learning environments. This could reduce dependence on manually curated requirements and development histories, while broadening coverage across languages, frameworks, and software domains.

There are important caveats. Implemented behavior is not automatically a complete or correct specification. Automatically derived tests may miss edge cases, and repeated rollouts cannot fully replace careful quality control. CodeMidas should therefore be viewed as a scalable way to expand the supply of coding-RL tasks, rather than as a complete solution to verifier reliability.

The broader implication is that future coding-agent training may draw less exclusively from bug reports and more from the observable behavior of real software: its functions, interfaces, dependencies, and execution traces.

Source: Hugging Face Daily Papers