How Seven PhD Students Trained a 7B Model with Hundreds of Agents

Introduction

Training a 7B language model is not simply a matter of launching a script. Data mixtures, distributed systems, failure analysis, evaluation, and repeated experiments normally require several specialized teams. A seven-member doctoral group at the Beijing Zhongguancun Academy says it completed ZGCM-1 from scratch during a single summer and documented much of the process in public.

More than a model release

The project releases more than final weights. It includes training and data-processing code, data recipes, weights from different stages, intermediate checkpoints, and logs. That makes the project useful as an engineering record. Researchers can inspect how capabilities developed, what a change affected, and how the team investigated failures instead of seeing only a final benchmark score.

The team reports that ZGCM-1 is close to Qwen3-8B on several general evaluations and competitive on some mathematics, search, and tool-use tasks. These claims still need to be read in the context of the evaluation setup, contamination controls, and model scale. Results on selected tasks should not be interpreted as a universal advantage over much larger systems.

What the agent team actually did

The researchers divided the project into data, algorithms, training, cluster operations, and evaluation. Their ZGent platform coordinated hundreds of agents that could:

• write cleaning scripts from human-defined quality requirements;
• inspect data distributions and adjust rules or thresholds;
• submit experiments, watch logs, and locate operational failures;
• identify storage and data-transfer bottlenecks;
• reuse validated workflows, debugging experience, and shared decisions;
• run ACE evaluations covering 18 capability classes, 183 items, and 2,503 probes.

This was not an autonomous model laboratory. The team rated 11 categories using an L1-to-L5 autonomy framework. Monitoring and deployment reached relatively high autonomy, while model architecture and learning-algorithm design remained at a lower level. Choosing research questions and making critical design decisions still depended on people.

Why loss curves were not enough

During one run, the loss continued to fall while model capabilities deteriorated. The investigation eventually traced the problem to data sharding and shuffling: the actual input proportions had drifted from the intended recipe. The incident pushed the team to save checkpoints more frequently and to track knowledge, mathematics, coding, and reasoning separately.

The group also combined local and global attention to extend context length, and used methods including Muon and FP8 to improve efficiency and low-precision stability. Its practical findings were not uniformly intuitive. Stricter filtering reduced the amount of supervised fine-tuning data by about 44.9 percent while improving overall evaluation results; too much long-chain reasoning data could hurt instruction following; and agent data could not replace general capabilities training.

Why the project matters

ZGCM-1 shifts attention from a final score to a reproducible development trail. It also offers a grounded example of AI4AI: agents can multiply the execution capacity of a small research group, but they do not yet replace scientific judgment. The team is now exploring models at 400B and 500B scales. The key question is whether this agent-based workflow can remain reliable as the engineering and coordination burden grows.

Source: QbitAI