AI Coding Agents Produce More Code, but Not More Software

AI coding agents are changing how software teams work, but faster code generation does not necessarily mean faster software delivery. A study by Harvard researchers Fiona Chen and James Stratton finds that companies produced substantially more code after adopting these tools, yet there was little measurable improvement in the amount of software completed.

What the study measured

The researchers analyzed aggregated data from Jellyfish covering roughly 300 million engineering work events between 2021 and March 2026. The dataset represented more than 700 software development companies and over 700,000 employees. By combining measured AI adoption with GitHub activity, the study compared key engineering indicators before and after organizations introduced AI coding assistants or more autonomous coding agents.

The main findings were:

• Lines of code increased by an average of 30% after AI coding agents were introduced;
• Total commits rose by 20%, while pull requests increased by 23%;
• The resolution rate for Issues and Epics tracked in tools such as Jira did not change significantly;
• The average time from pull-request submission to merge increased by 49%;
• The share of pull requests requiring changes nearly doubled, and comments per pull request rose by 35%;
• The proportion of workers carrying out code reviews increased by 14%.

Together, the results point to a production mismatch. Agents make it easier to create code, but every additional change still has to be understood, tested, integrated, and approved. When the generated code requires more scrutiny, the extra output at the beginning of the process creates work further down the pipeline rather than increasing the amount of finished software.

Review becomes the bottleneck

Software engineering is not simply a code-writing exercise. A change must fit an existing architecture, satisfy tests and security requirements, and serve a product need before it becomes a useful release. The study suggests that AI agents currently compress the coding phase without removing the coordination and verification work that follows it.

AI-based review has not yet closed that gap. By March 2026, about 80% of the firms measured were using some form of AI code review. However, AI agents accounted for only 23.3% of review comments and 10.8% of pull requests. Humans still performed most of the review work. The researchers also found no significant employment change after AI-agent adoption when comparing active-worker data with LinkedIn information, so the findings do not establish that these tools have broadly replaced software engineers.

What companies should take away

The results do not mean coding agents are useless. They may still be valuable for prototypes, repetitive changes, and narrowly defined implementation tasks. But code volume, commit counts, and pull-request activity are weak proxies for software delivered. Organizations should also track completed functionality, defects, rework, review load, and end-to-end delivery time.

The data ends in March 2026, while agent capabilities and team practices continue to evolve. Better task selection, stronger tests, automated review, and clearer ownership could reduce the current bottleneck. For now, the central lesson is straightforward: AI agents have increased the supply of code, not necessarily the throughput of software production. The competitive advantage will depend less on making agents write more and more on helping the whole delivery process absorb what they produce.

Source: Ars Technica AI