Why Anthropic’s AI-assisted AI development alarms safety researchers

Introduction

Frontier AI labs have long used software to improve the software that follows. A new assessment from the safety organization METR offers one of the clearest public looks yet at how far that process may have advanced inside Anthropic. After an 11-week investigation, METR concluded that the company’s models appear to be improving roughly twice as quickly as they were a year earlier. The researchers also argue that the change cannot be explained mainly by increasing training compute. AI assistance itself appears to be becoming a major source of acceleration.

Key findings

• The pace of progress has increased. METR’s benchmark analysis suggests Anthropic achieved about two years of historical progress during the past year. Researchers said they were confident that scaling compute alone did not explain most of the change.
• Internal productivity gains are substantial. In informal Slack polling, employees using Anthropic’s most capable internal model reported being about 12 times as productive as they would have been without AI assistance. This is self-reported feedback, not a standardized experiment across every role, so it should be interpreted cautiously.
• Humans still lead the process. Most of the observed acceleration appears to come from models assisting human researchers rather than operating independently. METR nevertheless found signs that autonomous contributions could become more important, and it did not identify a clear bottleneck that would necessarily stop further acceleration.
• Several advanced models remain unreleased. Anthropic completed training on three internal models during the past six months, with training finishing in July, August-September, and September-October. Their capabilities cannot be independently assessed by the public.
• The public report is heavily limited. Anthropic provided METR with access to nonpublic research, performance information, and employee interviews, but retained the ability to redact the published version. METR said much of the most informative evidence from its roughly 100-page full report was removed, and some data requests were denied.

Why it matters

The report does not show that AI can already improve itself without meaningful human involvement. Its more immediate implication is the emergence of a potentially powerful feedback loop: better models make researchers more productive; faster research may then produce better models sooner. If that loop continues, forecasts based mainly on training budgets and compute growth may become less reliable.

That dynamic also highlights a contradiction in Anthropic’s public posture. The company has positioned itself as a leading voice on AI safety, while CEO Dario Amodei has called for slowing development. At the same time, Anthropic is using AI to speed up the very research pipeline that may bring more capable systems to market faster. The issue is not simply whether labs should use AI internally. It is whether society can evaluate the consequences when the mechanisms of acceleration are opaque and the strongest evidence is inaccessible.

Anthropic argues that redactions protect commercial secrets and prevent the report from becoming a roadmap for competitors to move faster. That concern is understandable, but it leaves the public relying more heavily on METR’s interpretation rather than examining the evidence directly. Anthropic says it plans to publish its own account of AI progress and announce additional safety measures. The next test will be whether those steps provide enough transparency and independent scrutiny to keep pace with the technology itself.

Source: The Verge AI