Outside auditors: Anthropic's AI is improving twice as fast, with AI's help

What's happening

On November 5, 2026, METR (Model Evaluation and Threat Research), a donation-funded organization based in Berkeley, published a report on Anthropic. The question behind it: is AI now speeding up the development of AI? To find out, METR analyzed internal data from Claude's developers and interviewed staff from August 12 to October 29, 2026. METR calls the access "unprecedented."

The core finding: as of October 2026, Anthropic is likely improving its models on well-defined tasks about twice as fast as it did between March 2024 and October 2025. In METR's assessment, most of that acceleration probably comes from Anthropic itself using a lot of AI in its research. According to the Associated Press, the report says Anthropic made as much benchmark progress over the past year as it did between October 2023 and October 2025.

How the audit came about

The timing is notable. On September 12, Anthropic CEO Dario Amodei called in an essay for the industry to slow down on new capabilities so safety research can keep up. He also advocated letting independent auditors work directly inside leading AI labs and specifically named METR as a good fit. According to CNBC, Anthropic announced its collaboration with METR and SecureBio in mid-October. Talks with other organizations such as Hugging Face are ongoing.

Anthropic is building a three-tier system for this. Tier-one auditors can run narrowly scoped risk investigations. Tier two can publish reports without Anthropic's sign-off. Tier three gets employee-like access. According to CNBC, METR was classified as a tier-three auditor: six METR staffers received access to hardware, offices and tools "roughly comparable" to Anthropic's own risk teams.

The report itself paints a more limited picture. According to Unite.AI and METR, the team didn't have unfiltered, employee-level access and couldn't sample unreleased models. Anthropic could approve or deny requests and unilaterally redact non-public content. The work was done on Anthropic devices, using tokens provided free of charge. METR says it didn't accept any money and kept control over its methodology and conclusions. The project ran with 2.5 full-time staff. It analyzed:

• internal benchmark results,
• metrics from pretraining (the base training on huge amounts of text) and reinforcement learning (training by rewarding good solutions),
• training compute data,
• internal analyses of AI acceleration and internal documents,
• around 15 interviews with key staff.

The numbers

The yardstick was AECI, Anthropic's version of the Epoch Capabilities Index. The index combines many benchmarks into a single capability score. Models used only internally were included in the calculation. Each model version was dated to the point when most Anthropic employees could use it. That way, released and unreleased models can be compared on the same timeline.

The confidence interval is the range in which the true value lies with high statistical certainty. The robustness checks rerun the same question with different assumptions, for example about how heavily individual benchmarks are weighted. The result varied between 1.1x and 2.9x. So the 1.9 figure isn't set in stone.

METR says the staff interviews are consistent with a doubling, but the answers varied widely. METR treats the roughly 12x productivity gain from informal surveys with caution: the surveys took little effort, may not be representative, and when individuals work faster, the capability of the whole organization doesn't grow at the same rate.

Crypto outlet Cryptobriefing reports an acceleration of "up to threefold," around 80 percent of it attributable to AI. No such 80 percent figure appears in METR's public text. It only speaks of a "majority," and METR cites 3x merely as the upper end of what wouldn't surprise the team.

AI as the driver, and why that's tricky

METR is confident that more training compute doesn't explain the acceleration. The evidence for this, however, was redacted. Other factors probably played a big role in late 2025, but by current best estimates they don't explain the sustained acceleration. Anthropic itself says Claude is increasingly helping to develop its successors.

METR draws a clear line here: the AI mainly supports human researchers. It doesn't yet make highly significant, fully autonomous contributions to the core training process. But some employees reported signs that this could change soon, to varying degrees depending on the area.

This is where the real concern lies. If better AI builds better AI, a self-reinforcing loop can emerge. METR found no convincing evidence of bottlenecks that would significantly slow progress over the next twelve months: not in algorithmic progress, not in compute for training and experiments, and not in training data. From this, METR concludes there's a "substantial risk" that further progress snowballs into even stronger acceleration. That's explicitly a risk assessment, not an observed result. METR also admits that its own view of the bottlenecks changed several times as new information came in, so the team is uncertain on this point.

Limits of the measurement

METR itself lists reasons why the estimate could be too high or too low.

Too high if: the models were optimized for the benchmarks (overfitting, like cramming for one specific exam), if progress without AI involvement was underestimated, or if Anthropic's progress from 2023 to 2025 was unusually slow, making the new numbers look especially large. On top of that, AECI measures well-defined tasks that can be graded automatically or by a language model. Whether progress is just as large on messy everyday work is an open question.

Too low if: recent accelerations don't show up in the data yet, human bottlenecks disappear faster than expected, or the benchmarks underestimate real-world impact.

Another problem is saturation. A model referred to as "Model 4" already scored above 50 percent on all 15 benchmarks used. When tests get too easy, future progress becomes hard to measure. Think of a class where almost everyone gets top marks.

The redactions

The most sensitive point comes right at the start of METR's report. Anthropic redacted the majority of the evidence METR considered most informative. That includes details on how much of the acceleration is due to AI, estimates of algorithmic progress, the bottleneck analysis and forecasts for future models. Anthropic says this is to protect confidential information that could help other AI developers. According to AP, Anthropic didn't initially respond to requests for comment. METR writes candidly:

"As a result, the public must rely much more heavily on METR's judgment than would be ideal given the stakes of the questions answered here."

METR says there was also disagreement and considerable uncertainty within its own team.

The bottom line

For the first time, an outside organization with deep access has confirmed what many suspected: development is speeding up at a leading AI lab, and AI tools are likely a major reason why. That's more than marketing rhetoric. The findings back up exactly the warning about a self-accelerating loop that Amodei issued in September.

At the same time, the report shows the limits of such audits. The company being audited decides what goes public, and the most important evidence is missing. Only progress on easily measurable tasks was assessed, not general capability or the pace of new products. If you read about "threefold acceleration" or the singularity, take it with a grain of salt. The solid takeaway: roughly twice as fast, with a lot of uncertainty.

It also remains to be seen whether other labs like OpenAI or Google will allow similar audits, and whether standards will soon emerge for what such auditors may see and publish. Anthropic itself notes that no such standards exist yet.