Overview
- Model Evaluation and Threat Research (METR) was given extensive access to Anthropic’s internal data and staff interviews for an roughly 11‑week review and used that material to evaluate benchmark performance.
- METR concluded Anthropic’s performance on well‑specified benchmark tasks has roughly doubled in pace compared with the prior period and estimated AI-assisted development explained the bulk of the change.
- The group said it could confidently rule out growth in training compute as the primary cause and estimated AI might account for around 80 percent of the recent acceleration while noting uncertainty in that number.
- METR flagged limits to its analysis, including redactions and a focus on automatically graded tasks, and warned that AI helping design and tune models could compound progress through recursive self‑improvement meaning systems could boost their own capabilities with less human input.
- The findings sharpen questions about Anthropic’s governance and risk control as it prepares to list, and they put pressure on regulators, investors and customers to scrutinize how labs monitor AI‑assisted development and check for emergent risks.