Overview
- An independent team from Model Evaluation and Threat Research reported that Anthropic’s benchmarked model capabilities improved about 1.9 times faster than before, a leap the researchers link to increased AI‑assisted engineering.
- METR found that Anthropic trained three nonpublic models in the past six months that are used internally to test and train successors, giving the company an edge inside its development process.
- Internal Slack surveys and staff interviews reviewed by METR showed large self‑reported productivity jumps from the most capable internal model, with the research group citing roughly a 12‑fold median gain while warning of survey limits.
- METR was granted extended on‑site access but said Anthropic required heavy redactions of evidence in the published review, a move that has intensified scrutiny over transparency and independent oversight.
- Researchers warn that recursive self‑improvement—using models to help build better models—changes the pace and nature of R&D and could increase risks to control, prompting calls for stronger monitoring and outside evaluation; the finding also matters for competition between leading AI labs.