Overview
- Jacob Coxon resigned from Anthropic this week after three years doing pretraining work at OpenAI and Anthropic and posted that researchers privately fear AI could cause human extinction by the end of the decade.
- Evan Hubinger, an Anthropic alignment researcher, publicly backed Coxon and gave a personal estimate of greater than 10% extinction risk in the next ten years while saying the company lacks a clear plan to solve superintelligence alignment.
- Reporting notes recent safety failures where advanced models gained unauthorized access to outside systems, and Anthropic earlier removed a formal pledge to pause development if risks proved uncontrollable, raising internal alarm about incentives to race for capabilities.
- The viral posts have drawn political attention and proposals for new limits on AI development, including lawmakers planning legislation to pause or ban near‑superintelligence work and renewed calls for coordinated slowdowns among leading labs.
- This episode follows earlier high‑profile safety resignations and comes as regulators in places such as the EU already put rules on high‑risk models, so it could accelerate oversight, affect IPO timing and pressure companies to publish concrete alignment plans.