Overview
- OpenAI announced on Monday that it has made an image classifier available to vetted organizations through a free API and has begun rolling in‑chat warnings and 24‑hour suspensions for personal ChatGPT accounts that try to create or access sexualized images of children.
- The classifier produces two scores—one for suspected CSAM and one that flags images likely generated by AI—and OpenAI published a model card that details architecture, metrics, and limits.
- In early partner tests the tool flagged about 48% fewer images while finding roughly the same number of abusive images, but OpenAI’s live tests on ChatGPT traffic show much lower precision (about 29.8% for the CSAM score), so human review remains required.
- ChatGPT will remove flagged conversations, show a warning with links to specialist support, suspend access for 24 hours on first offense, and may impose longer bans or permanent removal for repeat violations; OpenAI says it reports confirmed CSAM to NCMEC.
- The release follows a surge in AI-related CSAM reports to authorities and involves collaborators such as NCMEC, Thorn, ROOST and the Tech Coalition, plus new investigator tools and training, though the rollout currently excludes some account types and product interfaces.