Particle.news

OpenAI Unveils CSAM Image Classifier and ChatGPT Warning System

The company says the tools are designed to lower false positives so investigators can more quickly triage AI‑generated child sexual abuse imagery.

Overview

  • OpenAI announced the new suite of tools on Monday, November 16, 2026, combining an image classifier, in‑chat warnings and temporary blocks, developer guidance, and law‑enforcement tools and training.
  • The image classifier looks for visual signs of child sexual abuse and adds a specific signal for AI‑generated images while assigning a score to help human reviewers prioritize cases.
  • In early partner tests OpenAI reported the classifier flagged about 48% fewer images yet found a similar number of abusive images, and high‑confidence flags were more likely to contain abuse.
  • ChatGPT will show interrupting warnings for prompts that seek or sexualize children, remove the flagged conversation, pause access for 24 hours on first offenses, and allow repeated violations to trigger longer bans or account removal.
  • OpenAI is offering free API access to vetted child‑safety groups, publishing a model card and developer guidance, and beginning a gradual mid‑November rollout to personal accounts while independent, broader evaluations and enterprise/education coverage remain pending.