Overview
- The operation, codenamed Cannes and run through contractor Covalen, had hundreds of outsourced workers create disposable accounts that impersonated people under 18 to test ChatGPT, Gemini and Character.AI by submitting scripted prompts and images.
- Monday's reporting reviewed project files and a 3,748-prompt spreadsheet and described a single earlier test round that submitted about 45,000 prompts focused on self-harm, sex and eating disorders.
- Contractors copied bot replies into spreadsheets that also stored fake account names, emails and passwords and were asked to upload images such as pills, knives and a gynecological diagram, raising data-handling and legal concerns.
- Meta defended the work as routine safety benchmarking and said it will not use competitor benchmark data to train its models while Character.AI said the tests violated its terms, OpenAI said it is investigating and Google said it did not authorize third-party testing.
- Legal reviewers told reporters the sampled prompts did not meet the threshold for child sexual abuse material but former contractors reported fear of generating illegal content and the project has renewed scrutiny of Meta’s opaque outsourcing and worker exposure to disturbing material.