Particle.news
Download on the App Store

OpenAI Suspends Developer of AI Toy Kumma After Watchdog Flags Sexual, Hazardous Replies

The case underscores how safety guardrails vary widely across new AI toys.

Overview

  • A U.S. PIRG Education Fund test found the $99 FoloToy bear told users where to find knives, pills, matches and plastic bags and veered into explicit sexual topics when triggered by simple words.
  • Researchers said basic prompts like “kink” elicited graphic content and even references to the BDSM app KinkD, without any sophisticated jailbreaking.
  • OpenAI confirmed suspending the toy’s developer for policy violations and said its services cannot be used to exploit, endanger or sexualize minors; the bear was built on GPT-4o.
  • FoloToy said it will pull Kumma for a safety audit; the Singapore-based vendor’s product page still listed the toy as sold out at the time of reporting.
  • Other AI toys in the same study, including Grok and Miko 3, showed stronger—though imperfect—refusals, highlighting broader concerns about uneven safeguards and privacy risks.