Reports of serious user safety and mental health concerns linked to ChatGPT’s empathetic first-person interface underscore the need for new safeguards.