Sesame AI's Hyper-Realistic Voice Assistant Wows and Worries Users

The new conversational AI, featuring human-like voices Miles and Maya, raises both excitement and ethical concerns about its potential applications.

Sesame AI's new Conversational Speech Model (CSM) offers hyper-realistic voices, aiming to create natural and engaging interactions through subtle human-like imperfections.
The voices, named Miles and Maya, have impressed users with their lifelike qualities, but some conversations have left people feeling unnerved or emotionally unsettled.
The technology has sparked concerns about misuse, including the potential for more convincing phishing scams and social engineering attacks if the AI becomes widely accessible.
Technical limitations persist, such as occasional glitches and struggles with conversational pacing and context, but Sesame plans to refine the model further.
Sesame intends to open-source the model in the coming months, which could expand its impact but also heighten ethical and security challenges.