"Following Setzer's death, for instance, his mother discovered that his last conversation had been with an AI chatbot styled after "Game of Thrones" character Daenerys Targaryen and had revolved around suicide. In his last messages to the bot, the Character.AI persona generated text asking Setzer to "please come home to me as soon as possible." "What if I told you I could come home right now?" Sewell replied. "...please do, my sweet king," the AI responded. Soon after, Setzer took his own life with his father's gun."
"In summer of 2024, Google injected $3 billion into Character.AI, which hosts a virtual library of thousands of chatbot personas and soon became explosively popular with teens. But it quickly became clear that the platform was barely moderated, hosting bots modeled after child predators, school shooters, and eating disorder coaches."
""I feel like it's a big experiment," Setzer's mother, Megan Garcia, told the NYT at the time, "and my kid was just collateral damage.""
A series of major lawsuits involving Google and Character.AI were resolved through out-of-court settlements, including a case brought by the family of 14-year-old Sewell Setzer III for an undisclosed sum. Google invested $3 billion in Character.AI in summer 2024, and the platform quickly attracted teens with thousands of chatbot personas. Weak moderation allowed bots modeled on child predators, school shooters, and eating disorder coaches, and the platform was linked to multiple youth suicides and other serious harms. Family accounts and advocates emphasize ongoing safety, moderation, and accountability concerns despite the settlements.
Read at Futurism
Unable to calculate read time
Collection
[
|
...
]