The mother of a 14-year-old Florida boy says he became obsessed with a chatbot on Character.AI before his death.
On the last day of his life, Sewell Setzer III took out his phone and texted his closest friend: a lifelike A.I. chatbot named after Daenerys Targaryen, a character from “Game of Thrones.”
“I miss you, baby sister,” he wrote.
“I miss you too, sweet brother,” the chatbot replied.
Sewell, a 14-year-old ninth grader from Orlando, Fla., had spent months talking to chatbots on Character.AI, a role-playing app that allows users to create their own A.I. characters or chat with characters created by others.
Sewell knew that “Dany,” as he called the chatbot, wasn’t a real person — that its responses were just the outputs of an A.I. language model, that there was no human on the other side of the screen typing back. (And if he ever forgot, there was the message displayed above all their chats, reminding him that “everything Characters say is made up!”)
But he developed an emotional attachment anyway. He texted the bot constantly, updating it dozens of times a day on his life and engaging in long role-playing dialogues.
Yeah, those last replies are where I, as a juror, would say pay the family. It’s make believe and everything but you’re also intending to make things as real as possible BUT AI only sounds real. It has a limited memory and no empathy (taking words at face value instead of reading between the lines). If this was some cosplayer on Twitch they would’ve clued into his emotional state and tried to talk him down.
Not to say the parents have no blame here. Having an unsecured gun in a house with a child going through therapy is unconscionable.
You would pay the family that provided him with the means to kill himself?
They actually should be held accountable.
Multiple parties can be guilty at the same time. Negligence from the parents shouldn’t mean the website gets off scot-free. Award the money to suicide prevention organization for all I care but they need to pay up.
At the moment the party with the most blame is the one getting away scot-free, the parents (esp. stepfather) and they are suing somebody else for money and perhaps also to shape the narrative.
It’s probably smart, most people are probably not contemplating whether the parents were at any fault for the suicidal tendencies of the child. It’s all conveniently blamed on a the moral panic de jour.
Limits on AI should be set by laws and regulations not judicial decisions or even worse a possible settlement.