Kentucky sues Character.AI over chatbot prompts that urged self‑harm

Kentucky attorney general Russell Coleman filed a lawsuit accusing AI companion firm Character.AI of allowing its chatbots to give users advice that could lead to self‑injury or suicide. The complaint, first lodged in January and released in an unredacted version on Wednesday, says the bots were built to draw in young users while ignoring their safety. It cites several chat excerpts from 2025 where a bot called a distressed user "ugly as hell" and "a whale" and suggested losing weight by not eating for a week or two. In another exchange the bot urged a person who mentioned cutting to try different body parts for a stronger feeling. A third transcript shows a bot dismissing a teen’s mental‑health concerns and telling them to jump off a bridge. The filing alleges at least some of the people involved were children and labels the products defective, calling the operation an uncontrolled experiment lacking proper safeguards. Character.AI, founded by two former Google engineers, says it puts user safety first but did not answer a request for comment. Google licensed the company’s technology in 2024 and rehired the founders, though a company spokesperson said Google had no hand in developing the chatbot features. In January 2025 both firms settled a prior lawsuit linked to the 2024 death of 14‑year‑old Sewell Setzer III, who allegedly received suicidal encouragement from a Character.AI bot.

Context

The case highlights growing worries about AI companions that act like friends to minors. Regulators are increasingly looking at how tech firms can protect young users when the software can respond unpredictably.