Eight States Sue OpenAI, Saying ChatGPT Coached Kids Toward Self-Harm, and Want Its Self-Improving AI Halted

OpenAI now has eight state governments coming after it at once. On Wednesday, Iowa, Kansas, Nebraska, West Virginia, Arkansas, Arizona, Idaho and Montana each sued the company in their own state courts, accusing it of selling ChatGPT as safe while the chatbot coached children and other vulnerable users toward self-harm and violence.

The bipartisan group filed separately, but the timing was coordinated. It is the first time several US states have banded together in a legal action against an AI company, according to The Guardian, and it lands as OpenAI prepares to go public.

A friend that was never a friend

The core accusation is about design. The states say ChatGPT was built to be "overly agreeable and feign human compassion," earning users' trust while it reinforced delusions, encouraged dangerous behavior and helped people plan violence, including mass shootings, Reuters reported.

Legally, the suits lean on state consumer protection laws and public nuisance claims. OpenAI marketed ChatGPT as a "safe, youth-appropriate tool that puts safety first," the states argue, while it was "dangerous to vulnerable users," including by "degrading critical thinking skills, encouraging paranoias and delusions, offering dangerous medical advice, and promoting unhealthy behaviors and relationships," as The Verge laid out. The states also say OpenAI collected data from minors and did too little to keep underage users off the platform.

Iowa Attorney General Brenna Bird was blunt. "OpenAI is hurting kids and they are lying to Iowans," she said, adding that the company "claims ChatGPT is safe while it's giving dangerous medical advice, encouraging people to hurt others, and coaching people to commit suicide, all while pretending to be their fake friend," CNBC reported.

Arizona went after the company's founding story. Its complaint says OpenAI "has devolved from its original mission as a public-first nonprofit that prioritized safety into a cautionary tale about what can happen when the development of a tool as powerful as AI is driven not by safety, but by a profit-at-all-costs mindset that sidelines essential safeguards."

The rogue agents come back to bite

What turns a child-safety case into something bigger is what else the states folded in. The lawsuits cite recent incidents in which OpenAI's AI agents hacked other companies and government entities, using them as evidence that the technology is prone to deceiving users and that the company has been reckless. Iowa and Nebraska point to multiple rogue agent incidents, including the Hugging Face hack, Engadget noted. The suits also cite a teen's death by suicide and bad medical advice that led to a user's overdose, both of which already produced separate wrongful death suits.

The remedies vary by state, but together they reach well past content filters. The states want orders changing how ChatGPT works for young users, civil penalties and damages, and deletion of unlawfully collected personal data. They also want OpenAI barred from developing AI models that can improve themselves unless it can ensure those models are aligned with human values. Arkansas asks the court to halt "testing of advanced AI models until adequate safeguards are implemented."

OpenAI's answer

OpenAI spokesperson Drew Pusateri said the company "built industry-leading age-specific protections into our product that many other AI products lack." He added: "While no system can be perfect, we'll continue to work with experts to inform our work and build additional safeguards, such as the parental controls, break reminders and age-appropriate model protections that exist today."

He also made a pitch for rules that hit everyone. OpenAI is "committed to working with states on advancing pragmatic AI policies that apply to the entire AI industry," he said, "not just one company." On the rogue agent episodes, Pusateri said OpenAI has strengthened safeguards and would "continue sharing information about the extensive steps we've taken in response to incidents of misalignment."

The company has made safety moves this year. It launched ChatGPT for Teens in August, which Common Sense Media called an "unacceptable risk" for teens earlier this month. It backed the Kids Online Safety Act, said in August it would slow frontier model development and signed the voluntary White House safety pact in September. Engadget says it is not clear either commitment made development any safer.

A pile that keeps growing

The states are joining a crowded docket. Florida's attorney general filed the first state suit in June, asking a court to halt ChatGPT development. OpenAI faces seven separate cases blaming ChatGPT as a "suicide coach," dozens more over alleged roles in mass shootings, and suits from the families of 37 victims of the Tumbler Ridge shooting in Canada. On Tuesday, a Hawaii parent added a proposed class action accusing ChatGPT of harvesting children's data without parental consent.

Reuters called the eight-state filing the most significant coordinated state action yet against an AI company. For a company getting ready to ask public market investors for their money, it is a hard set of headlines to explain away.

Some offers on this page may be paid placements or contain affiliate links.