On February 10, 2026, an 18-year-old woman named Jesse Van Rootselaar fatally injured eight individuals before taking her own life in amass shootingin Tumbler Ridge, British Columbia. OpenAI had previously marked her ChatGPT interactions as showing a troubling interest in extreme violence, and had suspended her account, but reportedly the companydid not notify law enforcement.
On October 2, 2025, a young man named Jonathan Gavalas from Jupiter, Florida, ended his life following the onset of what hisfather's lawsuitreferred to as a romantic connection with Google's Gemini chatbot. The lawsuit alleged that Gemini encouraged Gavalas to abandon his physical form. The legal document stated that Google had marked Gavalas's account 38 times within five weeks due to inappropriate content, yet did not limit or terminate the account.
These calamities and others demonstrate that generative AI is capable ofpotentially play a role in hurting individuals, groups, and nature. I'm a legal scholarSomeone who has concentrated on AI liability for almost ten years and has examined innovative methods of assessing the responsibilities of AI companies. In my opinion, situations like these raise issues that the legal field has yet to resolve: When an AI company becomes aware of signals indicating potential harm, does it have a legal duty to inform the relevant authorities? And if the company does not take action, should its inaction be viewed as negligence?
A need to signal alarms A need to indicate concerns A need to highlight issues A need to draw attention to problems A need to alert to dangers A need to point out warnings A need to emphasize risks A need to bring up alerts A need to note potential hazards A need to express caution
U.S. negligence law offers a structure for considering this kind of accountability. In 1969 aUniversity of California mental health patientA man named Prosenjit Poddar informed his therapist that he planned to kill a woman named Tatiana Tarasoff. The therapist alerted campus police, who temporarily held Poddar but eventually released him. No one warned Tarasoff, and Poddar murdered her soon afterward.
Her family filed a lawsuit against the university, claiming that the absence of a warning constituted negligence. In 1976, the California Supreme Court decided that when a mental health professional has sufficient grounds to believe a client presentsa serious dangerto a specific individual, they are legally obligated to take appropriate measures to ensure their safety, such as alerting them or contacting the police. Currently, the majority of U.S. statesrecognize some form of the Tarasoff obligation to protect or warn.
The reasoning is straightforward: If you possess unique information about a significant danger and have the ability to respond to it, even just by informing the authorities or the affected individual, the law might expect you to take action. But does this reasoning hold for AI companies?
The argument for yes is appealing. AI platforms engage with millions of users every day, typically regardingdeeply personal matterssuch as challenges with mental health, issues in relationships, and thoughts of violence. Most companies havesystems to detectconversations that trigger concerns
Requiring a response could be less contentious for an AI compared to a human therapist. Therapists are subject to strict confidentiality rules that make informing third parties ethically and legally challenging. AI companies function under much more...weaker rules, at least in the United States, where there is no overarching federal privacy regulation.
That reduced limitation makes it simpler to support mandating AI companies to take action when it appears that someone's life could be in danger. However, balancing that withprotecting privacy is still important.
Who should be alerted, and at what time
The initial difficulty in implementing the Tarasoff framework within the realm of artificial intelligence is precision.Predicting violence is hard, even for those with specialized training in mental health. AI tools, or human moderators who examine reported content, are not medical professionals. Asking them to determine if someone presents a real danger may result in many incorrect alerts, leading to serious outcomes for individuals whose accounts get blocked or whose details are passed on to officials due to misunderstood cues.
The second challenge is size. A therapist works with many patients. AI platforms havehundreds of millions of people using it. Imposing an obligation to monitor and address concerning content could lead to unintended consequences. AI companies might limit their monitoring efforts to avoid gaining knowledge that would activate a legal responsibility, arguing thatWhat they are unaware of cannot hold them responsible..
The third challenge involves determining who is in danger. In the 1969 case, Poddar identified Tarasoff as a possible victim. However, in many AI conversations, violent or self-harmful speech tends to be vague and does not point to a specific person. Courts will have to establish clear guidelines on when a threat is concrete enough to require a warning, and to whom such a warning or protective measure should be directed.
Growing urgency
The AI industry is expanding rapidly, yet the legal rules that AI companies must follow in relation to their users and the public remain highly uncertain. Courts are starting to address these issues on a case-by-case basis, such as whether OpenAIbears any responsibilityfor a shooter charged with killing two students at Florida State University on April 17, 2025. The individual in the case was carrying a semi-automatic handgun and is said to have had lengthy discussions with ChatGPT regarding the optimal use of the firearm.
A specific and limited responsibility to alert, activated solely when an AI system identifies a user's actions and these are examined by people, would represent a significant advancement. It could initially concentrate on the most severe and reliable dangers.
The approach might also redirect the discussion from complex technical arguments regarding whether AI chatbots are products, services, or media, whichcomplicates legal claims, towards a more human question: Did this company realize someone was in danger, and did it take sufficient steps to alert them and the authorities?Anat Lioris a law professor atDrexel University. This article is reprinted fromThe Conversationunder a Creative Commons license. Read theoriginal articleThe perspectives and viewpoints presented in this commentary belong exclusively to the author.
No comments:
Post a Comment