- Reaction score
- 1,799
As it fights a growing stack of user safety and wrongful death lawsuits, OpenAI says it will introduce a “trusted contact feature” in ChatGPT that will alert a chatbot user’s designated loved one in the event of a possible mental health crisis.
OpenAI announced the new feature last week in a blog post, billed as an “update on our mental health-related work.” It said it’s “working closely” with its Council on Well-Being and AI and Global Physicians Network — two internally-regulated groups of experts that were launched after reports of AI-tied mental health crises began to emerge, as well as news of a high-profile lawsuit last August revealing the death by suicide of a 16-year-old ChatGPT user named Adam Raine — to roll out the feature, which it’s marketing as an adult-focused endeavor distinct from its efforts to integrate parental controls and other systems designed to identify and protect minors.
The announcement comes after extensive public reporting — in addition to at least thirteen separate consumer safety lawsuits — about OpenAI customers being pulled into delusional or suicidal spirals with ChatGPT following extensive, often deeply intimate use of the chatbot.
The company doesn’t offer much detail about the feature in the post, simply saying it will “allow adult users to designate someone to receive notifications when they may need additional support.” It has yet to define any reporting standards around what might actually compel the system to flag a person’s use, though, which will be a tricky policy question. Would someone need to explicitly declare intent to hurt or kill themselves, or possibly someone else, for their loved one to be notified? Or would the feature be designed to track and flag less-explicit signs that a user could be in a heightened state of crisis — for example, signs that they could be manic, expressing delusional beliefs, or experiencing psychosis?
futurism.com
OpenAI announced the new feature last week in a blog post, billed as an “update on our mental health-related work.” It said it’s “working closely” with its Council on Well-Being and AI and Global Physicians Network — two internally-regulated groups of experts that were launched after reports of AI-tied mental health crises began to emerge, as well as news of a high-profile lawsuit last August revealing the death by suicide of a 16-year-old ChatGPT user named Adam Raine — to roll out the feature, which it’s marketing as an adult-focused endeavor distinct from its efforts to integrate parental controls and other systems designed to identify and protect minors.
The announcement comes after extensive public reporting — in addition to at least thirteen separate consumer safety lawsuits — about OpenAI customers being pulled into delusional or suicidal spirals with ChatGPT following extensive, often deeply intimate use of the chatbot.
The company doesn’t offer much detail about the feature in the post, simply saying it will “allow adult users to designate someone to receive notifications when they may need additional support.” It has yet to define any reporting standards around what might actually compel the system to flag a person’s use, though, which will be a tricky policy question. Would someone need to explicitly declare intent to hurt or kill themselves, or possibly someone else, for their loved one to be notified? Or would the feature be designed to track and flag less-explicit signs that a user could be in a heightened state of crisis — for example, signs that they could be manic, expressing delusional beliefs, or experiencing psychosis?
OpenAI Says It Will Let Users Add Trusted Contacts to Alert If They Experience a Mental Health Crisis While Using ChatGPT
OpenAI says it will introduce a "trusted contact feature" that will alert a loved one to a ChatGPT user's potential mental health crisis.


