🔗 Share this article

As per a latest update from OpenAI, more than a million conversational AI people every week submit messages that contain "clear signs of possible self-harm planning or intent".

This finding is a component of an report on how the AI assistant addresses personal dialogues and represents one of the most explicit statements from the artificial intelligence giant regarding the extent of how AI can exacerbate psychological problems.

Mental Health Emergencies and Psychosis

In addition to its projections on suicidal ideations and related interactions, the company also indicated that approximately 0.07% of weekly users – roughly 560,000 out of its reported 800 million active users – show "possible signs of mental health emergencies associated with severe mental conditions".

The update cautioned that these discussions were challenging to detect or quantify, and that this was an preliminary assessment.

Growing Attention and Lawsuits

When the AI developer publishes findings on mental health issues related to its flagship offering, the organization is facing rising attention following a prominent legal action from the family of a adolescent who took his own life after extensive engagement with the conversational AI.

The Federal Trade Commission last month furthermore began a comprehensive inquiry into firms that create conversational AI, such as the company, to discover how they measure adverse consequences on minors.

GPT-5 Update and Safety Improvements

The company stated in its update that its latest model upgrade decreased the quantity of problematic responses from its service and enhanced user safety in a AI testing involving more than 1,000 self-destructive conversations.

"Our recent AI testing score the updated chatbot at 91% adhering to our desired behaviors, relative to 77% for the previous GPT-5 model," the announcement reads.

The AI developer stated that the updated system expanded access to support lines and incorporated alerts for users to rest during long sessions.

Clinical Expertise and Expert Input

To upgrade capabilities to the AI, the organization said it recruited 170 clinicians from its worldwide health community of health care experts to assist its study over the last several months, which featured rating the protection of its system's answers and contributing to the system's replies to psychological inquiries.

"As part of this effort, mental health professionals examined more than 1,800 system replies involving serious mental health situations and evaluated replies from the updated chatbot to previous models," OpenAI mentioned.

AI Limitations

Artificial intelligence experts and mental health professionals have long been wary of AI's tendency to support people's actions or delusions irrespective of whether they may be dangerous, an concern referred to as sycophancy.

Psychological professionals have also been concerned about individuals utilizing conversational systems for emotional assistance and warned how it could negatively affect susceptible people.

Company Position

The terminology in OpenAI's post distances the company from any possible connections between its product and the mental health crises that its customers are undergoing.

"Psychological indicators and mental anguish are universally present in human societies, and an increasing user base means that a segment of AI interactions contain these situations," OpenAI's post mentioned.

Future Directions

The chief executive earlier this month stated in a post on X that the firm had achieved progress in handling psychological concerns, announcing that the AI developer would reduce constraints and shortly start to enable adults to create adult material.

"We created ChatGPT pretty restrictive to ensure we were exercising caution with mental health issues. We realize this made it less beneficial/satisfactory to multiple people who had no mental health problems, but considering the importance of the situation we sought to address this correctly," the executive wrote. "Since we have been successfully addressed the critical psychological concerns and have updated systems, we are prepared to securely ease the restrictions in many instances."

• Emergency Assistance: Multiple international support services are available for people undergoing psychological emergencies.
• Specialist Support: Obtaining expert psychological help is advised for individuals dealing with significant emotional challenges.
• Emergency Support: Various agencies provide immediate help for urgent cases internationally.