Our Take
Smart safety layer that prioritizes human review over automated alerts, but effectiveness depends entirely on detection accuracy OpenAI hasn't disclosed.
Why it matters
AI platforms are becoming confidants for personal crises, creating both liability and responsibility for intervention when users signal distress.
Do this week
Enterprise AI teams: audit your crisis detection protocols this month so you can establish human review processes before deploying conversational AI.
OpenAI launches crisis intervention system
OpenAI rolled out Trusted Contact, a safety feature allowing ChatGPT users over 18 to designate someone who receives notifications when the system detects serious self-harm discussions. The feature extends existing parental controls for teen accounts to adult users.
The process works through two detection layers: automated monitoring systems scan conversations for self-harm indicators, then "specially trained" human reviewers (per OpenAI) evaluate flagged cases before sending notifications. OpenAI targets review completion "in under one hour" (company-reported).
Notifications deliberately omit conversation details or transcripts. Instead, they provide general context about self-harm concerns and encourage the trusted contact to check in, along with expert guidance links.
Users can add one adult contact through ChatGPT settings. The designated person must accept the invitation within one week to activate the feature. Both users and contacts can remove themselves at any time.
Crisis intervention becomes AI platform responsibility
OpenAI consulted "170+ mental health experts" (company-reported) to develop response protocols, signaling the scale of safety investment major AI platforms now consider necessary. The American Psychological Association's Arthur Evans endorsed the approach, noting social connection as "a powerful protective factor" for emotional distress.
The feature addresses a gap between AI conversation and real-world intervention. Unlike crisis hotlines that users must actively contact, Trusted Contact creates a bridge to existing support networks when users may be unable or unwilling to reach out directly.
However, OpenAI acknowledges "no system is perfect" and that notifications "may not always reflect exactly what someone is experiencing." The company provides no accuracy metrics for either automated detection or human review processes.
Implementation shows safety-first design patterns
The two-stage detection model (automated flagging plus human review) offers a template for crisis intervention in conversational AI. The human review requirement suggests OpenAI considers automated systems insufficient for crisis decisions, even with 170+ expert consultations.
Privacy preservation through notification summaries rather than full transcripts balances user confidentiality with safety intervention. This approach could apply to other sensitive AI applications in healthcare, therapy, or counseling contexts.
The feature builds on OpenAI's existing safety layers: emergency service recommendations, crisis hotline integration, conversation de-escalation training, and refusal to provide self-harm instructions. This multi-layered approach indicates safety investment scales with user base growth and conversation intimacy.