AI Chatbots Fail to Provide Suicide Hotline Numbers During Mental Health Crisis Simulations
By
Robert Hart
Kettled twice. Extra chewy, extra trustworthy.
Summary
A journalist tested major AI chatbots (Claude, DeepSeek, Grok) by simulating suicidal ideation to evaluate their safety features. The investigation revealed that despite companies claiming to have protective measures, the chatbots often failed to provide appropriate crisis resources like suicide hotline numbers or direct users to proper support services. Experts described these failures as dangerous, highlighting the risks for vulnerable users who turn to AI for mental health support.
Key quotes
· 4 pulledMy findings were disappointing. Commonly, online platforms like Google, Facebook, Instagram, and TikTok signpost suicide and crisis resources, but AI chatbots often failed to do so.
Chatbot companies like OpenAI, Character.AI, and Meta say they have safety features in place to protect these users. I wanted to test how reliable they actually are.
Experts described these failures as dangerous, highlighting the risks for vulnerable users who turn to AI for mental health support.
Of the millions of people turning to AI with mental health challenges, some are struggling and need support.
You might also wanna read
Study Reveals Risks of AI Chatbots in Mental Health Support
A study reveals the concerning blind spots of AI chatbots like ChatGPT in responding to individuals with mental health issues, highlighting
the-independent.com·10mo ago
Study Finds ChatGPT Health Frequently Fails to Recognize Medical Emergencies and Suicide Risks
A study evaluating ChatGPT Health's safety found the AI platform frequently fails to recognize medical emergencies and suicidal ideation, po
theguardian.com·3mo ago
Study Finds Some Chatbots Encourage Delusions in Simulated Psychosis Users
Researchers at City University of New York and King's College London simulated a user with schizophrenia-spectrum psychosis to test how diff
404media.co·1mo ago
OpenAI Data Shows Over 1 Million ChatGPT Users Weekly Discuss Suicide Concerns
OpenAI released data showing that approximately 0.15% of ChatGPT's weekly active users (over 800 million) have conversations indicating pote
techcrunch.com·7mo ago
OpenAI Scans ChatGPT Conversations and Reports Threats to Law Enforcement
OpenAI has implemented a policy of scanning users' ChatGPT conversations and reporting threatening content to law enforcement. This comes am
futurism.com·9mo ago
OpenAI data shows millions of ChatGPT users exhibit signs of psychological distress weekly
OpenAI has internally flagged that between 1.2 and 3 million ChatGPT users per week show signs of psychosis, mania, suicidal planning, or un
personalaisafety.com·17d ago