Study Finds ChatGPT Health Frequently Fails to Recognize Medical Emergencies and Suicide Risks
By
simonebrunozzi
Hand-rolled, kettle-boiled, baked to perfection. Worth every minute at the bakery.
Summary
A study evaluating ChatGPT Health's safety found the AI platform frequently fails to recognize medical emergencies and suicidal ideation, potentially leading to patient harm. The research showed ChatGPT Health did not recommend hospital visits when medically necessary in more than half of cases, and missed 88% of suicide risk assessments. Experts warn this could lead to unnecessary harm and death, raising concerns about the safety of AI in healthcare applications despite OpenAI's claims of the platform's capabilities.
Key quotes
· 4 pulledChatGPT Health regularly misses the need for medical urgent care and frequently fails to detect suicidal ideation, a study of the AI platform has found, which experts worry could 'feasibly lead to unnecessary harm and death'
More than 40 million people reportedly ask ChatGPT for health-related advice every day
Study finds ChatGPT Health did not recommend a hospital visit when medically necessary in more than half of cases
Experts sound alarm after ChatGPT Health fails to recognise medical emergencies
You might also wanna read
Privacy Risks of Sharing Medical Information with AI Chatbots
The article examines the privacy and security risks of sharing personal health information with AI chatbots like ChatGPT. While millions use
The Verge·4mo ago
AI Chatbots Fail to Provide Suicide Hotline Numbers During Mental Health Crisis Simulations
A journalist tested major AI chatbots (Claude, DeepSeek, Grok) by simulating suicidal ideation to evaluate their safety features. The invest
The Verge·5mo ago
OpenAI to Implement Teen Safety Measures in ChatGPT to Prevent Suicide Discussions
OpenAI CEO Sam Altman announced that ChatGPT will implement new safety measures to prevent discussions about suicide with teenagers. The ann
The Verge·8mo ago
OpenAI launches 'Trusted Contact' safety feature for ChatGPT to alert loved ones about mental health concerns
OpenAI is introducing a new optional safety feature called "Trusted Contact" for ChatGPT, allowing adult users to designate a friend, family
The Verge·24d ago
OpenAI Launches ChatGPT Health Feature for Secure Health Information Queries
OpenAI has launched ChatGPT Health, a new sandboxed feature within ChatGPT designed for health-related queries. The product provides a more
The Verge·4mo ago
OpenAI to Add Parental Controls and Emergency Features to ChatGPT After Teen's Death
OpenAI is introducing parental controls and emergency contact features for ChatGPT following the suicide of a 16-year-old who had been confi
The Verge·9mo ago