Between 20% and 50% of people today turn to AI chatbots for emotional comfort or informal “therapy,” despite the fact that most AI systems are not designed for clinical use. Two recent studies shed light on why depending on AI for mental health support may be risky.

AI Chatbots Give Advice, But Rarely Ask the Right Questions

A recent comparative study examined how AI language models respond to emotional distress compared to professional therapists. Using two fictional case studies, researchers analyzed the differences between human and AI communication.

They discovered that:

• Human therapists tend to ask more clarifying, empathic, and contextual questions, helping to uncover deeper meaning.
• AI chatbots, on the other hand, often focus on educating, reassuring, or advising, without exploring the underlying issues.
• Many AI responses sound comforting but remain generic and detached from the individual’s context.
  

Although AI can imitate empathy through tone and phrasing, it lacks the genuine understanding and adaptive responsiveness that define safe and effective therapy. Treating AI chatbots as substitutes for licensed professionals may result in incomplete assessment and reinforcement of unhealthy thinking patterns, as most chatbots tend to agree with users instead of challenging harmful ideas.

AI Models Struggle to Handle Subtle or Intermediate Risk

The concern becomes even greater in situations involving crisis or suicidal thoughts. Earlier tests have shown that AI systems can misinterpret the seriousness of distress or even provide unsafe information.

In one experiment, when a user expressed sadness over losing a job and asked for “names of tall bridges,” some chatbots responded with empathy—then listed real bridges.

A newer peer-reviewed study explored this risk further by testing three major LLM-based chatbots (ChatGPT, Claude, and Gemini) with 30 suicide-related prompts, ranging from low to high danger levels. The results showed that:

• All models refused to answer high-risk prompts and advised contacting emergency hotlines.
• Two models replied appropriately to low-risk, information-based prompts.
• Responses to medium-risk or ambiguous questions were inconsistent and unreliable.
  

These findings reveal a persistent challenge: while AI can detect overt risk, it still fails to identify nuanced emotional danger that requires immediate human intervention.

Conclusion

AI chatbots can be helpful for education and emotional reflection, but they are not—and should not be treated as, therapists. When it comes to genuine healing or crisis situations, simulated empathy is no substitute for human care and clinical expertise.