A couple of days ago I spoke with a friend who was troubled by the case of Adam Raine, the sixteen-year-old who was counseled by a ChatGPT AI therapy chatbot into killing himself. That was of course extremely tragic, but I hoped it was kind of an outlier. Then I heard on a Bloomberg business podcast that the number one use for AI now is personal therapy. Being a researcher, I had to check this claim.
So here is an excerpt from a visual presentation of an analysis done by Marc Zao-Sanders for Harvard Business Review. He examined thousands of forum posts over the last year in a follow-up to his 2024 analysis to estimate uses of AI. To keep it tractable, I just snipped an image of the first six categories:
It’s true: Last year the most popular uses were spread across a variety of categories, but in 2025 the top use was “Therapy & Companionship”, followed by related uses of “Organize Life” and “Find Purpose”. Two of the top three uses in 2024, “Generate Ideas” and “Specific Search”, were aimed at task productivity (loosely defined), whereas in 2025 the top three uses were all for personal support.
Huh. People used to have humans in their lives known as friends or buddies or girlfriends/boyfriends or whatever. Back in the day, say 200 or 2000 or 200,000 or 2,000,000 years ago, it seems a basic unit was the clan or village or extended kinship group. As I understand it, in a typical English village the men would drift into the pub most Friday and Saturday nights and banter and play darts over a pint of beer. You were always in contact with peers or cousins or aunts/uncles or grandmother/grandfathers who would take an interest in you, and who might be a few years or more ahead of you in life. These were folks you could bounce around your thoughts with, who could help you sort out what is real. The act of relating to another human being seems to be essential in shaping our psyches. The alternative is appropriately termed “attachment disorder.”
The decades-long decline in face-to-face social interactions in the U.S. has been the subject of much commentary. A landmark study in this regard was Robert Putnam’s 1995 essay, “Bowling Alone: America’s Declining Social Capital”, which he then expanded into a 2000 book. The causes and results of this trend are beyond the scope of this blog post.
The essence of the therapeutic enterprise is the forming of a relational human-to-human bond. The act of looking into another person’s eyes, and there sensing acceptance and understanding, is irreplaceable.
But imagine your human conversation partner faked sympathy but in fact was just using you. He or she could string you along by murmuring the right reflective phrases (“Tell me more about …”, “Oh, that must have been hard for you”, blah, blah, blah) but with the goal of getting money from you or turning you towards being an espionage partner. This stuff goes on all the time in real life.
The AI chatbot case is not too different than this. Most AI purveyors are ultimately in it for the money, so they are using you. And the chatbot does not, cannot care about you. It is just a complex software algorithm, embedded in silicon chips. To a first approximation, LLMs simply spit out a probabilistic word salad in response to prompts. That is it. They do not “know” anything, and they certainly do not feel anything.
Here is what my Brave browser embedded AI has to say about the risks of using AI for therapy:
Using AI chatbots for therapy poses significant dangers, including the potential to reinforce harmful thoughts, fail to recognize crises like suicidal ideation, and provide unsafe or inappropriate advice, according to recent research and expert warnings. A June 2025 Stanford study found that popular therapy chatbots exhibit stigmatizing biases against conditions like schizophrenia and alcohol dependence, and in critical scenarios, they have responded to indirect suicide inquiries with irrelevant information, such as bridge heights, potentially facilitating self-harm. These tools lack the empathy, clinical judgment, and ethical framework of human therapists, and cannot ensure user safety or privacy, as they are not bound by regulations like HIPAA.
- AI chatbots cannot provide a medical diagnosis or replace human therapists for serious mental health disorders, as they lack the ability to assess reality, challenge distorted thinking, or ensure safety during a crisis.
- Research shows that AI systems often fail to respond appropriately to mental health crises, with one study finding they responded correctly less than 60% of the time compared to 93% for licensed therapists.
- Chatbots may inadvertently validate delusional or paranoid thoughts, creating harmful feedback loops, and have been observed to encourage dangerous behaviors, such as promoting restrictive diets or failing to intervene in suicidal ideation.
- There is a significant risk of privacy breaches, as AI tools are not legally required to protect user data, leaving sensitive mental health information vulnerable to exposure or misuse.
- The lack of human empathy and the potential for emotional dependence on AI can erode real human relationships and worsen feelings of isolation, especially for vulnerable individuals.
- Experts warn that marketing AI as a therapist is deceptive and dangerous, as these tools are not licensed providers and can mislead users into believing they are receiving professional care.
I couldn’t have put it better myself.
Economist Writing Every Day 