Economist Writing Every Day

What’s the connection between social media use and mental health, especially among young people? You’ve probably heard a lot about this recently, in the media, by politicians, and among friends chatting about their kids. Lots of assertions are made, but there is also a bit of research on this topic. As someone who frequently uses social media myself, as well as a parent of young children, and a teacher that works every week with young college students, I am particularly interested in this topic.

Jonathan Haidt and various co-authors have been trying to catalog all the research on the topic and figure out if there is a connection between the decline in teenage mental health and the rise of social media use. Haidt also has a new book on this topic, as well as the decline of “free play” among kids, which I have not yet read but I’ve looked through his documents that contain all of the underlying and summaries of the research he is citing. I’ll read the book soon, as I’m certainly part of the intended audience (see the last sentence of the above paragraph). And while this research is very much outside of my area of expertise, my training as an economist has taught me how to read academic papers and to be convinced by evidence, so once again I’m very much the intended audience on this score as well.

Please read this post as my attempt to understand the evidence and start to form conclusions and/or critique what Haidt is saying. It’s a work in progress, and I’ll write more as I read and think more about it.

For your convenience, here are the key documents from Haidt and co-authors. Don’t take this a full substitute for reading the book, but even if you do read his book you will probably need to refer back to these documents if you really want to dig into the details of the data and research.

1. Trends in mental health
2. Possible connections between social media and mental health
3. Possible connections between free play and mental health (I won’t focus on this area today)

First things first: there is clearly something going on with mental health. Suicide rates, particularly but not only for very young Americans, are growing at alarming rates. Here’s a chart from document #1 linked above:

There’s also an interesting discussion in document #1 between Haidt and Chris Ferguson which starts on page 167 on whether one should use changes in percentage terms or in absolute numbers. I encourage you to read the exchange, as both make valid points, but in light of that exchange I also decided to replicate Haidt’s chart using changes in raw numbers of suicides rather than percentages.

Two notes. First, I use these same age bins as they do, even though (for reasons they don’t explain) they skip over many 5-year age bins (25-29, 35-39, etc.). Second, because the size of the populations have changed for these age groups between 2000-09 to 2020, my chart shows much the absolute number of deaths increased in 2020 over what it would have been if the age group had the same suicide rate as they did from 2000 to 2009.

Notice that looking at it this way, it’s not very young girls that stand out as the group most affected by the increase in suicides. Instead, it is men in their 20s and 30s. The increase for girls ages 10-14 was the largest in percentage terms, but in absolute terms it doesn’t particularly stand out. This is, of course, because the base rate was so small, which is one of the reasons brought up by Ferguson in the exchange with Haidt found in document #1. I think it’s useful to look at rates of change for some purposes (clearly something changed with young girls!), but it’s also useful to look at raw numbers (if we really want to save lives, focusing on men in their 20s and 30s might be the most useful area).

Those age differences aside, once again it is clear that something is going on in the US with suicides. “Deaths of despair” has been talked about quite a bit, but the explanations for “deaths of despair” (e.g., decline of employment opportunities in Appalachia) probably can’t explain very well the rise of suicides among girls ages 10-14.

So what does explain it? Haidt and co-authors try to make the case that the rise of social media (combined with the decline of free play) are a primary cause. The evidence is outlined in document #2 linked above. The document is 340 pages long, but the best evidence is in sections 3.1-3.4, especially section 3.1 which are randomized experiments which they claim support their theory (social media causes bad mental health outcomes). There are 16 studies. I won’t summarize each one, but I’m especially interested in the groups that they study. Haidt is particularly interested in the mental health of heavy social media users, especially young girls, and especially Americans. But none of these studies are about that group. Here are the age groups studied in those 16 papers:

• Adults: 8 (two only females)
• College students: 7 (one only females)
• Teenage girls: 1 (ages 14-18)

So only one study (here it is) focused on teenage girls. And this one wasn’t even done in the US! Most of the other studies were (a few were in Germany), but the teenage girls study was done in the Netherlands. It’s an interesting study, where they manipulated Instagram images to make them look more attractive for some participants. The girls who saw the manipulated images reported lower body satisfaction. The result was statistically significant, but I’m not really sure whether we should interpret this effect as being large. On a 9-point scale (if I’m reading this correctly), body image was 4.94 in the control group and 4.57 in the treatment group. Is this large? Should we care about? Does it persist 1 month or 1 year after the study? All of this is unclear.

Think what you will of that study, this is one single study of 144 girls in the Netherlands. And it’s not even really studying what we want, i.e., intensive social media use. Is that really all there? Not necessarily. There are studies of young adults. Maybe the same results can be generalized to young girls. Maybe not? But this seems rather strange. Why are there no other good studies on the age group we are most interested in?

There are another 6 experimental studies that Haidt lists, which show no effect or mixed effects of social media on health. Five of these studies had undergraduate subjects (again, not totally useless, but not the group Haidt is really interested in). But one of these studies had 13–16-year-old subjects. Not girls specifically, but it’s the right age group (here’s the study). This study had the treatment group give up Facebook for 5 days. The results were mixed. While they did have lower stress, they also reported lower life satisfaction. And it’s also unclear if these results would hold up long-term, rather than just a 5-day No Facebook period.

So really, we don’t have much evidence to go on. But that’s my biggest concern with Haidt’s research on this topic and his strong policy conclusions, including those that involve government intervention to prevent some social media use by young people. We don’t have much evidence. Of course, we often have to operate under conditions where the evidence isn’t very good, but still we must make decisions (hello COVID in 2020!). But our lack of evidence should give us caution on how we move forward.

This is my main contention with Haidt at this point. He’s making strong claims when there just isn’t much good evidence backing his main causal claim, especially for young kids. I look forward to reading the book (on order from Amazon right now, but there seems to be a lot of interest in his book so it will take a few days for me to get it), and perhaps in his book I will find he is more cautious about making strong claims about causation. But from reading his Substack, and listening to this Conversation with Tyler, he seems to be very convinced that the science is settled, or at least strong enough that we need to take action, including government action.

I will write more after reading and thinking more on the topic.