Economist Writing Every Day

According to the Johns Hopkins COVID tracker, the US has now surpassed 800,000 COVID deaths during the pandemic. The CDC COVID tracker is almost to 800,000 too. But is this number right? Confusion about COVID deaths and total deaths has been rampant throughout the pandemic, especially when comparing across countries.

One method that many have suggested is excess deaths, which is generally defined as the number of deaths in a country above-and-beyond what we would expect given pre-pandemic mortality levels. It’s a very rough attempt at creating a counterfactual of what mortality would have looked like without the pandemic. Of course, you can never know for sure what the counterfactual would look like. Would overdoses in the US have increased anyway? Hard to say, though they had been on the rise for years even before the pandemic.

So don’t treat excess deaths as a true counterfactual, but just a very rough estimate. I wrote about excess deaths in the US way back in January 2021 (feels like a lifetime ago!), and at the time for 2020 it looked like the US had about 3 million total deaths (in the first 48 weeks of 2020), which was about 357,000 deaths more than expected (again, based on historical levels of the past few years), or about 13.6% above normal.

But once we had complete data for 2020, deaths were even higher: about 19% above expected, or somewhere around 500,000 excess deaths. This compares with the official COVID death count of about 385,000 in 2020 for the US.

What happens if we update those numbers with the most recent available mortality data for 2021? Keep in mind that data reporting is always delayed, so I’ll just use data through October 2021. The following chart shows both confirmed COVID deaths and total excess mortality, cumulative since the beginning of 2020.

As we can see in the chart, there are a lot more excess deaths than confirmed COVID deaths. There were already over 1 million excess deaths through the end of October 2021 in the US, cumulative since January 2020. This compares with about 766,000 confirmed COVID deaths. That’s a big gap!

We could spend a lot of time trying to understand this gap of 250,000 deaths. Is this under-reporting of COVID deaths? Is it deaths caused by government restrictions? Is it caused by the overwhelming of the health system?

I won’t be able to answer any of those questions today. Instead, let’s ask a different question: is the potential US undercount of COVID deaths unusual?

This question is actually a hard one to answer. We don’t have good enough data for most countries to do an accurate calculation. Total mortality data is often delayed by months in some countries. To get around this issue, The Economist put together some modeled estimates of excess mortality for just about every country, right up to the most recent week. Conveniently, the excellent site Our World in Data has put this data into a very user-friendly format.

Now I know what you are thinking. Modeled data? Can we really trust models about COVID these days, given how bad some models were during the pandemic? First, let me alleviate some of your fears because this is modeling past data (which is still incomplete) rather than future data, which is where many COVID models failed.

To further see if this data is any good, let’s look at what they have for the United States. Here’s a chart of their data for mid-November, so not exactly the same date I used above (end of October) by within a few weeks.

Remarkably similar to the chart I posted above! Of course, the US has particularly good data on mortality in real time, which is why it’s pretty easy to model and why the confidence interval in the chart is pretty tight — we’re reasonably certain this number (about 1 million) is a good estimate and will be correct once we have complete data. In total, up to the most recent week in The Economist data, the US COVID death toll is undercounting, in some sense, about 23% of the total excess deaths.

Compared to developed countries, this is actually pretty bad. Italy is also a 23% undercount and Spain is at 21%, but as we look at other major developed countries the COVID death counts are much closer to the excess death counts.

In the remaining G7 countries (UK, France, Germany, Canada), the COVID death count is actually larger than the excess death estimates. The UK is pretty close: there’s about a 4% overcount of COVID deaths compared with excess deaths. But Canada’s is massive: they have almost 30,000 confirmed COVID deaths, but only about 18,000 modeled excess deaths. That’s an overcount of 67%! Of course, Canada’s mortality data reporting is slower than the US, so perhaps these modeled estimates will turn out to be wrong, but even using actual excess deaths through mid-July 2021, Canada is at about 14,000 excess deaths.

What’s really shocking (or not, depending on your priors), is that once we get out of the developed world, the excess deaths undercounts are massive. To pick one country not-totally-at-random, let’s look at China. Officially, they only have 4,636 confirmed COVID deaths. On a per capita basis, that makes China essentially the lowest country in the world, even below countries like New Zealand and Taiwan which have managed to mostly keep the virus out.

However, the modeled excess deaths in China are massive, and come with a great deal of uncertainty. The central estimate of the model is around 800,000 deaths, but there could be as many as 2 million. Using the central estimate of the model, that’s a 99% undercount of deaths in China. On a per capita basis, this is still fewer deaths than the US, but using official statistics gives us a massively different picture than what the total death statistics will eventually tell us.

The rest of the developing world looks similar. Here’s just a short list of countries that have a roughly 99% undercount: Burundi, South Sudan, Tajikistan, China, Bhutan, Chad, Democratic Republic of Congo, Nicaragua, Niger, Tanzania, and Nigeria. And there’s another 34 countries with a 90-98% undercount. For the developing world, the official COVID statistics tell us nothing about the impact of the pandemic. And claims that, for example, Africa has avoided the pandemic are just plain wrong. Once we have total mortality data for the developing world, we’ll see the true toll of the pandemic, and it will likely be very bad.

Finally, there is also a group that, at least according to these modeled estimates, have had fewer deaths during the pandemic than would be expected. Not simply that the COVID death toll is an overcount, but that they had fewer deaths in 2020 and 2021 than in prior years. Here’s the list: Seychelles, South Korea, Mauritius, Australia, Iceland, Liberia, Taiwan, Benin, Sierra Leone, Vanuatu, and New Zealand. Let’s not declare victory too soon and wait for the actual data. But even using actual death data, which is delayed but largely accurate, we can see that for Australia, New Zealand, and Taiwan they are below prior years for total deaths, and South Korea is just slightly above prior years.

I tried hard to think of a good way to visualize lots of countries along this overcount/undercount measure of COVID deaths and total excess deaths, but I didn’t come up with anything great. One thing that would be really interesting to compare is how good the count is compared with either a country’s level of economic development or its level of democracy. Do more developed countries have more accurate reports? Do more democratic countries have more accurate reports? Does development or democracy matter more? Good questions, but best left for another day when we have more complete data, rather than modeled estimates.