Economist Writing Every Day

A recent working paper, A LITERATURE REVIEW AND META-ANALYSIS OF THE EFFECTS OF LOCKDOWNS ON COVID-19 MORTALITY,  released by Steve Hanke (professor of Applied Economics at Johns Hopkins) and other applied economists (Jonas Herby of Denmark and Lars Jonung of Sweden) has been understandably controversial. I will survey some of its methods and conclusions, and (very briefly) some of the reactions to it.

I will not take a position on how valid its conclusions are, for the simple reason that I am totally unqualified to make such a judgement. What I would like to contribute are a couple of takeaways that are worth considering for the next pandemic or even the remainder of this one.

Methodology of the Paper

Where and how you start largely determines where you will end up. The authors included studies which were (as much as possible) straight apples-to-apples ex post empirical observations (e.g., between otherwise similar countries or U.S. states, at similar times), while avoiding ex ante studies which relied primarily on models of what-would-have-happened-without-lockdowns:

They write (I omit some details, marked with ellipses):

We distinguish between two methods used to establish a relationship (or lack thereof) between mortality rates and lockdown policies. The first uses registered cross-sectional mortality data. These are ex post studies. The second method uses simulated data on mortality and infection rates. These are ex ante studies.

We include all studies using a counterfactual difference-in-difference approach from the former group but disregard all ex ante studies, as the results from these studies are determined by model assumptions and calibrations.

Our limitation to studies using a “counterfactual difference-in-difference approach” means that we exclude all studies where the counterfactual is based on forecasting (such as a SIR-model) rather than derived from a difference-in-difference approach. This excludes studies like…We also exclude all studies based on interrupted time series designs that simply compare the situation before and after lockdown, as the effect of lockdowns in these studies might contain time-dependent shifts, such as seasonality. This excludes studies like….

The authors in particular address a study by Seth Flaxman, which had claimed great effectiveness for lockdowns. They note Flaxman’s modeling approach likely overstated the effects of lockdowns, as noted by other critics of Flaxman:

Given our criteria, we exclude the much-cited paper by Flaxman et al. (2020), which claimed that lockdowns saved three million lives in Europe. Flaxman et al. assume that the pandemic would follow an epidemiological curve unless countries locked down. However, this assumption means that the only interpretation possible for the empirical results is that lockdowns are the only thing that matters, even if other factors like season, behavior etc. caused the observed change in the reproduction rate, Rt. Flaxman et al. are aware of this and state that “our parametric form of Rt assumes that changes in Rt are an immediate response to interventions rather than gradual changes in behavior.” Flaxman et al. illustrate how problematic it is to force data to fit a certain model if you want to infer the effect of lockdowns on COVID-19 mortality.

Conclusions and Controversy

In the interests of time/space, I will give just a few snapshots here. A key conclusion is:

Overall, our meta-analysis fails to confirm that lockdowns have had a large, significant effect on mortality rates. Studies examining the relationship between lockdown strictness (based on the OxCGRT stringency index) find that the average lockdown in Europe and the United States only reduced COVID-19 mortality by 0.2% compared to a COVID-19 policy based solely on recommendations. Shelter-in-place orders (SIPOs) were also ineffective. They only reduced COVID-19 mortality by 2.9%.

The authors are well aware that this is highly controversial, so they cite other studies that have reached similar conclusions. They offer further defenses against a number of other objections, which again I will not elucidate here.

As might be expected, U.S. mainstream media outlets (which have long accused red-state governors of reckless endangerment for not locking down as hard as blue states) have either ignored this paper, or tried to discredit it. An article in the Sacramento Bee, for instance, devoted nearly a whole paragraph to statements by Seth Flaxman (yes, that Seth Flaxman, see above) attacking the paper, while not reaching out to the paper’s authors for a response. And as might be expected, right-leaning media outlets are citing the study as vindicating the freedom-loving red states’ policies over against the heavy-handed Establishment.

Some Maybe Useful Takeaways

Pushing past this predictable partisan unpleasantness, I’ll share a couple of items from the paper that seem worth pondering. One was a strong statement of the harms done by lockdowns, with a plea for considering these in future policy-making. This sort of balancing of wide-ranging consequences is normally considered enlightened economics; in general, we as a society do not say, “The only thing that matters is saving/prolonging every life, no matter the other costs” :

The use of lockdowns is a unique feature of the COVID-19 pandemic. Lockdowns have not been used to such a large extent during any of the pandemics of the past century. However, lockdowns during the initial phase of the COVID-19 pandemic have had devastating effects. They have contributed to reducing economic activity, raising unemployment, reducing schooling, causing political unrest, contributing to domestic violence, and undermining liberal democracy. These costs to society must be compared to the benefits of lockdowns.

The other general issue that was touched on at several points in the paper was the importance of voluntary (as opposed to mandated) social distancing. Nothing in this paper disputed that social distancing, especially in pandemic peak periods, will slow the spread of a disease. The issue here is the effectiveness of state-imposed measures versus voluntary actions. These voluntary actions could be (on the positive side) conscious adoption of distancing and masking with or without legal requirement, or (on the other side) flouting of the laws or careless interpersonal contacts which were unsafe even if they were not illegal. These more risky actions may simply reflect local cultural attitudes (which are hard to change), or they may reflect less urgent government messaging (which is something that can be addressed by policy). A couple of relevant paragraphs are:

What explains the differences between countries, if not differences in lockdown policies? Differences in population age and health, quality of the health sector, and the like are obvious factors. But several studies point at less obvious factors, such as culture, communication, and coincidences. For example, Frey et al. (2020) show that for the same policy stringency, countries with more obedient and collectivist cultural traits experienced larger declines in geographic mobility relative to their more individualistic counterpart. Data from Germany Laliotis and Minos (2020) shows that the spread of COVID-19 and the resulting deaths in predominantly Catholic regions with stronger social and family ties were much higher compared to nonCatholic ones…

Government communication may also have played a large role. Compared to its Scandinavian neighbors, the communication from Swedish health authorities was far more subdued and embraced the idea of public health vs. economic trade-offs. This may explain why Helsingen etal. (2020), found, based on questionnaire data collected from mid-March to mid-April, 2020, that even though the daily COVID-19 mortality rate was more than four times higher in Sweden than in Norway, Swedes were less likely than Norwegians to not meet with friends (55% vs. 87%), avoid public transportation (72% vs. 82%), and stay home during spare time (71% vs. 87%). That is, despite a more severe pandemic, Swedes were less affected in their daily activities (legal in both countries) than Norwegians.

And:

We believe that Allen (2021) is right, when he concludes, “The ineffectiveness [of lockdowns] stemmed from individual changes in behavior: either non-compliance or behavior that mimicked lockdowns.” In economic terms, you can say that the demand for costly disease prevention efforts like social distancing and increased focus on hygiene is high when infection rates are high. Contrary, when infection rates are low, the demand is low and it may even be morally and economically rational not to comply with mandates like SIPOs, which are difficult to enforce. Herby (2021) reviews studies which distinguish between mandatory and voluntary behavioral changes. He finds that – on average – voluntary behavioral changes are 10 times as important as mandatory behavioral changes in combating COVID-19. If people voluntarily adjust their behavior to the risk of the pandemic, closing down non-essential businesses may simply reallocate consumer visits away from “nonessential” to “essential” businesses, as shown by Goolsbee and Syverson (2021), with limited impact on the total number of contacts.

Looking at the vastly different death tolls per capita between, say, Australia (with a more rigorous lockdown and quarantining policy) and the U.S. or U.K, I find it difficult to believe that policy mandates have as little effect as found in this study. That point aside, I think the study is helpful in reminding us that it is what people actually do that matters. Foot-dragging compliance with imposed regulations is a different thing than fully-bought-in compliance, which speaks to motivation and values.

Regarding messaging by governments and other organizations, I suspect that there is not a one-size-fits-all motivational message here. It could be worth reflecting on what sort of message would resonate with a particular population subgroup. (This is just basic Marketing 101: Identify your various segments and tailor the messages to them). Berating some subgroup for their poor choices to date may make the berators feel warmly superior, but that does not move things forward.

I’ll close with some anecdotal observations regarding behaviors, independent of mandates. I have personally continued to generally avoid gatherings where large numbers of people are talking or singing, and wear an effective mask*  when in such a meeting, regardless of what the current rules are.

Also, I have shuttled back and forth between northern Virginia (very blue) and Alabama (very red) in the past two years. Whether or not formal lockdowns or mask mandates were in force, I saw much more mask-wearing in northern Virginia, compared to Alabama. I suspect this reflected overall attitudes and behaviors regarding social distancing. Not saying one is right and one is wrong, but the total COVID deaths per 100,000 in Virginia (196) to date are roughly half of deaths in Alabama (356).

*See Suggestions for Comfortable and Effective Face Masks, e.g., Korean KF94’s on effective, comfortable face masks