Weigh costs, benefits, and evidence quality

Living means making decisions with imperfect information. But Covid provides many examples of how people and institutions are often still bad at this. A few common errors:

  1. Imperfect evidence = perfect evidence. “Studies show Asprin prevents Covid”. OK, were the studies any good? Did any other studies find otherwise?
  2. Imperfect evidence = “no evidence” or “evidence against”. In early 2020, major institutions like the WHO said “masks don’t work” when they meant “there are no large randomized controlled trials on the effectiveness of masks”
  3. Imperfect evidence = don’t do it until you’re sure Inaction is a choice, and often a bad one. If the costs of action are low and the potential benefits of action high, you might want to do it anyway. Think masks in 2020 when the evidence for them was mediocre, or perhaps Vitamin D now.
  4. Imperfect evidence = do it, we have to do something Even in a pandemic, it is possible to over-react if the costs are high enough and/or the evidence of benefits bad enough (possibly lockdowns, definitely taking up smoking)

Any intro microeconomics class will explain the importance of weighing both costs and benefits. But how do we know what the costs and benefits are? For many everyday purchases they are usually obvious, but in other situations like medical treatments and public policies they aren’t, particularly the benefits. We have to estimate the benefits using evidence of varying quality. This creates more dimensions of tradeoffs- do you choose something with good evidence for its benefits, but high cost? Or something with worse evidence but lower costs? Graphing this properly should take at least 3 dimensions, but to keep things simple lets assume we know what the costs are, and combine benefits and evidence into a single axis called “good evidence of substantial benefit”. This yields a graph like:

Applied to Covid strategies, this yields a graph something like this:

This is not medical advice- I say this not merely as a legal disclaimer, but because my real point is the idea that we should weigh both evidence quality and costs, NOT that my estimates of the evidence quality or costs of particular strategies are better than yours

Judging the strength of the evidence for various strategies is inherently difficult, and might go beyond simply evaluating the strength of published research. But when evaluating empirical studies on Covid, my general outlook on the evidence is:

Of course, details matter, theory matters, the number of studies and how mixed their results are matters, potential fraud and bias matters, and there’s a lot it makes sense to do without seeing an academic study on it.

Dear reader, perhaps this is all obvious to you, and indeed the idea of adjusting your evidence threshold based on the cost of an intervention goes back at least to the beginnings of modern statistics in deciding how to brew Guinness. But common sense isn’t always so common, and this is my attempt to summarize it in a few pictures.

Suggestions for Comfortable and Effective Face Masks, e.g., Korean KF94’s

With Covid cases and deaths surging despite widespread vaccinations, face masks are back in. Back in the dark days of early-mid 2020, all commercial masks of any kind were allocated to medics/first responders. Back then, the only mask option for the rest of us was to cobble together something made of regular cloth. But studies I looked at show that the protective performance of those cloth masks, and even standard rectangular surgical masks, is really quite poor [1].

A cloth or surgical mask is definitely better than nothing, but is much inferior to other mask options which are now widely available. If you are going to bother with a mask at all, why not use a more effective one? A well-known effective mask is the KN-95. It has a kind of aggressive beak-like profile, as shown below, and typically uses elastic earloops. It gives good protection because it seals to the face (including around the nose, thanks to a malleable metal strip there) and is made of appropriate multi-layer filter materials. It is the standard protective respiratory mask in China, whereas in the U.S. the standard protective mask is an N95, with elastic straps that go around the whole head, not the ears.

Image Source: Amazon

I got a box of ten KF95’s back in June of 2020. I loved them – they were comfortable, worked OK with my glasses, and clearly sealed well to my face. However, I gave some of these away to family members, lost a few, and used the rest so many times so they started to lose their shapes.

There are lots of KN95’s for sale on Amazon, all made in China. Not all of these may be of the same quality. Some but not all of these brands were tested and approved by the FDA for emergency use; this article from March 2021 notes some of these brands that were for sale on Amazon at that time. It seems the approved Powecom masks are still for sale.

A problem with most of these Amazon KN95’s is that the earloops are painfully tight around the ears. I pored over the comments to try to select masks where at least some of the reviewers claimed the masks didn’t hurt. Alas, all my KF95’s are pretty much unbearable for a guy like me with maybe an oversized head. (I compared the length of their earloops with my original comfortable KF95, and indeed the earloops are clearly shorter on all the new ones).

In the course of reading dozens of reviews of KN95 masks, I saw several comments recommending KF94 masks instead. These are made in South Korea. They are standard personal protective equipment in that country, and as such must meet certain standards for fine particle capture. They look a little different than most masks, but seem less beak-like than the KN95s. They have a flattish rectangular middle part which is the main filter, with two triangular sections that cover the nose and the chin:

Image: Amazon

So I got a box of KF94’s, large size, and they are wonderfully comfortable for me. No stress on the ears, and sealing over the whole face. The shape of the mask keeps it from rubbing on your mouth. The “Large” size I got was actually a tiny bit looser than felt optimal, so I tied tiny knots in the lower part of the earloops to shorten them a bit. My wife uses a mask extender strap (e.g., HX AURIZE Mask Extender Strap on Amazon) around the back of her head to pull the KF94 earloops a little tighter, with the added benefit that if she wants to take the mask off temporarily, it can hang around her neck via the extender strap. In sum, the KF94s are a win, and I highly recommend them.

I see on Amazon that small (for e.g. 7-12 year old children) and medium KF94 masks are also available. One caveat on buying is to make sure that you are buying from an actual Korean seller, else you risk getting an inferior Chinese knockoff.

Back to my unusable KN95’s. I know that you can use mask extender straps like the HX Aurize straps linked above, or similar homemade hacks, to go behind your head and take some of the direct pressure off the back of the ears. However, I found using a behind-the-head strap still put pressure on part of my ears, and was just an added complication. I thought, surely there must be some way to make those darned earloops simply longer. What I did for one mask was to cut the earloops close to the bottom of the mask, and tie in a small rubber band into each loop, to make them effectively longer. (I put a dab of glue on the cut ends of the earloops, to keep them from unravelling). That worked out well, so I can recommend this as a “hack”. I also see on Amazon that you can order ¼” wide white elastic ear loop type band material, and I think I will buy some. I can then take more of my tight KN95 masks, cut the existing earloops, and tie in an extra inch or two of this elastic to get the length right for my head size.

ENDNOTE

[1] Some studies on masks:

(A) https://pubmed.ncbi.nlm.nih.gov/32845196/  Kim, et al. 2020. They had seven Covid-inflected patients cough five times with various masks on, and with petri dish sitting in front of them to catch germs. A surgical mask did no better than no mask at all (3 out of 7 patients’ petri dishes got infected in both cases), whereas zero out of 7 patients’ petri dishes got infected for a full N95 respirator made by 3M (not a Chinese KN95) or for a Korean-made KF94 mask.

(B) https://www.acpjournals.org/doi/10.7326/M20-6817   Bundgaard, et al., 2020. Done in Denmark around April-June 2020. From 6000 participants, all of whom initially tested Covid-negative, half were randomly selected to wear standard surgical-type masks while in public and half to not wear masks. (These are the usual rectangular masks that do not seal tightly to the face). Incidence of Covid infection after about a month was assessed for each group. For mask-wearers, the infection rate was about 1.8% versus 2.1% for the non-masked group. According to the standard statistical definitions, this was not enough to show that wearing that type of mask gave significant protection against becoming infected. That said, the difference between the 1.8% and the 2.1% is compatible with a 46% reduction to a 23% increase in infection on 95% confidence intervals. Depending on how you want to slice the numbers, it seems fair to say that there may have been “some” effect of the masks here. Also, it should be noted that this study did not test whether wearing a surgical mask would help keep an infected person from spreading the disease (I suspect the answer to that would be “yes, sort of”).

(C) https://pubmed.ncbi.nlm.nih.gov/33087517/ Ueki, et al., 2020. They used two full size human mannequin heads, and tied masks on their faces. The “Spreader” head was piped to have a stream of covid-aerosol-laden air coming out of its mouth. The “Receiver” head had a pipe that pulled air in through its mouth and through a gelatin membrane filter to collect the covid viruses that made it through the masks. Some of the results are shown below. I am not sure how to summarize them accurately in a few words. Note that these plots are on log scales, so small visual differences in the bars are actually big (see the numbers at the bottom of the bars). It seems clear that the cloth (cotton) and the surgical masks blocked some virus spreading compared to no masks, but a full N95 mask was much more effective (the N95 was tested with its edges naturally resting on the contours of the mannequin face, and also “fit” with the edges sealed against the face with adhesive tape). A KN95 or KF94 mask was not tested here.

POSTSCRIPT

After publishing this POST, I noted Jeremy Horpedahl’s post from last week, suggesting that the costs of wearing masks may be worth it even if they give a 10-15% decrease in viral incidence. Jeremy referenced an article by Bryan Caplan who questions the trade-offs with wearing masks having only marginal effectiveness vs. the discomfort and the dehumanization of having people’s faces obscured. Caplan in turn referenced a survey of research by Jeffrey Anderson (August, 2020) which summarized many real-life randomized controlled trials with populations wearing/not-wearing masks (presumably the surgical kind, not N95/KN95/KF94 better-sealing types) which found generally no benefit to wearing these types of  masks in reducing the incidence of virus transmission. (These studies were mainly pre-Covid, dealing with SARS and other viruses). This overall result is roughly consistent with the Danish study mentioned above, which did not find a significant difference for using those types of masks to protect from viral infections.

Would You Pay $3,000 to Not Wear a Mask?

How well do masks work at preventing disease transmission? This is a question that many of us have been asking throughout the pandemic. I have been trying to read as much about mask effectiveness as I can (for example, here’s a Tweet of mine from way back in June 2020). I think the bottom line is that, if you want really good RCTs of mask use during the COVID pandemic, there is surprisingly little evidence in any direction. But there are lots of studies, less well done but still OK, suggesting that masks do provide some protection.

I don’t want to wade into all of that research here, because Bryan Caplan has been doing that lately himself. His reading of the literature is that masks aren’t a silver bullet, but he suspects “that masks reduce contagion by 10-15%.” Still he thinks that the costs of masks (inconvenience, discomfort, and dehumanization) are large enough that they don’t pass a cost-benefit test. But this seems like a very strange conclusion given that he suspects masks reduce contagion by 10-15%! So let’s be explicit about the cost-benefit analysis.

[I am assuming that reducing contagion by 10-15% means 10-15% fewer cases and deaths. I see this as a bare minimum, since contagious disease can follow exponential growth trends, so 10-15% less contagion could mean that cases/deaths are reduced by more than 10-15%, but I’m making a simplifying assumption and the hard case.]

Quantifying the costs of the pandemic deaths is tricky, and it’s something that Bryan and I have debated before. Perhaps this is just a rehash of that debate (Bryan is highly skeptical of the VSL estimates), but I think it’s worthwhile to plug in some numbers.

What numbers should we use?

Continue reading

Coase and COVID

Update: I added a comment on the post to clarify why I don’t think that having seniors stay at home is the correct Coasean solution. In short: social isolation has high costs!

Bryan Caplan has an interesting post on COVID and reciprocal externalities. Caplan starts off with the straightforward Coasean statement: “Yes, people who don’t wear masks impose negative externalities on others. But people who insist on masks impose negative externalities, too.”

For those not familiar with Coase’s 1960 article, one of his fundamental insights about property rights is that when property rights are not clearly defined, both parties can be imposing costs on one another. The externalities are reciprocal, not just in one direction. The efficient outcome, when bargaining is not possible, is to allocate the property right such that the “least cost avoider” is the one that adjusts their behavior. In other words, you allocate the property right to the party who would obtain the property right if bargaining were possible.

But Caplan uses this Coasean framework to come to the opposite conclusion that I would. Why?

Continue reading