The most interesting new paper in months dropped in last week: “Does Income Affect Health? Evidence from a Randomized Control Trial of a Guaranteed Income“

All the broad strokes are there in the abstract: $1000 per month, 2,000 participants (half treated), for three years. It’s the biggest, highest salience experimental test of a universal basic income program to date. There’s a lot of detail, but the broad strokes findings are that nothing happened. That is to say, there was a lot of precise null effects. And that is absolutely fascinating! I’ve gone back in forth on my feelings about a large scale UBI policy, and this is certainly more evidence that gives me pause, but my biggest takeaway is that policy research really should culimate in a series of field experiments whenever possible. Not because of identification or external validity or any of the other reasons economists fight in intellectual perpetuity, but because it’s easier to accept null results as sufficiently precise. It’s easier to acknowledge and accept that there is no observed effect because the treatment mechanism truly had no net effect within an experimental design.

Conducting research using observational data to produce causally identified conclusions is to fight a battle on multiple fronts. These fronts usually relate to the all the possible sources of bias, of endogeneity, within your analysis. You’re observing x causing y to increase, but the reality is that x is correlated with z, and that is what is actually causing y to increase. That’s a tough fight, believe me, as hypothetical sources of varying degrees of plausibility are hurled at your analysis from all directions. But at least there is an argument to be had. There’s something to fight against and over.

Null results face a far more insidious argument: there’s just too much noise in your analysis. Too many sources of variation, to much measurement error, too much something (that I don’t have to bother unpacking) and that’s why your standard errors are too big to identify the true underlying effect. There’s also a simple, and annoying, institutional reality: there’s no t-test for a precise zero. There’s no p value, so threshold for statistical significance that says this is a “true zero”. All we can say is that the results fail to reject the null. It’s subjective. And in a world of 2 percent acceptance rates at top journals, good luck getting through a review process where the validity of statistical interpretaton is assessed in a purely subjective manner.

Field experiments enjoy far more grace with null results. As random control trials, they can argue that their null effects are, in fact, causally estimated. If conducted within sufficient power (i.e. number of observations relative to feasible impact), then the results are simply the results. There’s no arguments to be had about instrumental variables, regression discontinuity cutoffs, or synthetic control designs. Measurement error will rarely be a problem given an appropriate design. External validity…well there’s no getting around external validity gripes, but should those concerns appear then the opposition has already accepted the statistical validity of your null results. You’ve already won.

I’m not puffing up my own team here, either. I’ve conducted several lab experiments, but never a field experiment. They are large, lengthy, and costly endeavors. I aspire to run a couple before my career runs its course, but I’ve built nothing on them to date. But they’ve grown in my estimation, even admidst concerns over participant gaming and external validity, precisely because you can run your experiment, observe no measurable impact on anything, and proclaim in earnest that that is precisely what happened. Nothing.

A President has about a 1 in 6 chance of dying while in office*, making it by far the most dangerous job in the United States. Sources of danger include stress, public speaking in inclement weather, and targeted violence. They clearly age rapidly, estimated by at least one doctor to be at twice the normal rate. Given this danger, their compensation seems comparably modest. One possible explanation is asymmetric information, though this should diminish after four years on the job, which leaves open the possibility of coercion. In what looks to me to be a truly awful and underpaid occupation, that so few Presidents step down after their first term is at least suggestive evidence that many Presidents serve under the duress of their parties. If it was any other job, OSHA would be inundated with complaints.

I’ve often said that Nixon resigning was the greatest day in the history of American democracy. The commander in chief of the most powerful army in the history of the world to that point was asked to give up power. And he did, with no blood shed, a moment arguably unprecedented in the history of the world. Yesterday the President chose not to run for re-election. He was, with little doubt, pressured by members of his party to do so. This is a truly great thing. Regardless of whether or not he would have won or lost, been a good President or not, the point is that powerful people wanted to influence a decision made by the most powerful person in the world and did so, again without any threat of violence. This is as good a sign as you can ask for that democracy is maintaining its single greatest advantage over other forms of government: the peaceful transition and acquisition of power.

*Yes, I know the 1 in 6 number is based on deaths throughout the 250 years and that it, like other jobs, has probably gotten safer. I still think the relative danger ranking remains accurate but, in any case, maybe just learn to play along.