Economist Writing Every Day

I was once told that papers are never finished, only surrendered. It’s one of those turns of phrase who’s observational accuracy has only increased. I don’t know that I’ve felt good about submitting a paper for review in over a decade, and that includes the one’s that were accepted and subsequently published.

When I submitted papers early in my career I felt great. There was both a sense of accomplishment and eagerness to learn what the reviewers might think, a hopeful optimism. That eagerness didn’t reflect overwhelming confidence so much as naivete as to what the review process entailed. Now I know too much.

What I know, what I always know, is that more could be done. More alternative empirical specifications could be added to the robustness section. Newer models could be considered for the underlying mechanism. Older models too. Different literatures could be engaged and contended with. Summary statistics could be visualized. Specifications could be bootstrapped, a different identification stratgy used. I never applied for administrative data in Denmark. Wait, they don’t have this policy in Denmark. I could have tried Sweden. Or Dallas. Wasn’t there a close election in Baltimore in 1994?

This isn’t a rant or lament about the journal reviewing process. For every petty or uninformed referee report I’ve received in my career I’ve received three that were entirely fair and one that was so good the reviewer deserved to go in the acknowledgements of future drafts. This is more a reflection on a trap born of our own knowledge and imaginations.

There are so many tools at our disposal, so many data sets, so many options that I worry that we are collectively succumbing to a paradox of choice. The paradox of choice, for those who do not recall, was a theory that suggested that the number of options facing consumers was net lowering their utility because of the search and decisionmaking costs those options entailed. I think this theory is deeply wrong, but I am also going to be incredibly unfair to it here and simply dismiss it out of hand as a consumer theory. Instead, I want to consider a more collective application to the modern social scientific enterprise.

Every research paper is an attempt to contribute new ideas and refine old ones. There is occasional handwringing over the paucity of new ideas in economic research and the abandonment of broad swaths of traditionally difficult economic subjects. Explanations for these pathologies tend to be more sociological than economic in construct, invoking political preferences or mood affiliation. Others focus on the institutions of academic research, specifically faculty hiring and tenure. I’d like to add the paradox of choice to the mix.

There are countless methodological, theoretical, and rhetorical choices that can be made that will result in nearly identical research contributions. If your aim is to contribute a wholly new idea, then every one of those choices comes with the opportunity cost of the countless alternatives. If, on the other hand, your contribution is a refinement of a pre-existing idea in an already rich vein of research, then the choices you made are the contribution. For refinements, the choices made are a reason to recommend acceptance of your paper. For newer, more original contributions, your choices can be more easily framed as reasons to reject it. A more cynical academic might fear that the more original the contribution, the more likely the referee is to succumb to the Nirvana fallacy, disapproving of your paper’s choices relative to an imagined paper more perfectly in line with the choices the referee would have made if they had thought of the idea.

Now consider these two mechanisms in parallel for a young researcher. Not a wonderkid that faculties on other continents are already talking about. Consider an above average newly minted PhD from a top 25 economics department. They are executing their first research project since accepting a tenure track position, a defined question with explicit policy relevance. There are dozens of data sets they could pursue, hundreds they could build, and a countless number they could imagine feasibly existing. They could pick a workhorse model or contruct an entirely new pathway forward from dozens of building blocks. There are 3-4 “hot” identification strategies in their field, but they could also consider something off the beaten path.

Research projects aren’t binary constructs, “new” or “refining contributions”, but it’s not unreasonable to place their contributions on a spectrum of “entirely new” (i.e. Newtonian physics) to “marginal refinement” (i.e. weakening the asssumptions in a minor mathematical proof). From the start, our new faculty member will observe the inherent riskiness of overdifferentiating from the field, turning every choice into a reason referees might reject their paper. This will push them down the spectrum towards marginal refinements. Then they will start the iterative process of executing and writing up their research.

As they execute their analysis they will see the forking paths of alternative choices. Different specifications will be added to robustness tables. Alternative models will merit their own appendix. They will begin to write defensively, trying to anticipate and refute arguments from their mental model of a reviewer. They will try to divert an imagined conversation away from the conclusion that the choices made in the paper are wrong. The risk of newness only becomes starker. There must be, and remains, the contribution in the paper, but it will become narrower, buttressed on all sides by the rising masonry of appendices and references, it’s only weakness the narrow channel through which its contribution is made. This iterative process will continue until the opportunity cost of time not spent on their next project forces the unconditional surrender of their paper to that still unvanquished tyrant, diminishing returns.

All of this is weighing on young faculty shoulders. A million choices to be made, a million reasons to be rejected. So what do you do? You find your tribe. A tribe not based in the schools of thought that dominated the 1970’s but in the schools of methodological choices. This is how we estimate gravity models of trade. This is how we estimate monopsony rents. This is how we model the impact of the minimum wage on employment. If you want to be cynical, there are no doubt similar tribes of policy outcomes, but I don’t think those are what haunt the face-on-desk stress dreams of assistant professors working on a Sunday night.

We can get more new ideas the same way we can get bolder, more enthusiastic young researchers. Not by reducing their choices, but by lowering the price of those choices. Easier said than done, and maybe I’ll write up some thoughts on how lower the prices of researcher choices, but the first step is likely cultural i.e. I have no idea how to pull it off. The most important step may simply be reorienting how we read papers, shifting the focus from “What did the authors do right or wrong?” to “What do we learn from this?”