Economist Writing Every Day

Will AI kill the research paper?. I don’t know, probably not. But I do know that what has constituted a research paper has changed many times before and will change many times again.

Before the the 1940’s, economics research papers were largely prose. Analytic in nature, sure, but prose. Some graphs, maybe a box. A little math, but math largely for the sake of demonstrating logical relationships. Then Samuelson hit, reframed economics as thermodynamics and differential calculus. What was previously a research paper was was now a polemic, a monograph at best. Thought experiments were out, high theory was in.

This era of high theory flourished in the 70s, the math changed, and at some point computers arrived with the possibility of data sufficiently rich and numerous you couldn’t just plot all of the observations in Figure 1. That data couldn’t stand on its own, though. To be a credible publication you really needed to bundle your analysis with some theory that generated testable predictions. Pure theory papers gave way to an era of applied imperialism as economic models found themselves applied to every quantified context under the social scientific sun.

Causal identification became a thing of interest, and we got really good at telling stories again. Specifically, stories about instrumental variables. You needed a story to convince anyone, but we told so many that some folks started to notice that these stories were often pretty weak. That, in part, turned up the heat on a credibility revolution that was already in swing, which meant now you needed even better data and you needed to defend you identification strategy to the death. What was a paper before was now an embarassment you should probably consider retracting (nb: no one retracted anything, but that doesn’t mean people were suggesting it behind their backs).

Which kept rolling in data set after data set until we woke up one day and realized you either need to go out in the world and create your own actual experiment (nothing quasi- about it) or you needed to cultivate access to better…no, better…no, the very best-est, most detailed and granular administrative data ever, preferably a universe if possible. Data so perfect as to allow for contributions unassailable in their legitimacy. Do you have friends at the Danish Census? If you want tenure you should probably start flirting with someone at the Danish Census.

So a paper was a paper. Until it wasn’t a paper anymore. Until that wasn’t a paper anymore. Until that wasn’t a paper. The Recursive Dundee Theory of Research*, if you will. They all met the criteria of a contribution, until they didn’t.

So what does this mean for AI and research papers now? Well, if we look to thermodynamics in the 40s and cheap computing power in the 90’s for analogues, then I’d say it’s going to reshape the criteria for a contribution in no small part because it lowers the cost of mediocrity. Mediocre analysis will no doubt persist, but it will shift over into blog posts and journals no one ackowledges as legitimate. Do remember, please, that mediocrity is a relative concept. The quality of blog posts and publications in scam journals will likely massively improve as what can be accomplished in an afternoon’s work is radically increased. Don’t worry, I have no intention of improving beyond my current warm bath of blogging unremarkableness, but others will likely cave in to the pressure.

What about the papers in top journals, though? The papers Tyler is presumably talking about. Will AI kill those economic research papers? Probably not, but it will likely improve it significantly. Why? For the same reason that Michael Kremer says that technology and quality of life improve with the size of the human population. More people means more ideas, and there is nothing more important to economic growth than the sheer number of ideas. And no, I do not mean ideas generated by AI’s. I mean the raw number of researchers with the capacity to make major contributions is increasing dramatically because we’re all getting research assistants. We’re all getting copy editors. We’re all getting support. That’s how AI is going to change the research paper: by giving more ideas the support they need to reach the light of publication. The bar is going to get higher for the same reason that the level of sports improve as you widen the geography they pull from. There’s someone at a directional state school who didn’t get the placement they deserved out of grad school. Sure they have to teach a 3-3 load, but they’re licking their chops right now because they don’t need an army of grad assistants. Summer is here and they’ve got everything they need to make a contribution.

Or I don’t know. Maybe AI will do all of our thinking in 50 years. Forecasting technology beyond 5 years is like forecasting weather beyond 5 days: I can’t do it and neither can you.**

*Apologies to Justin Wolfers and all my Aussie friends for a bit of cultural appropriation. I promise to put some Vegemite on toast while enjoying a flat white and explaining Aussie Rules Football to a friend within 90 days.

**Except for Neil Stephenson. That guy’s the Warren Buffet of Sci Fi forecasting. Maybe he’s the one in a billion person actually experiencing one in a billion level luck, but that doesn’t make it any less impressive.