Author: James Bailey

Most Published Research Findings Are Directionally Correct

James BaileyLeave a comment

As a new quick rule of thumb inspired by the Nature papers, you could do worse than “cut estimated effect sizes in half”. If a published paper says that a college degree raises wages 100%, then chances are the degree really does raise wages, but more like 40–50%. In 2005, John Ioannidis said that “most published research findings are false”. By 2026, we seem to have improved to “most published research findings are exaggerated.”

That’s the conclusion of my piece out today at Econlog: “Is Economics Finally Becoming Trustworthy?“

There’s plenty of both good and bad news for economics and the social sciences in both my piece and the Nature special issue it describes. It’s kind of like the Our World in Data motto:

In short, our attempt to replicate hundreds of papers showed that published social science results shouldn’t be trusted precisely today, but they seem to be getting more reliable over time, and they are much more reliable than chance. Economics and political science look the best, though we are still very far from perfect:

You can read the full piece here.

How To Blog

James Bailey1 Comment

Scott Alexander of Astral Codex Ten shares writing advice for aspiring bloggers here. He is very much worth listening to on this- he earns more from Substack subscriptions than he does as a doctor, and even that undersells his influence. He distinguishes two types of writers worth reading:

If you write brilliantly, the world will read your review of watching paint dry. If you’re a normal person, you have to content yourself with reaching the limited subset of people who are interested in your topic.

There’s one new brilliant person per year, maybe less. But dozens of bloggers – dozens! – reach the lower bar of providing value to the people who care about the same things they do. That could be you!

Here we just aim to be the second sort of writers. For me Scott is the first sort, and if that’s true for you too, you should just go read his whole 6000 word post. But many people find his writing overly long, so for you I offer a summary with highlights:

TLDR: Honesty is key. Write what you know and care about. Write what you really think. Write original by being, doing, and thinking original:

The problem with bloggers is that they read blogs. So when they want to write about their exciting new opinion, it’s probably an opinion they got from a blog.

The problem with blog readers is that they also read blogs. So when they read your opinion that you got from a blog, they’ll think “Oh yeah, I read that on that other guy’s blog last week.”

The end result is that everyone talks about the same thing over and over, hoping they can be the one to educate the last person on Substack who hasn’t heard that social priming studies don’t replicate, or that stairwell restrictions are bad housing policy….

If everything good in writing comes from contact with the world, then your goodness is proportional to how direct your contact is. Best-case scenario, you live with a Southeast Asian tribe yourself and report your results. Second-best case, you at least read the book by the guy who did that and form your own opinion. Third-best case, now you’re reading a blog post by someone who read the book, three levels distant from the world. But even that’s getting rarer. Now people are reading tweets by someone who read the review of the book by the person who met the tribe, and forming opinions based on those. At that point, almost all the work is being done by the prejudices of your sources, rather than brute facts about Southeast Asians.

You contribute to the blogosphere by injecting first-level facts about the world, or second-level primary sources by experts who have gotten the first-level facts. You draw down those contributions by playing too many games of telephone with popular topics that you got from the blogosphere itself.

Read Matt Levine:

You can write brilliantly about anything. Consider Matt Levine. I have no interest in finance. Even though I could make hundreds of thousands of dollars by understanding finance better, every time I consider doing this I bounce off the fact that all the relevant books include terms like “credit default swap” and “ergodicity”. But the first time I read Matt Levine’s finance newsletter, I thought “I am going to read this approximately every day for the rest of my life”, and I was right – even though I will never trade credit default swaps or do anything else that would make reading Matt Levine directly valuable to me.

Likewise, many discussions of Freddie deBoer start “I’m a huge fan of Freddie, even though I disagree with everything he says, and find him personally abrasive, and his topics are unoriginal and repetitive, and I hate him, and I hope he dies.” Then in what sense are you a fan? “Well, I read all of his posts.” Good work if you can get it.

I don’t think anyone wants to read our reviews of watching paint dry, but I do think we are sometimes the first to get to the bottom of something important, and I hope that’s part of what keeps you coming back:

There’s almost no topic so overdone that you can’t be the first person to do a good job writing about it…. When you demand even the flimsiest of details, nobody’s written a good blog post even on the ultra-controversial topics that everyone talks about every day.… Some of the posts I’m proudest of involved taking a topic everyone was talking about, then being (as far as I can tell) the first blogger to put in significant effort to see which side was right.

(Not the first person; often the answers are hidden in old scientific papers. But ordinary people won’t know about those papers unless someone blogs about them.)

Raise Rates- But Not Because Of Oil

James BaileyLeave a comment

Next week the Fed will almost certainly hold interest rates steady. Stephen Miran will probably dissent saying the Fed should be cutting rates. Kevin Warsh, Trump’s nominee for Fed Chair, would also like to see cuts. But other prominent voices think that rising oil and gas prices mean we should be raising rates.

I still think that rate hikes make more sense than cuts- but not because of oil. The high oil and gas prices we’re seeing are obviously driven by supply shocks from the Iran war- not increasing demand. Raising rates to fight an oil shock would mean repeating a classic mistake.

But raising rates to fight core inflation that is at 3% makes perfect sense. Especially when inflation (overall or core) hasn’t been at or below the Fed’s supposed 2.0% target in over 5 years, and market forecasts predict it will stay well above 2.0% for the next 5 years.

Especially when real GDP is growing, and NGDP is still above trend, and the unemployment rate is 4.3%. Financial conditions are so loose that stock markets are hitting all time highs in the middle of a war.

Various Taylor Rules suggest that the Fed Funds rate should be between 4.25% and 6.25%, but the Fed currently has us at 3.75%.

I see so many good arguments to raise rates- there is no reason to bring up a bad one like oil prices. If we must latch on to a headline to find the argument to raise rates, let’s focus on a shoe company’s stock going up 600% because they announced they were pivoting AI.

The Novelist Paradox

James Bailey5 Comments

If novelists are so smart, why don’t they succeed at much besides writing fiction?

When I read a good novel I think “the author must be very smart to be able to write this and understand people so well”.

But novelists tend not to be very successful at things in life other than writing fiction, certainly not at anything like the rate of people who write good non-fiction books.

Just off the top of my head, people who wrote good / highly acclaimed non-fiction books while also being highly successful in other fields:

  • Julius Caesar
  • Marcus Aurelius
  • Benjamin Franklin
  • Richard Feynman
  • Winston Churchill
  • Barack Obama
  • JD Vance
  • Many top economists (Keynes, Hayek, Friedman)

While off the top of my head, novelists who reached anything like that level of success in other fields include:

… No one?

The best that comes to mind is people that started philosophical movements related to their writing, like Ayn Rand, Scott Alexander, or Eliezer Yudkowsky. But that’s clearly a different kind of success than for most non-fiction authors. Likewise when I ask Claude the best examples I get are doctors, lawyers, and academics, not world leaders. I’ve been kicking this idea around for years but was inspired to finally write it down because I found out that before Ben Hur was a movie it was a wildly successful novel, and the novel was written by former Civil War general Lew Wallace (not a great general as they go, but its still impressive to be a general at all). But I still think that is the exception.

In fact, worse than just “not being world-changingly successful”, some of my favorite living novelists sometimes seem crushed by the weight of everyday tasks like giving public talks, maintaining relationships, or completing their work anywhere close to on time.

Naming the living novelists I’m thinking of would be mean so here’s F Scott Fitzgerald

The paradox: if novelists are so smart, why aren’t they more broadly successful?

Potential resolutions:

  1. I’m wrong and novelists actually are broadly successful.
  2. Novelists are so smart, but tend to have other deficits that keep them from being broadly successful, or from wanting to try, e.g. being neurotic introverts
  3. Novelists aren’t so smart, it’s more of a narrow skill that we shouldn’t expect to indicate general intelligence, like being good at painting or football.

The question can be flipped: why can’t / won’t many very successful people write fiction? Are they more grounded in the real world when it treats them well? I’m not ‘very successful’ but I write a lot. In my case I’m not convinced I could even write a bad novel, much less a good one. Wouldn’t know where to start.

Cunningham’s Law Update: John Giebfried writes in with excellent counterexamples along the lines of resolution #1-

Making Friends In Politics Is Possible

James BaileyLeave a comment

I knew getting involved in politics was a great way to make enemies, but it never occurred to me until I saw it in action that it can also be a way to make friends.

I’m still not very involved, even as academics go. I think many of us are a bit too eager to talk about political issues in general, but too slow to engage with the policy process in areas directly tied to our research. It’s hard to keep track of every relevant bill and proposed regulation, but I think we bring the most value when we’re the 3rd person to weigh in to share what the research says on an obscure topic, rather than the 3000th person to weigh in on a hot-button issue with a take that sounds just like everyone else on the same side.

My biggest surprise when testifying in state legislatures or public hearings has been that friends follow through while opponents don’t. People who disagree with me will say so at the time, then leave it at that. But people who agree with me will follow up afterwards with messages like “thanks for saying that” or “let’s get coffee”, or let me know when related issues come up.

Perhaps this is unusual, just some good luck in a small sample size, or a reflection of the fact that I only weigh in on relatively obscure issues far from the culture war. But again, I never even thought of this as a possibility. I still wouldn’t run for office any time soon. But if this wasn’t already obvious to everyone else, I encourage you to add this as one term in your own equation as you weigh the pros and cons of political engagement: “nudge the policy process in directions you like” + “engagement takes time and energy and makes enemies” + “maybe friends too”.

How Much To Trust Research Papers? My Rules Of Thumb

James Bailey1 Comment
  1. Trust literatures over single papers
  2. Common sense and Bayes’ Rule agree: extraordinary claims require extraordinary evidence
  3. Trust more when papers publicly share their data and code
  4. Trust higher-ranked journals more up to the level of top subfields (e.g. Journal of Health Economics, Journal of Labor Economics), but top general-interest journals can be prone to relaxing standards for sensationalist or ideologically favored claims (e.g. The Lancet, PNAS, Science/Nature when covering social science)
  5. More recent is better for empirical papers, data and methods have tended to improve with time
  6. Overall effects are more trustworthy than interaction or subgroup effects, the latter two are easier to p-hack and necessarily have lower statistical power
  7. Trust large experiments most, then quasi-experiments, then small experiments, then traditional regression (add some controls and hope for the best)
  8. The real effect size is half what the paper claims

That last is inspired by a special issue of Nature out today on the replicability of social science research. An exception to rule #4, this is an excellent project I will write more about soon.

Experimental Banking Reveals the Value of Leisure

James Bailey1 Comment

In 2014 India required banks to offer no-cost accounts. This led hundreds of millions of people to open bank accounts for the first time, and more than doubled the number of Indian women who had a bank account:

This increased households’ collective ability to save and borrow, but didn’t shift decision-making power towards women despite the larger change for them. That is the finding of a paper by Tarana Chauhan, a Brown University postdoc who is currently on the job market. The paper is a well-executed example of a difference-in-difference analysis of observational data- that is, carefully examining data that other people generated to examine events that help establish causality. But the validity of difference-in-difference strategies in separating correlation from causation can always be questioned, and always is in economics seminars.

So Dr. Chauhan, this time with coauthors Berber Kramer, Patrick Ward and Subhransu Pattnaik, followed up by directly running an experiment. They got a company to offer subsidized loans to hundreds of randomly selected Indian farmers, then surveyed the farmers to see if they behaved differently than a control group that didn’t get loans. The loans carried a 14% interest rate, which seems high to Americans but was apparently 10pp lower than the other options available in India. They wanted to know whether farmers would use the loans to improve farm productivity, and whether this would have any differential effects on women.

The first stage of the experiment worked: households took the loans and got more engaged with the financial system.

Some used the money for smartphones:

But for the most part they seem not to have spent the money on farming- they didn’t buy significantly more land, seeds, fertilizer, or farm equipment. They did spend more on “non-farm business equipment” and “large consumer durables”. Despite not producing more food themselves, they reported higher food security. Income stayed flat, but women were able to shift some time away from work and toward leisure:

I find these results surprising given how poor the households receiving the loans are. They earn the equivalent of about $1,000/yr, putting them around the global “extreme poverty” line. At that income level I’d think they would value additional income highly relative to leisure, and yet when they get the loan, work time goes down and leisure time increases. Could it really be the case that they’ve already hit their income target, and are on the backward bending part of the labor supply curve? Some other possibilities are that they don’t expect that investing in farming would increase yields enough to be worthwhile, or that they worry any increased income would be taken away through explicit or implicit taxes. But the households generally seem better off as a result of the loan.

The other surprise- enough of the loans were paid back that the lenders made a profit despite the research pushing the interest rate below-market.

Does Broadband Bring Jobs?

James BaileyLeave a comment

No, according to a new paper from the University of Georgia’s Michael Kotrous.

Many people expected it to, partly by thinking about the jobs that could benefit from faster internet, and partly by looking at the experience of Chattanooga, Tennessee. Chattanooga was the first major city to get gigabit-speed broadband, and they did see a huge improvement in the labor market right afterwards:

But as the graph shows, the introduction of broadband there coincides with the end of the nationwide Great Recession. Was the boom in jobs after 2009 because of the broadband, or would it have happened anyway as party of the recovery from recession? A synthetic control strategy shows that Chattanooga’s recovery was pretty typical for cities like it, so the broadband angle probably didn’t do much:

This might seem like a historical curiosity about one city, but the federal government is currently trying to spend $42 billion to expand broadband to more places, partly motivated by the idea of bringing jobs. I thought the Broadband Equity Access and Deployment Program‘s big problem is how slow it is- Congress created with the Infrastructure Investment and Jobs Act of 2021, but money didn’t start getting sent out until late 2025, and it could be many more years before it leads to any useable broadband. Even then it now seems unlikely to bring jobs, though there could be other benefits.

This paper’s author Michael Kotrous is currently on the economics job market. As his former professor and coauthor, I recommend hiring him if your school gets the chance.

Is a US Oil Export Ban Coming?

James Bailey3 Comments

The Iran regime’s military strategy seems to be that by bombing the oil infrastructure of their neighbors and neutral shipping, US gasoline prices will go so high that Americans will demand an end to the war.

How many Americans would be willing to pay $6/gallon gas for months for a ~50% chance of toppling a regime that oppresses 90 million people and destabilizes its region on the other side of the world? Probably only a minority of voters, especially when the President didn’t make the case to the American people or Congress beforehand.

But the US produces more than enough oil for its own needs. Why does the Strait of Hormuz being closed mean higher gas prices here? Only because US oil companies can sell to global markets, and they won’t choose to sell a barrel of oil to a US refiner for $60 when they could sell it to a foreign refiner for $100. If the government took away the foreign option, US oil producers would sell to US refiners at prices consistent with pre-war sub-$3/gallon gasoline.

Naturally there would be costs to an export ban. US oil producers would miss out on windfall profits, while Russian producers would benefit. Foreign customers of US oil, many of them in allied countries, would be angered by the missed shipments and global oil prices would soar further.

But if the US administration wants to avoid a midterm wipeout driven by high gas prices, I see only 3 options:

  1. Get lucky and see the Iranian regime fall quickly
  2. Negotiate an end to the war quickly (which might itself be unpopular if they can’t get a good deal) or just declare victory and go home (but its not clear whether Iran would re-open the strait now just because the US stopped bombing)
  3. Restrict Exports

I say “restrict” not “ban” because I don’t think a complete export ban is necessary to stabilize US prices. You could instead do an export tax (high enough to stop many exports but low enough to allow the buyers with the highest values / fewest alternatives to stay in the market), or you could do a ban but allow a few export waivers for favored buyers or sellers (which seems like Trump’s style), or similarly a quota limiting exports to a certain number (say, limit each company’s monthly exports to 90% of their volume in the same month last year).

This has an obvious precedent: the Biden administration stopped issuing new permits to export liquified natural gas in 2024 to prevent prices spiking here during the Ukraine war (which led to even higher prices for our European allies). But a total ban on oil exports would be a much bigger deal.

Will the Trump administration actually try something like this? It will be an interesting test of US political economy to see what happens when the interests of the military-industrial complex conflict with the interests of oil producers.

Iran on Markets, Markets on Iran

James BaileyLeave a comment

We’re bombing Iran, and Iran is now bombing most of its neighbors. Oil prices are up ~20% since the bombing began last weekend, and stocks are down.

Iranian “Supreme Leader” Khamenei is now dead. Prediction markets sort of saw this coming; I mentioned here a month ago that markets thought it more likely than not that Khamenei would be “out of office” this year.1

Real-money US-regulated exchanges can’t directly cover the war, but others can and do, such as the international Polymarket:

Polymarket’s argument for why they offer these markets

This market shows that regime change is likely, but will take time- a 51% chance by the end of the year, but only a 13% chance by the end of the month.

How would this be achieved? Markets see a 60% chance that there will be US troops in Iran this year, though this market could be triggered by just a few special forces operators, or by troops visiting for humanitarian purposes after domestically-driven regime change. There will likely be a US-Iran ceasefire by the end of May. It’s not clear at all who will be running Iran at the end of the year:

Iran is far from the only country whose future leadership is unclear. Last month I noted that the current leaders of Britain, Hungary, and Cuba would likely be out of office by year end. These are all now looking even more likely than they did a month ago:

So I’ll repeat:

Myself, I find most of these market odds to be high, and I’m tempted to make the “nothing ever happens” trade and bet that everyone stays in office. But even if all these markets are 10pp high, it still implies quite an eventful year ahead. Prepare accordingly.

  1. US-regulated exchanges can’t offer markets on death. Kalshi’s rules stated that if Khamenei died, the market would refund everyone at current prices rather than paying as if he were “out of office”. When he died many people got mad at Kalshi- some who had bet he’d be “out of office” and were mad that they weren’t paid at 100%, others that Kalshi was offering something too close to a death market- “how else would he lose power” (even though Maduro and Assad provide clear recent examples) ↩︎