Iran on Markets, Markets on Iran

We’re bombing Iran, and Iran is now bombing most of its neighbors. Oil prices are up ~20% since the bombing began last weekend, and stocks are down.

Iranian “Supreme Leader” Khamenei is now dead. Prediction markets sort of saw this coming; I mentioned here a month ago that markets thought it more likely than not that Khamenei would be “out of office” this year.1

Real-money US-regulated exchanges can’t directly cover the war, but others can and do, such as the international Polymarket:

Polymarket’s argument for why they offer these markets

This market shows that regime change is likely, but will take time- a 51% chance by the end of the year, but only a 13% chance by the end of the month.

How would this be achieved? Markets see a 60% chance that there will be US troops in Iran this year, though this market could be triggered by just a few special forces operators, or by troops visiting for humanitarian purposes after domestically-driven regime change. There will likely be a US-Iran ceasefire by the end of May. It’s not clear at all who will be running Iran at the end of the year:

Iran is far from the only country whose future leadership is unclear. Last month I noted that the current leaders of Britain, Hungary, and Cuba would likely be out of office by year end. These are all now looking even more likely than they did a month ago:

So I’ll repeat:

Myself, I find most of these market odds to be high, and I’m tempted to make the “nothing ever happens” trade and bet that everyone stays in office. But even if all these markets are 10pp high, it still implies quite an eventful year ahead. Prepare accordingly.

  1. US-regulated exchanges can’t offer markets on death. Kalshi’s rules stated that if Khamenei died, the market would refund everyone at current prices rather than paying as if he were “out of office”. When he died many people got mad at Kalshi- some who had bet he’d be “out of office” and were mad that they weren’t paid at 100%, others that Kalshi was offering something too close to a death market- “how else would he lose power” (even though Maduro and Assad provide clear recent examples) ↩︎

Humanity’s Last Exam in Nature

Last July I wrote here about “Humanity’s Last Exam”:

When every frontier AI model can pass your tests, how do you figure out which model is best? You write a harder test.

That was the idea behind Humanity’s Last Exam, an effort by Scale AI and the Center for AI Safety to develop a large database of PhD-level questions that the best AI models still get wrong.

The group initially released an arXiV working paper explaining how we created the dataset. I was surprised to see a version of that paper published in Nature this year, with the title changed to the more generic “A benchmark of expert-level academic questions to assess AI capabilities.”

One the one hand, it makes sense that the core author groups at the Center for AI Safety and Scale AI didn’t keep every coauthor in the loop, given that there were hundreds of us. On the other hand, I’m part of a different academic mega-project that currently is keeping hundreds of coauthors in the loop as it works its way through Nature. On the third, invisible hand, I’m never going to complain if any of my coauthors gets something of ours published in Nature when I’d assumed it would remain a permanent working paper.

AI is now getting close to passing the test:

What do we do when it can answer all the questions we already know the answer to? We start asking it questions we don’t know the answer to. How do you cure cancer? What is the answer to life, the universe, and everything? When will Jesus return, and how long until a million people are convinced he’s returned as an AI? Where is Ayatollah Khamenei right now?