The conventional wisdom among economists is that large, liquid asset markets like the US stock market are incredibly informationally efficient. The Efficient Market Hypothesis (EMH) means that these markets near-instantly incorporate all publicly available information, making future prices essentially impossible to predict (a random walk with drift). As a result, economists’ investment advice is that you shouldn’t try to beat the market, because its impossible except through luck; instead you should aim to tie the market by owning most all of it via diversified low-fee index funds (e.g. SPY or VT).

This idea usually sounds crazy when people first hear it, but it works surprisingly well. You’d think that at least half of participants would beat the market average each year, but active strategies can generate such high fees that its actually much less than that. Further, people who beat the market one year aren’t more likely than average to beat it the next, suggesting that their winning year was luck rather than skill. Even Warren Buffet, who economists will sometimes concede is an exception to this rule, thinks that it is best for the vast majority of people to behave as if the EMH is true:

In 2008, Warren Buffett issued a challenge to the hedge fund industry, which in his view charged exorbitant fees that the funds’ performances couldn’t justify. Protégé Partners LLC accepted, and the two parties placed a million-dollar bet.

Buffett has won the bet, Ted Seides wrote in a Bloomberg op-ed in May. The Protégé co-founder, who left in the fund in 2015, conceded defeat ahead of the contest’s scheduled wrap-up on December 31, 2017, writing, “for all intents and purposes, the game is over. I lost.”

Buffett’s ultimately successful contention was that, including fees, costs and expenses, an S&P 500 index fund would outperform a hand-picked portfolio of hedge funds over 10 years. The bet pit two basic investing philosophies against each other: passive and active investing.

This has been the approach I’ve taken for most of my life, but over the last 3 years I’ve gone from ~99% believing in efficient markets to perhaps ~80%. Missing on crypto felt forgivable, since it was so new and unusual; I recognized that in the early days of a small, illiquid market the EMH might not apply, I just misjudged what counted as “early days” (I figured that by 2011 “everyone” knew about it because Bitcoin had been discussed on Econtalk; its up ~1000x since).

But with the Covid era the anomalies just kept piling up. All through February 2020, the smart people on Twitter were increasingly convincing me that this would be a huge pandemic; the main thing reassuring me was that stocks were up. But by late February they finally started crashing; instead of trusting the markets, I apparently should have trusted my own judgement and bought puts. Then investors starting buying the “wrong” Zoom instead of the one whose business benefitted from Covid:

Then we saw “meme stock mania” with many stocks spiking for reasons clearly unconnected with their fundamental value. Many at Wall Street Bets were clear that they were buying not because of business fundamentals, or even because they thought the price would go up, but because they liked the company, or wanted to be part of a movement, or wanted to send a message, or “own the shorts”.

Anecdotes got me to start taking some of the anti-EMH economics literature more seriously. For instance, Robert Shiller’s work showing that while it might be near-impossible to predict what a single stock will do tomorrow better than chance, predicting what the overall market will do over the longer run is often possible.

By revealed preference, is still mostly buy the EMH. About 80% of my net worth (not counting my home) is in diversified low-fee index funds. But that means 20% isn’t; its in individual stocks or actively traded ETFs with more-than-minimal fees. Why do this? I see 4 reasons buying individual stocks isn’t crazy:

1. Free trading: Buying a bunch of individual stocks used to incur huge fees. Now, many brokerages offer free trading. Even if the EMH is true, buying a bunch of individual stocks won’t lose me money on average, just time.
2. Still diversified: Buying into active funds instead of passive ones does tend to mean higher fees, and that is a real concern, but they do still tend to be quite diversified. Even buying individual stocks can leave you plenty diversified if you buy enough of them. Right now I hold about 45, with none representing more than 0.5% of my portfolio; one of them going bankrupt causes no problems. If anything I’m starting to feel over-diversified, and that I should concentrate more on my highest-conviction bets.
3. Learning: Given the above, even if the EMH is 100% true, my monetary losses due to fees and under-diversification will be tiny. The more significant cost is to my time- time spent paying attention to markets and trading. This is a real cost, enough that I think anyone who finds this stuff boring or unpleasant really should take the conventional econ advice of putting their money in a diversified low-fee index fund and forgetting about it. But I’m starting to find financial markets interesting, and I think keeping up with markets is a great way to learn about the real economy- they always suggest questions about why some companies, sectors, factors, or countries are outperforming others. In some EMH models, the return to trading isn’t zero, but instead is just high enough to compensate traders for their time. In this case, people who find markets interesting have a comparative advantage in trading.
4. Outperforming Through New Information: All but the strongest version of the EMH suggests that those with “private information” can outperform the market. Reading about the very top hedge funds I think they really are good rather than lucky, and the reason is that they have information that others don’t. Sometimes this is better models but often it is simply better data; Jim Simons got historical data on markets at a frequency that no one else had, and analyzed it with supercomputers no one else had. That’s a genuine information advantage, and I don’t think it’s a coincidence that he wound up with tens of billions of dollars. This should be incredibly encouraging to academics. We can’t all be Jim Simons (who was a math professor and codebreaker before starting Renaissance Technologies; Ed Thorpe was another math prof who got rich in markets), but discovering and creating private information is exactly what we do all day as researchers. My hard drive and my head are full of “private information” that others can’t trade on; of course right now most of it is about things like “how certificate of need laws affect self-employment” that have no obvious connection to asset prices, and there is a lot more competition from people trying to figure out markets than from people trying to figure out health economics. But discovering new information that no one else knows is not only possible, it is almost routine for academics, and its not crazy to think this can lead to outperforming the market.

Overall I think economists have gone a bit too far talking themselves and others out of the idea that they could possibly beat the market. I’ll discuss some more specific ideas in the next few weeks, but for now I leave you with 3 big ideas: you can’t win if you don’t try; winning is in fact possible; and if you are smart about it (avoid leverage, options, concentration) then defeat is not that costly.

Disclaimer: This is not investment advice. I say this both as a legal CYA, and because I don’t (yet?) have the track record to back up my big talk