I like to take existing datasets, clean them up, and share them in easier to use formats. When I started doing this back in 2022, my strategy was to host the datasets with the Open Science Foundation and share the links here and on my personal website.
OSF is great for allowing large uploads and complex projects, but not great for discovery. I saw several of my students struggle to navigate their pages to find the appropriate data files, and they seem to have poor SEO. Their analytics show that my data files there get few views, and most of the ones they get come from people who were already on the OSF site.
This year I decided to upload my new projects like County Demographics data to Kaggle.com in addition to OSF, and so far Kaggle is the clear winner. My datasets are getting more downloads on Kaggle than views on OSF. I’ve noticed that Kaggle pages tend to rank highly on Google and especially on Google Dataset Search. I think Kaggle also gets more internal referrals, since they host popular machine learning competitions.
Kaggle has its own problems of course, like one of its prominent download buttons only downloading the first 10 columns for CSV or XLSX files by default. But it is the best tool I have found so far for getting datasets in the hands of people who will find them useful. Let me know if you’ve found a better one.
The Smoot-Hawley Tariff of 1930 was opposed by a thousand economists, but passed anyway, exacerbating the Great Depression. Now that the biggest tariff increase since 1930 is on the table, the economists are trying again. I hope we will find a more receptive audience this time.
The Independent Institute organized an “Anti-Tariff Declaration” last week that now has more signatures than the anti-Smoot-Hawley declaration, including many from top economists. One core argument is the sort you’d get in an intro econ class:
Overwhelming economic evidence shows that freedom to trade is associated with higher per-capita incomes, faster rates of economic growth, and enhanced economic efficiency.
But I thought the Declaration made several other good points. Intro econ textbooks say that tariffs at least benefit domestic producers (at the expense of consumers and efficiency), but in practice these tariffs have been mainly hurting domestic producers, because:
The American economy is a global economy that uses nearly two thirds of its imports as inputs for domestic production.
I get asked to sign a petition of economists like this every year or so, but this is the first one I have ever agreed to sign onto. Most petitions are on issues where there are good arguments on each side, like whether to extend a particular tax cut, or which Presidential candidate is better for the economy. But the argument against these tariffs is as solid as any real-world economic argument gets.
The full Declaration is quite short, you can read the whole thing and consider signing yourself here.
The tariffs still have me thinking about buying VIX calls and stock puts (especially when policy changes loom on certain dates like July 8th), and on the bigger question of finding the sort of investments that did well in the 1970’s, another decade of stagflation that was kicked off by a President who broke America’s commitment to an international monetary system that he thought no longer served us.
That’s how I concluded last week. So this week I’ll answer the question- what were the best investments of the 1970’s? When the dollar is losing value both at home and abroad, holding dollars or bonds that pay off in dollars does poorly:
Stocks can do alright with moderate inflation, but US stocks lost value in the stagflation of the 1970’s. Foreign stocks and commodities generally performed better. Real estate held its value but didn’t produce significant returns; gold shone as the star of the decade:
Gold is easy to invest in now compared to the 1970s; you don’t have to mess with futures or physical bullion, there are low-fee ETFs like IAUM available at standard brokerages.
Of course, while history rhymes, it doesn’t repeat exactly; this time can and will be different. I doubt oil will spike the same way, since we have more alternatives now, and if it did spike it wouldn’t hurt the US in the same way now that we are net exporters. Inflation won’t be so bad if we keep an independent Federal Reserve, though that is now in doubt. At any time the President or Congress could reverse course and drop tariffs, sending markets soaring, especially if they pivot to tax cuts and deregulation in place of tariffs ahead of the midterms.
Things could always get dramatically better (AI-driven productivity boom) or worse (world war). But for now, “1970s lite” is my base case for the next few years.
April 2nd drove the point home- when someone in a position to know tells you something big is coming on a precise date, it is a smart time to act. As opposed to doing what I have done, which is think about acting but ultimately do nothing.
Ahead of April 2nd this year, the White House made a big deal of how they had a big announcement on trade coming April 2nd and I thought “this could go better or worse than markets expect, but some big move is coming, this seems like a great time to invest in volatility through something like VIX options expiring shortly after April 2nd”, but then I didn’t buy VIX options. I didn’t totally understand how they worked, didn’t want to buy without finding out, and didn’t make time to find out. My instinct was right though- the VIX more than doubled last week, so the right options on it much more than doubled.
Ahead of the war in Ukraine in February 2022, US intelligence warned that Russia was planning to invade imminently, and I thought “they don’t have a great recent track record but it is very unusual for them to announce something so big will happen so soon, this is probably happening, this would be a good time to buy puts” but then didn’t buy puts, which of course did great as markets crashed following the invasion.
Yesterday the S&P 500 shot up 9% on the news that most of Trump’s new tariffs were paused. I thought this reaction was excessive given that the tariffs weren’t canceled, merely paused 90 days. Note that an exact date is being offered- July 8th! I sold some stocks last night and put in orders for S&P puts and VIX calls, but the limit options orders didn’t fill today as it seems the market caught up to my take from last night. The S&P is down 4% as I write this. This morning I was was researching which puts to buy, leaning toward SPY or XSP at-the-money puts for July 19 (first options date available after the 90-day tariff delay expires), then markets opened and their prices jumped 20+% in seconds as I watched. They are up over 50% now.
It is possible that the administration will fully clarify their stance on tariffs one way or another before July 8th, or even that Congress takes back their tariff power before then and makes their own deal. But I think it is more likely than not that we get a big announcement from the White House on July 8th about which tariffs will be implemented. In which case July 8th will be another wild market day.
This may already be priced in, but so far this April the situation has been changing so rapidly and touching so many parts of the markets and the real economy that even some of the most efficient markets (like US stock and bond markets) seem to be struggling to process what is happening. My ill-timed post from November praising the S&P has some lines that hold up well:
I’m now back up to 90% belief in efficient markets, at least for stocks.
This efficiency seems to change a lot over time. Probably fewer than 10% of US stocks have obvious mis-pricings right now; really none stand out as super mispriced to a casual observer like me. Instead, it seems like every 10 years or so a broad swathe of the market is driven crazy by a bubble or a crash, and you get lots of mispricing- like tech in 2000, forced/panic selling at the bottom in 2009, or meme stocks in 2021. The rest of the time, the stock market is quite efficient. So, in typical times, just be boring and buy and hold a broad index fund.
Ever since April 2nd, we have not been in typical times. At some point they will return and most people are probably best served by just holding through this (selling at the bottom and never getting back in is a big failure mode in investing). But for now the tariffs still have me thinking about buying VIX calls and stock puts (especially when policy changes loom on certain dates like July 8th), and on the bigger question of finding the sort of investments that did well in the 1970’s, another decade of stagflation that was kicked off by a President who broke America’s commitment to an international monetary system that he thought no longer served us.
Myron Scholes was on top of the world in 1997, having won the Nobel Prize in economics that year for his work in financial economics, work that he had applied in the real world in a wildly successful hedge fund, Long Term Capital Management. But just one year later, LTCM was saved from collapse only by a last-minute bailout that wiped out his equity (along with that of the other partners of the fund) and cast doubt on the value of his academic work.
Roger Lowenstein told the story of LTCM in his 2001 book “When Genius Failed“. I finally got around to reading this classic of the genre this year, and I’d say it is still well worth picking up. The story is well-told, and the lessons are timeless-
Beware hubris
Beware leverage
Bigger positions are harder to get out of (especially once everyone knows you are in trouble)
In a crisis, all correlations go to one
Past results don’t necessarily predict future performance
Sometimes things happen that are very different from anything that happened in your backtest window.
The book came out in 2001 but it presages the 07 financial crisis well- not about mortgage derivatives specifically, but the dangers of derivatives, leverage, using derivatives to avoid regulations restricting leverage, and over-relying on mathematical models of risk based on past behavior. If Fed had let LTCM fail, could we have avoided the next crisis? Perhaps so, as their counterparties (most major Wall Street banks) who got burned would have been more careful about the leverage and derivatives used by themselves and their counterparties, and regulators may have taken stronger stances on the same issues.
Perhaps some more recent well-contained blowups foreshadow the next big crisis in the same way, like FTX or SVB?
Tariffs are going up to levels last seen in the 1930 Smoot-Hawley tariffs that helped kick off the Great Depression:
Tariffs are taxes- roughly, a national sales tax with an exemption for domestically-produced goods and services. I think the words make a difference here- “raising tariffs on countries who we run a trade deficit with” just sounds abstruse to most people, while “raising taxes on goods bought from firms in net-seller countries” sounds negative, but they are the same thing.
Of course, in this case the plan is to raise taxes to at least 10% on goods from all other countries even if they aren’t net-sellers, and raise taxes up to 49% on those that are. This is not a negotiating tactic. We know this from the math- the new tax formula uses net imports from a country rather than a country’s tariff rates, so a country could cut their tariffs on US goods to zero today and it wouldn’t necessarily reduce our “reciprocal” tariffs at all; at best it would reduce them to 10%. We also know it isn’t about negotiating because the administration says it isn’t. Their goal, obviously, is to reduce trade, not to free it.
They say they are doing this to bring manufacturing back to America and to promote national defense. But American manufacturers don’t seem happy. Even before the latest huge tax increase, trade war was their biggest concern:
The National Association of Manufacturers Q1 2025 Manufacturers’ Outlook Survey reveals growing concerns over trade uncertainties and increased raw material costs. Trade uncertainties surged to the top of manufacturers’ challenges, cited by 76.2% of respondents, jumping 20 percentage points from Q4 2024 and 40 percentage points from Q3 of last year.
The National Association of Manufacturers responded to the latest tax increase with a negative statement; so even the one major group that might have benefitted from tariffs is unhappy. Foreign producers and US consumers will of course be very unhappy. I think Trump is making a huge political blunder alongside the economic one- he got elected largely because Biden allowed inflation to get noticeably high, but now Trump is about to do the same thing.
I also see this as a huge national security blunder. For tariffs on China, I at least see their argument- we should take an economic hit today in order to become less reliant on our peer-competitor and potential adversary. But the tariffs on allies make no sense- they are hitting the very countries that are most valuable as economic and/or military partners in a conflict with China, like Canada, Mexico, Japan, South Korea, Vietnam, India, and Taiwan (!!!). One of our biggest advantages vs. China has been that we have many allies and they have few, and we appear to be throwing away this advantage for nothing.
What can you or I do about this? Stock up on durable goods before the price increases hit. Picking investment winners is always hard, but things this makes me consider are gold, stocks in foreign countries that trade little with the US, and companies whose stocks took a big hit today despite not actually being importers. Finally, we can try nudging Congress to do something. The Constitution gives the power to levy taxes to the legislative branch, but in the 20th century they voted to delegate some of this power to the executive. Any time they want, Congress could repeal these tariffs and take back the power to set rates. I have some hope they actually will- just yesterday the Senate voted to repeal some tariffs on Canada, and more votes are planned. The alternative is to risk a recession and a wipeout in the midterms:
The US Department of Health and Human Services has announced it is cutting 10,000 of its 82,000 jobs and restructuring:
As part of the restructuring, the department’s 10 regional offices will be cut to five and its 28 divisions consolidated into 15, including a new Administration for a Healthy America, or AHA, which will combine offices that address addiction, toxic substances and occupational safety into one central office.
AHA will include the Office of the Assistant Secretary for Health, the Health Resources and Services Administration, the Substance Abuse and Mental Health Services Administration, Agency for Toxic Substances and Disease Registry, and the National Institute for Occupational Safety and Health.
These divisions do many different jobs, but as usual what stands out to me is their data- both because it is what I have found directly useful in the past, and because it is what I still have some control over now. Writing your Representatives or writing an op-ed has a minuscule chance of changing Federal policy, but if you download data, you definitely have that data.
What worries me here is that some of the agencies being consolidated might discontinue some of their data products going forward, or even pull some of what they have already created offline. I don’t think this is farfetched given what has happened so far, and given that even in good times these agencies pull down data they painstakingly prepared. For instance, HRSA only publicly posts the State- and County-level Area Health Resources File back to 2019, even though they have annual data going back to 2001.
Probably all 13 of the reorganizing divisions have data worth looking into, and given the staff cuts, even data products in the other divisions could be at risk. But my plan is to focus on the two reorganizing divisions whose data I have previously found useful- HRSA and SAMHSA. HRSA has a nice data download page with 16 different datasets, including the Area Health Resources File, which offers detailed information on the health care workers and facilities in each US county. SAMHSA offers the National Substance Use and Mental Health Services Survey, the Treatment Episode Data Set, and the National Survey of Drug Use and Health. I have previously cleaned and archived the state-level version of the NSDUH, but not the individual-level version that is for now still available from SAMHSA.
All of these datasets are easy to download now, and some will probably become very hard to access later, so now is a good time to take a few minutes and save whatever you think you might need.
Certificate of Need laws require many types of health care providers to obtain the permission of a state board before they are allowed to open or expand in many US states. But there is a lot of variation from state to state in which types of providers are covered by these laws. I put together this map to show the 15 states that require new home health care agencies to obtain a Certificate of Need:
CON states see reduced competition, which tends to be bad news for patients and new entrants, but good for existing providers and the private equity firms consideringbuying them.
But some CON states like Rhode Island have proposed reforms that would exempt home health agencies from the CON process, putting them in line with the majority of states that put new entrants on an even footing with incumbent providers.
I never thought of myself as a businessman- until 2015 when the IRS told me I was, and that I therefore needed to pay them more money to cover the self-employment tax. Naturally I was confused and angry about this at first, but in the long run it turns out they were doing me a favor.
If you make a tiny amount of 1099-MISC or 1099-NEC income on occasion, the IRS is probably* fine with characterizing this as ordinary income from a hobby. But if you earn 1099 income at all regularly, they will likely want to characterize you as a business, and want you to pay a self-employment tax similar to the payroll tax that W2 employees pay (though it will look higher to you, since you will pay both the employee and employer halves of the tax). If you make an intermediate amount of 1099 income, you might have the choice of whether to call this hobby income or business income; I had thought it would be better to avoid the complications and extra taxes of being a business, but it turns out that being a business unlocks new opportunities for deductions than can far outweigh the self-employment tax.
For example, a home office, business-related travel expenses, and advertising expenses can be deductible. For a writer, this could cover conferences, website expenses, computers, and much more. It also means you can start a SEP IRA– in addition to a personal IRA if you like. This alone could allow you to deduct thousands of dollars in income per year (technically up to $69k if you make at least $276,000 per year in business income, though if you make that much, you’re the one who should be giving me advice). The SEP IRA has the advantage over a personal IRA of a much higher income limit and, potentially a higher contribution limit, though again the beauty is that you don’t have to choose- you can just do both.
While this post is mainly about business, I also think regular IRAs might still be underrated. I didn’t start one until 2022, but I should have done it much earlier. First I thought I was too poor (low income, then higher income but with student loans to pay off first), then I thought I was too rich (above the income limits). It turns out though that you can still start a personal IRA even when you are above the income limits- it just means you only get one tax benefit instead of two, but that one tax benefit is still pretty good.
Every IRA has the benefit of investments growing tax-free; if you meet the income limits then IRAs get the additional benefit of avoiding income taxes either when you put the money in (for traditional) or when you pull it out (for Roth). But even if you “only” get the benefit of tax free growth, that can still be a huge monetary benefit depending on your investment strategy. It is also a big time benefit- every taxable brokerage account means at least one** extra tax form to deal with every year, while an IRA account avoids this.
Another great benefit to IRAs (SEP or regular) is that you can still start one now and make contributions for the 2024 tax year. I was just doing my taxes and kicking myself for not doing some things differently back in 2024 when it would have helped; but IRAs are like a form of time travel where you can still go back and fix things, at least until April 15th.
*Disclaimer- Not official tax advice, I’m not an accountant, I’m just a 37 year old guy with lifetime 1-1-1 record against the IRS. Three times they have told me I owed them more than I paid on a tax return. Once I won (I told them I owed nothing and explained why, and they agreed). Once I lost (I told them oh shit, you’re right and paid them). For the story I started this post with, I call it a draw (they told me I owed them X, I told them I owed nothing and explained why, then they told me I actually owed them 1/3X and I just paid it).
**More than one if like me you accidentally invest in a partnership and as a resultget a K-1 on top of the usual 1099-DIV for that overall brokerage account
I’ve now posted individual-level responses to the 1978-2025 Michigan Consumer Surveys to Kaggle in CSV and Stata formats. The University of Michigan’s Consumer Surveys are a widely followed source for data on consumer confidence and inflation expectations:
Their official site is good if you just want summary tables or charts like this:
But what if you want detailed crosstabs to see how sentiment differs for different groups, or microdata so that you can run regressions? With enough clicks you can get this from what UMich calls their “cross-section archive“. But it is pretty hidden, my student looking into this thought they just didn’t offer individual-level data; and even once you get their data, it is in an unlabelled CSV file with hard-to-understand variable names and codes. So I wanted to make it clear that the full data with all responses for all years is available, and if you use my Stata version it is even reasonably easy to understand (the code I adapted for labelling it is on OSF). Then you can run your regressions, or make charts like this:
The College-Only Covid Recovery
If you’re new here, a reminder that you can find other cleaned-up versions of popular datasets on my data page.
Economist Writing Every Day 