As I was reading through What is Real?, it occurred to me that I’d like a review on an issue. I thought, “Experimental physics is like experimental economics. You can sometimes predict what groups or “markets” will do. However, it’s hard to predict exactly what an individual human will do.” I would like to know who has written a little article on this topic.
I decided to feed the following prompt into several LLMs: “What economist has written about the following issue: Economics is like physics in the sense that predictions about large groups are easier to make than predictions about the smallest, atomic if you will, components of the whole.”
First, ChatGPT (free version) (I think I’m at “GPT-4o mini (July 18, 2024)”):
I get the sense from my experience that ChatGPT often references Keynes. Based on my research, I think that’s because there are a lot of mentions of Keynes books in the model training data. (See “”ChatGPT Hallucinates Nonexistent Citations: Evidence from Economics“)
Next, I asked ChatGPT, “What is the best article for me to read to learn more?” It gave me 5 items. Item 2 was “Foundations of Economic Analysis” by Paul Samuelson, which likely would be helpful but it’s from 1947. I’d like something more recent to address the rise of empirical and experimental economics.
Item 5 was: “”Physics Envy in Economics” (various authors): You can search for articles or papers on this topic, which often discuss the parallels between economic modeling and physics.” Interestingly, ChatGPT is telling me to Google my question. That’s not bad advice, but I find it funny given the new competition between LLMs and “classic” search engines.
When I pressed it further for a current article, ChatGPT gave me a link to an NBER paper that was not very relevant. I could have tried harder to refine my prompts, but I was not immediately impressed. It seems like ChatGPT had a heavy bias toward starting with famous books and papers as opposed to finding something for me to read that would answer my specific question.
I gave Claude (paid) a try. Claude recommended, “If you’re interested in exploring this idea further, you might want to look into Hayek’s works, particularly “The Use of Knowledge in Society” (1945) and “The Pretense of Knowledge” (1974), his Nobel Prize lecture.” Again, I might have been able to get a better response if I kept refining my prompt, but Claude also seemed to initially respond by tossing out famous old books.
I Googled the query “Physics Envy in Economics” . The top result was a Wikipedia article called “Physics Envy”. So, ChatGPT had dredged up an oft-repeated phrase, but hadn’t really answered my question. The second link was: Overheard at MIT: Why Economics Isn’t Like Physics Gated, so I don’t know how good it is. Perhaps closer to what I wanted than “The General Theory of Employment, Interest, and Money” by John Maynard Keynes, which was the first reading suggestion by ChatGPT.
I tried my own prompt for Google.
I was less impressed by this AI result. If you’d like to enjoy the nostalgia of old-fashioned websites, click the Koopmans link. Koopman has done work in both physics and econ, but I think Elicit ultimately got me to a better source (indirectly).
Elicit is supposed to be good at providing academic references. Elicit wrongly presented a book review as an article. Through a bit of clicking, I was able to discover the book called More Heat than Light. That book would be longer than what I wanted but would contain lots of helpful details and references. Elicit was even onto something in terms of pointing me to open-access book reviews, since I don’t want to buy the book right now. Elicit was better than the LLMs, even though there was a mistake in the way it initially presented the top results.
I was hoping for a short post or article. I didn’t end up finding what I wanted. I wanted something that would teach me a bit more without forcing me to read widely or search for hours. However, I do have a pointer for a book that is probably on topic.
With a lot of work, I could write a “literature review” for this topic, given that some gaps still exist, and the strands are many. It still seems like it’s hard to get the machines to write one for you. Overall, I think Google or Elicit are better tools than the LLMs, currently, for the purpose of getting sources for a literature review. I tried the following prompt with ChatGPT: “Could you write me a literature review on the topic of how economics models are like predictive physics models?” Again, I felt like the result showed a bias toward older books (and also some of the references were hallucinated in terms of publication date or outlet). It could be a starting point, but I would have like some actual empirical results included, especially from controlled experiments.
When I was a research assistant during my first year of graduate school, a faculty member wanted me to help her with a literature review for her paper. It was boring difficult work, and I’m not sure if I helped her very much. At the end of the day, she still had to write it. I put the possibly useful information (e.g. the abstract of relevant papers) in an Excel sheet for her. You could say I was supposed to play the role of the LLM-assistant. I could write and reason. I had access to the literature. So, could I shield her from having to read the papers?** From what I remember, the service I provided was hardly better than what she could get from Google. Doing a good up-to-date literature review for a truly novel paper is difficult.
If you were ever wondering how economists think about physics, the results from https://marginalrevolution.com/?s=physics are not a bad place to start! Since they’ve been at it for 20 years at MR, the results are not-too-old but also comprehensive. The top result, Applying physics to gdp forecasting, was probably the best, accessible resource out of everything I’d seen while dabbling with the generic tech products.
** I do not dislike reading papers. What makes writing a lit review difficult is that you have limited time and the universe of papers is vast.
Economist Writing Every Day 
Perplexity Pro says there is no relevant lit:
“No economist has specifically written about economics being like physics in the sense that predictions about large groups are easier than predictions about individual components. However, this concept does relate to some important ideas in economics:
Macroeconomics vs Microeconomics
Economists generally divide their field into macroeconomics (studying large-scale phenomena) and microeconomics (studying individual decision-making and markets). While not exactly the same as the physics analogy, this division recognizes that different approaches are needed at different scales.
Aggregation and Emergent Behavior
Some economists have explored how individual behaviors can lead to emergent properties at larger scales that are difficult to predict from micro-level analysis alone. This relates to the idea that macro predictions may be easier in some cases.
Statistical Approaches
Many economic models rely on statistical methods applied to large datasets, which can yield insights about aggregate behavior even if individual actions are unpredictable. This is somewhat analogous to statistical mechanics in physics.
Critique of “Physics Envy”
While not directly addressing the prediction issue, some economists have critiqued the field’s attempts to emulate physics too closely. For example:
These critiques suggest that while there may be some parallels between economics and physics, economists generally do not embrace direct analogies between the fields when it comes to predictive power at different scales.In summary, while the specific comparison to physics predictions has not been prominently made by economists, related concepts about scale, aggregation, and methodology are important in economic thought and debate.”
LikeLike
Interesting. Thanks! Maybe it was an unfair test of ChatGPT – to ask it for something that doesn’t exist.
LikeLike