How ChatGPT works from geography and Stephen Wolfram

Joy Buchanan1 Comment

By now, everyone should consider using ChatGPT and be familiar with how it works. I’m going to highlight resources for that.

My paper about how ChatGPT generates academic citations should be useful to academics as a way to quickly grasp the strengths and weakness of ChatGPT. ChatGPT often works well, but sometimes fails. It’s important to anticipate how it fails. Our paper is so short and simple that your undergraduates could read it before using ChatGPT for their writing assignments.

A paper that does this in a different domain is “GPT4GEO: How a Language Model Sees the World’s Geography” (Again, consider showing it to your undergrads because of the neat pictures, but probably walk through it together in class instead of assigning it as reading.) They describe their project: “To characterise what GPT-4 knows about the world, we devise a set of progressively more challenging experiments… “

For example, they asked ChatGPT about the populations of countries and found that: “For populations, GPT-4 performs relatively well with a mean relative error (MRE) of 3.61%. However, significantly higher errors [occur] … for less populated countries.”

ChatGPT will often say SOMETHING, if prompted correctly. It is often, at least slightly, wrong. This graph shows that most estimates of national populations were not correct and the performance was worse on countries that are less well-known. That’s exactly what we found in our paper on citations. We found that very famous books are often cited correctly, because ChatGPT is mimicking other documents that correctly cite those books. However, if there are not many documents to train on, then ChatGPT will make things up.

I love this figure from the geography paper showing how ChatGPT estimates the elevations of mountains. This visual should be all over Twitter.

There are 3 lines because they did the prompt three times. ChatGPT threw out three different wrong mountains. Is that kind of work good enough for your tasks? Often it is. The shaded area in the graph is the actual topography of the earth in those places. ChatGPT “knows” that this area of the world is a mountain. But it will just put out incorrect estimates of the exact elevation, instead of stating that it does not know the exact elevation of those areas of the world.

Another free (long, advanced) resource with great pictures is Stephen Wolfram’s 2023 blog article “What Is ChatGPT Doing … and Why Does It Work?” (YouTube version)

The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.

If you feel like you already are proficient with using ChatGPT, then I would recommend Wolfram’s blog because you will learn a lot about math and computers.

Scott wrote “Generative AI Nano-Tutorial” here, which has the advantage of being much shorter than Wolfram’s blog.

EDIT: New 2023 overview paper (link from Lenny): “A Survey of Large Language Models“

Government Purchases and How Markets Avoid Messes

Zachary BartschLeave a comment

The government is unique among economic institutions insofar as it can use coercion legally. But not all activities are coercive. Clearly, taxation is overwhelmingly coercive. Some people say that they are happy to pay taxes, but the voluntary gifts to the US Treasury are itsy-bitsy (just over $1m for FY 2023). Most regulations also include the threat of fines or jail time for non-compliance.

But once the government has the money in their coffers, there is plenty that they can do consensually. Once they have the resources, they are often just another potential transactor in the markets for goods and services. While the government can transact as well as anyone else, there is a fundamental theoretical difference for how we should interpret those transactions. Specifically, there is a principal-agent problem such that we can’t quite identify the welfare that is enjoyed by consumers when the government makes purchases. We really have very little idea.

Garett Jones uses the analogy of the government confiscating potatoes. The worst use would be for the government to throw the valuable resources into the river. Those resources help no one. Improved welfare would be yielded if the government just transferred those potatoes back to people. Sure, there’s the transaction cost of administration, but people get their potatoes back. Finally, the great hope is that the government takes the potatoes and makes tasty potato fritas such that they return to the public something more valuable than they took. These might be things that fall into the public goods category or solving collective action problems generally.

The above examples illustrates that how the government spends matters a lot for the welfare implications of the newly purchased government resources. But, we need to recall that there is an entire private segment of the market that is affected by the government transactions.

Short-Run Analysis

In a competitive market, firms face increasing marginal costs and make decisions about their levels of output. When the government makes purchases, it’s simply acting as another demander. How does the entry of a larger demander affect everyone else in the market? See the below GIF.

Continue reading →

Does More Health Spending Buy Better Outcomes for States?

James Bailey3 Comments

When you look across countries, it appears that the first $1000 per person per year spent on health buys a lot; spending beyond that buys a little, and eventually nothing. The US spends the most in the world on health care, but doesn’t appear to get much for it. A classic story of diminishing returns:

Source: https://twitter.com/MaxCRoser/status/810077744075866112/photo/1

This might tempt you to go full Robin Hanson and say the US should spend dramatically less on health care. But when you look at the same measures across US states, it seems like health care spending helps after all:

Source: My calculations from 2019 IHME Life Expectancy and 2019 KFF Health Spending Per Capita

Last week though, I showed how health spending across states looks a lot different if we measure it as a share of GDP instead of in dollars per capita. When measured this way, the correlation of health spending and life expectancy turns sharply negative:

Source: My calculations from 2019 IHME life expectancy, Gross State Product, and NHEA provider spending

Does this mean states should be drastically cutting health care spending? Not necessarily; as we saw before, states spending more dollars per person on health is associated with longer lives. States having a high share of health spending does seem to be bad, but this is more because it means the rest of their economy is too small, rather than health care being too big. Having a larger GDP per capita doesn’t just mean people are materially better off, it also predicts longer life expectancy:

Source: My calculations from 2019 IHME life expectancy and 2019 Gross State Product

As you can see, higher GDP per capita predicts longer lives even more strongly than higher health spending per capita. Here’s what happens when we put them into a horse race in the same regression:

The effect of health spending goes negative and insignificant, while GDP per capita remains positive and strongly significant. The coefficient looks small because it is measured in dollars, but what it means is that a $10,000 increase in GDP per capita in a state is associated with 1.13 years more life expectancy.

My guess is that the correlation of GDP and life expectancy across states is real but mostly not caused by GDP itself; rather, various 3rd factors cause both. I think the lack of effect of health spending across states is real, between diminishing returns to spending and the fact that health is mostly not about health care. Perhaps Robin Hanson is right after all to suggest cutting medicine in half.

Young People Have a Lot More Wealth Than We Thought

Jeremy Horpedahl32 Comments

I’ve written numerous times about generational wealth on this blog. My biggest post was one comparing different generations using the Fed’s Distributional Financial Accounts back in September 2021. I’ve posted several updates to that post as new the quarterly data was released, but this post contains a major update. I’ll explain in great detail below about the updates, but first let me present the latest version of the chart (through 2023q3):

Regular readers will notice a few differences compared with past charts. The big one is that young people have a lot more wealth than it appeared in past versions of this chart! You’ll also notice that I have relabeled this line “Millennials & Gen Z (18+)” and shifted that line over to the left a few years to account for the fact that this isn’t just the wealth of Millennials, and therefore the median age of this group is lower than in my past charts. The two dollar figures I highlighted are at the median age of 30 for these age cohorts (unfortunately we don’t have data for Boomers at that age).

Continue reading →

Hazards of the Internet of Things 2. Big Brother Is Watching Your Every Breath

Scott Buchanan1 Comment

There seems to be something of a generational divide as to how important is your personal privacy. Folks under, say, age 40, have lived such a large fraction of their lives with Facebook and Amazon and Google and Twitter logging and analyzing and reselling information on what they view and listen to and say and buy, that they seem rather numb to the issue of internet privacy. Install an Alexa that ships out every sound in your home and a smart doorbell that transmits every coming and going to some corporate server, fine, what could possibly be the objection?  So what if your automobile, in addition to tracking and reporting your location, feeds all your  personal phone text messages to the vehicle manufacturer?

For us older folks whose brain pathways were largely shaped in a time when communication meant talking in person or on a (presumably untapped) phone, this seems just creepy. Polls show that a majority of Americans are uneasy about the amount of data on them being collected, but “do not think it is possible to go about daily life without corporate and government entities collecting data about them.”

There are substantive concerns that can be raised about the uses to which all this information may be put, and about its security. Per VPNOverview:

Over 1,800 data leaks took place last year in the US alone, according to Statista. These breaches compromised the records of over 420 million people.” . With smartwatches having access to so much sensitive information, here’s what kind of data can fall into the wrong hands in case of a data leak:

  • Your personal information, including name, address, and sometimes even Social Security Number
  • Sensitive health information collected by the smartwatch
  • Login credentials to all the online platforms connected to your smartwatch
  • Credit card and other payment information
  • Digital identifiers like your IP address, device ID, or browser fingerprint
  • Remote access information to smart home devices

Several times a year now, I get notices from a doctor’s office or finance company or on-line business noting blandly that their computer systems have been hacked and bad guys now have my name, address, birthdate, social security number, medical records, etc., etc. (They generously offer me a year of free ID fraud monitoring. )

The Internet of Things (IoT) promises to ramp up the snooping to a whole new level. I took note four years ago when Google acquired Fitbit. At one gulp, the internet giant gained access to a whole world of activity and health data on, well, you. The use of medical and other sensors, routed through the internet, keeps growing. One family member uses a CPAP machine for breathing (avoid sleep apnea) at night; the company wanted the machine to be connected on the internet for them to monitor and presumably profit from tracking your sleep habits and your very breath. And of course when you don a smart watch, your every movement, as well as your heartbeat, are being sent off into the ether. (I wonder if the next sensor to be put into a smart watch will be galvanic skin response, so Big Tech can log when you are lying).

According to a senior systems architect: “The IoT is inevitable, like getting to the Pacific Ocean was inevitable. It’s manifest destiny. Ninety eight percent of the things in the world are not connected. So we’re gonna connect them. It could be a moisture sensor that sits in the ground. It could be your liver. That’s your IoT. The next step is what we do with the data. We’ll visualize it, make sense of it, and monetize it. That’s our IoT.”

When my kids were little, we let them use cassette tape players to play Winnie the Pooh stories. With my grandkids, the comparable device is a Yoto player. This also plays stories (which is good, better than screens), but it only operates in connection with the internet. The default is that the Yoto makers collect and sell personal information on usage by you and your child (which would include time of day as well as choice of stories). You can opt out, if you are willing to take the trouble to write to their legal team (thanks, guys).

There are cities in the world, in China but also some European cities, where there are monitoring cameras (IoT) everywhere. Individuals can be recognized by facial features and even by the way they walk; governmental authorities compile and track this information. These surveillance systems are being sold to the public with the promise of increased “security.” Whether it really makes we the people more secure is heavily dependent on the benevolence and impartiality of the state powers. Supposing a department of the federal government with access to surveillance data became politicized and then harassed members of the opposing party?

I’ll conclude with several slides from  Timothy Wallace’s 2023 presentation on the Internet of things:

The dystopian  novel 1984 by George Orwell was published in 1949.  It describes a repressive totalitarian state, headed by Big Brother, which was characterized by pervasive surveillance. Ubiquitous posters reminded citizens, “Big Brother is watching you.” Presumably the various cameras and microphones used in the mass surveillance there were paid for and installed by the eavesdropping authorities. It is perhaps ironic that so many Americans now purchase and install devices that allow some corporate or governmental entity to snoop them more intimately than Orwell could have imagined.

Avoid subfield tunnel vision

mdmakowskyLeave a comment

Folks are dunking on a tweet and, indirectly, the underlying research connecting mosquito nets to the degredation of seagrass meadows.

I’ll be honest, the implication that free mosquito nets are net negative for poverty and health sent me into that special kind of rage that can only be fomented by someone on the internet being both condescending and egregiously wrong at the same time. Do I even need to go over why this is bad? Why malaria prevention at a continental level outweighs hypothesized marginal seagress loss? I didn’t think so.

What I want to talk about is is subfield tunnel vision. A common piece of advice passed on to each generation of PhD students is to become a genuine expert in something. If you write a dissertation on the effect of malaria nets on elementary school attenance in Uganda, then you should become an expert, on the bleeding edge of all related-research, on mosquitos, nets, and primary education in Africa. As you career takes shape, the both the questions that strike you as important and the opportunities presented to you by institutions and administrators will shape your research. Bit my bit you will be shaped (and occasionally sanded down) into an ever-narrower expert. And that’s fine, that’s the story of incentives to specialize that comes for us all (NB: if your mind went to one of the public intellectual generalists you admire, do note that being a generalist in the modern world is very much its own niche specialty).

Specialization is good, but do take care that while your expertise becomes narrower that your view of world remains wide. We all know the relevant cliche about all the world becoming a nail whilst holding a hammer, but this is about about the rationalizing of tools and techniques. This is about how your specialization fits within the world and, more specifically, how the consquences of choices you might advise stand in the grand utilitarian calculus.

The authors of the paper in question are the Chief Scientific Officer and Chief Conservation Officer of Project Seagrass. These are people who have dedicated their lives to the preservation of seagrass meadows and, in turn, ocean health and the global stock of fish. Should we be surprised that their paper’s abstract closes with “We conclude that the use of mosquito nets for fishing may contribute to food insecurity, greater poverty and the loss of ecosystem functioning”? No, we should not. First, you could argue that they are just putting, in words, the implied signs of their analysis, and not the relative magnitudes. Maybe they aren’t implying mosquito nets are a net negative. You could be generous and argue that all they are trying to say is “Mosquito nets are great, but as soon as the malaria vaccine is universally distributed we should ditch all these nets because the costs will outweigh the benefits.

I don’t think they are, though. They lean heavily on Short et al (2018) and their claim that fishing nets are the primary use of freely provided mosquito netting. That Short et a result appears to be based almost entirely on an online survey with 113 respondents. The implication is that the massive reduction in malaria specifically attributed to the distribution of free mosquito nets is in fact a mirage, that these nets are instead finding their way into the ocean as improvised capital for small scale fishing operations (“artisanal fishing”, in the parlance of the paper), with the resulting consequence of catching additional juvenile fish at the margin, harming the future stock of fish.

The second part of that equation seems entirely feasible! Unintended costs happen. What I want to emphasize is that the authors are narrowly focused on establishing the cost of future fish in seagrass meadows while also being overly credulous of what is, I’m sorry, a ridiculously crappy survey that dismisses the enormous benefits of those nets to save human lives, especially children under the age of 5.

I don’t think the authors are being selfish, have ill-motivations, or have been bought off by a global conspiracy of wealthy fishery magnates. I just think they have succumbed to a bias that afflicts every scholar at one time or another, myself included. Gatekeepers won’t publish your paper in top journals, fund your research with needed grants, or invite you to prestigious conferences unless you hype your work to the absolute maximum of feasible importance. Spend a couple years as the conductor on your subfield’s hypetrain and maybe you start to believe it just a wee bit too much. Your subject of concern remains concrete, the questions imperative, while everything else increasingly fades into the realm of the abstract, the consequences negotiable.

As for the twitter commenter being dunked on, I just think it’s classic overeagerness to denigrate everything touched by someone you find odious. SBF is a bad person, did bad things, has bad hair. Sure, but maybe don’t? Maybe leave the most successful malaria prevention endeavor in the history of the world out of your public disgust for a <checks notes> young cryptocurrency embezzler? Don’t let your deserved anger for a genuinely bad person make you dumber at the margin. Bad people already impose costs on us all. Letting them skew your view of everything they touch just makes their societal footprint bigger.

Intelligence for School Closing

Joy Buchanan1 Comment

I don’t have much time to write this week because I lost so many work hours to schools closing for “weather.”

Tyler has been saying that we should welcome more intelligence (in the form of LLMs – I’m not getting any smarter). What would we want intelligence for? How about reducing the error rate on school closing?

First, I will recognize that things are already getting better due to computers. The internet and texting and radar help. Compared to when I was a child in New Jersey, it’s more efficient to text all the parents the night before, as opposed to having people get up at 6am to scan the radio for news. Weather forecasting has presumably gotten better.

Now my rant: Right around what was already a three-day official weekend, school was closed three times. Even my kids were irate when that last day was announced. In my opinion, only one of those closures was justified for extreme weather.

There is a lot of dumb in a city. People complain about routine processes being suboptimal. It would be great if we humans could figure out ways to apply more intelligence to these local problems and make less mistakes.

This is a joke for any readers in cold climates. My Alabama kids thought it was fun to collect icicles because they have almost never seen them before.

Teaching Taxes w/GIFs

Zachary Bartsch2 Comments

Last time the gifs were simply about price & quantity and welfare. I’m sharing some more GIFs, this time in regard to welfare and taxes.

First, see the below gif. It shows us that both consumer surplus (blue area) and producer surplus (red area) always rise if there is a demand increase (assuming the law of supply and law of demand).

Next, let’s consider a basic tax. We can represent it as the difference between what the demander pays and what the supplier receives. The bigger the tax, the bigger the difference between the two.

Now let’s combine the tow ideas: If taxes rise, then the quantity transacted falls, price paid rises, price received falls, and both consumer and producer surplus fall. Not only that, since there is an inverse relationship between the tax rate and the quantity transacted, it may be that increasing the tax rate more *reduces* revenue. The idea that there is a tax revenue maximizing tax rate is illustrated below right and is known as the Laffer curve.

Continue reading →

Where is Health Care The Biggest Part of the Economy?

James Bailey1 Comment

State health care spending usually gets reported in terms of dollars per capita, leading to maps like this that show Alaska as the highest-spending state and Utah as the lowest:

Source: https://www.kff.org/other/state-indicator/health-spending-per-capita/

But states differ greatly in how rich they are and how much they have to spend. I wanted to know the states where health care takes up the largest and smallest share of the economy, so I got the data:

Health Care Spending as Share of State Gross Domestic Product in 2019:

Source: I divided 2019 National Health Expenditure Provider data on total health spending by 2019 Gross State Product data.

You can see that health spending as a share of GDP looks pretty different from health spending in raw dollars. We’ve gone from a high-spending North and low-spending South to more of a mix. Health spending is now highest in West Virginia, where it makes up more than a fourth of the economy; and lowest in Washington State and Washington D.C., where it makes up less than one ninth of the economy.

The biggest change when considering things this way is in Washington D.C., which has the highest spending in $ terms but the lowest as a share of GDP because it has an enormous GDP per capita. Many other states that spend a lot in $ also fall a lot in the rankings due to high GDP per capita, including Alaska, New York, and Massachusetts. The states that rise the most in this ranking are poor states like Arkansas, Alabama, and Mississippi. Mississippi rises the most, gaining 37 spots in the rankings of highest-spending states when we go from $ per capita to share of GDP.

I share the data here so you can do your own comparisons:

Continue reading →

Follow the Money in Politics

Jeremy Horpedahl5 Comments

As we enter election season, I can sympathize with those that want to ignore it as much as possible. But if you do want to follow it closely, here is my advice: talk is cheap, so follow the money.

And by money, I am not referring to campaign contributions. I mean prediction markets, where people are putting their money where their mouth is, rather than just making predictions based on their own intuition (or their own “model,” which is just a fancy intuition).

There are a number of betting markets online today, but a good aggregator of them is Election Betting Odds.

For example, here is their current prediction for which party will win the Presidency:

Continue reading →