You can find my paper with Will Hickman “Do people trust humans more than ChatGPT?” at the Journal of Behavioral and Experimental Economics (JBEE) online, and you can download it free before July 30, 2024 (temporarily ungated*).
Did we find that people trust humans more than the bots? It’s complicated. Or, as we say in the paper, it’s context-dependent.
When participants saw labels informing them (e.g. “The following paragraph was written by a human.”) about authorship, readers were more likely to purchase a fact-check (the orange bar).
Informed subjects were not more trusting of human authors versus ChatGPT (so we couldn’t reject the null hypothesis about trusting humans, in that sense). However, Informed subjects were significantly less likely to trust their own judgement of the factual accuracy of the paragraph in the experiment, relative to readers who saw no authorship labels.
Some regulations would make the internet more like our Informed treatment. The EU may mandate that ChatGPT comply with the obligation of: “Disclosing that the content was generated by AI.” Our results indicate that this policy would affect behavior because people read differently when they are forced to think up front about how the text was generated.
Inspiration for this article on trust began with observing the serious errors that can be produced by LLMS (e.g. make up fake citations). Our hypothesis was that readers are more trusting of human authors, because of these known mistakes by ChatGPT. This graph shows that participants trust (left blue bar = “High Trust”) statements *believed* to have been written by a human (so, in that sense, our main hypothesis has some confirmation).
Conversely, in the Informed treatment, readers are equally uncertain about text written either by humans or bots. Informed readers are suspicious, so they buy a fact-check. “High Trust” (the blue bar) is the option that maximizes expected value if the reader thinks the author has not made factual errors.
So, in conclusion, we find that human readers can be made more suspicious by framing. In this case, we are thinking of being cautious and doing a fact-check as a good thing. The reason is that, increasingly, the new texts of society are being written by LLMs. Evidence of this fact has been presented by Andrew Gray in a 2023 working paper: “ChatGPT “contamination”: estimating the prevalence of LLMs in the scholarly literature” Note that is the scholarly literature, not just the sports blogs or the Harry Potter – Taylor Swift- crossover fanfics.
What about the medical doctors? What is the authority on whether you are getting surgery or not? See: “Delving into PubMed Records: Some Terms in Medical Writing Have Drastically Changed after the Arrival of ChatGPT”
Economics as a discipline really likes to boil things down to their essentials. There are plenty of examples. How many goods can one consume? Just two, bread and not bread. How can you spend your time? You can labor or leisure. How do you spend your money? Consume or save. It’s this last one that I want to emphasize here.
First, all income ultimately ends up being spent on consumption. Saving today is just the decision to consume in the future. And if not by you, then by your heirs. One determinant of inter-temporal consumption decisions is the real rate of return. That is, how many apples can you eat in the future by forgoing an apple eaten today? The bigger that number is, the more attractive the decision to save.
Further, since most saving is not in the form of cash and is instead invested in productive assets, we can also characterize the intertemporal consumption problem as the current budget allocation decision to consume or invest. The more attractive capital becomes, the more one is willing to invest rather than consume. The relative attractiveness between consumption and investment informs the consumption decision.
How attractive is investment? I’ll illustrate in two graphs. First, if the price of investment goods falls relative to consumption goods, then individuals will invest more. The graph below charts the price ratio of investment goods to consumption goods. Relative to consumption, the price of investment has fallen since 1980. Saving for the future has never been cheaper!
Of course, as in a price taker story, I am assuming that individuals don’t affect this price ratio. Truly, prices are endogenous to consumption/investment decisions. For all we know, it may be that the prices of investment goods are falling because demand for investment goods has fallen. But that doesn’t appear to be the case.
I’m back from Manifest, a conference on prediction markets, forecasting, and the future. It was an incredible chance to hear from many of my favorite writers on the internet, along with the CEOs of most major prediction markets; in Steve Hsu’s words, Woodstock for Nerds. Some highlights:
Robin Hanson took over my session on academic research on prediction markets (in a good way; once he was there everyone just wanted to ask him questions). He thinks the biggest current question for the field is to figure out why is the demand for prediction markets so low. What are the different types of demand, and which is most likely to scale? In a different talk, Robin says that we need to either turn the ship of world culture, or get off in lifeboats, before falling fertility in a global monoculture wrecks it.
Play-money prediction markets were surprisingly effective relative to real-money ones in the 2022 midterms. Stephen Grugett, co-founder of Manifold (the play-money prediction market that put on the conference), admitted that success in one election could simply be a coincidence. He himself was surprised by how well they did in the 2022 midterms, and said he lost a bunch of mana on bets assuming that Polymarket was more accurate.
Substack CEO Chris Best: No one wants to pay money for internet writing in the abstract, but everyone wants to pay their favorite writer. For me, that was Scott Alexander. We are trying to copy Twitter a bit. Wants to move into improving scientific publishing. I asked about the prospects of ending the feud with Elon; Best says Substack links aren’t treated much worse than any other links on X anymore.
Razib Khan explained the strings he had to pull for his son to be the first to get a whole genome sequence in utero back in 2014- ask the hospital to do a regular genetic test, ask them for the sample, get a journalist to tweet at them when they say no, get his PI’s lab to run the sample. He thinks crispr companies could be at the nadir of the hype cycle (good time to invest?).
Kalshi cofounder Luana Lopes Lara says they are considering paying interest on long term markets, and offering margin. There is enough money in it now that their top 10 or so traders are full time (earning enough that they don’t need a job). The CFTC has approved everything we send them except for once (elections). We don’t think their current rule banning contest markets will go through, but if it does we would have to take down Oscar and Grammy markets. When we get tired of the CFTC, we joke that we should self certify shallot futures markets (toeing the line of the forbidden onion futures). Planning to expand to Europe via brokerages. Added bounty program to find rules problems. Launching 30-50 markets per week now (seems like a good opportunity, these can’t all be efficient right?).
There was lots else of interest, but to keep things short I’ll just say it was way more fun and informative doing yet another academic conference, where I’ve hit diminishing returns. More highlights from Theo Jaffee here; I also loved economist Scott Sumner’s take on a similar conference at the same venue in Berkeley:
If you spend a fair bit of time surrounded by people in this sector, you begin to think that San Francisco is the only city that matters; everywhere else is just a backwater. There’s a sense that the world we live in today will soon come to an end, replaced by either a better world or human extinction. It’s the Bay Area’s world, we just live in it.
In my Inbox I read the following sentence, summarizing an article on child health in Arkansas: “The latest Annie E. Casey Foundation KIDS COUNT Data Book shows 2022 was the deadliest year on record for child deaths in Arkansas.”
Deadliest on record! That certainly grabbed my attention. I clicked the link and read the article. Indeed, they emphasize three times that 2022 was the “deadliest year” for kids in Arkansas, including with a chart! And the chart does seem to support the claim: in 2022 there were 44 child and teen deaths per 100,000 in Arkansas, higher than any year on the chart.
But wait a minute, this chart only goes back to 2010. Surely the record goes back further than that? Indeed it does. It took me three minutes (yes, I timed myself, and you have to use 4 different databases) to complete the necessary queries from CDC WONDER to extract the data to replicate their 2010-2022 chart, and to extend the data back a lot further: all the way to 1968 (though in 30 seconds I could have extended it back to 1999).
And what do we find in 1968? The death rate for children and teens in Arkansas was twice as high as it was in 2022. Not just a little higher, but double. With some more digging, I might be able to go back further than 1968, but from the easily accessible CDC data, that’s as far back as “the record” goes. Of course, I knew where to look, but I would hope that a group producing a data book on child health also knows where to look. And you don’t need to extend this very far past the arbitrary 2010 cutoff in the article quoted: 2008 and every year before it was more deadly than 2022 for children in Arkansas. Here’s a chart showing the good long-run trend:
Now there is a notable flattening of the long-run trend in the past 15 years or so, and a big reversal since 2019. What could be causing this? The article I read doesn’t get specific, but here’s what they say: “The state data isn’t broken out into cause of death, but firearm-related deaths have become the leading cause of death among U.S. teens in recent years. Deaths from accidents such as car crashes account for most child deaths.”
But using CDC WONDER, we can easily check on what is causing the increase since 2019. “Firearm-related deaths” is an interesting phrase, since it lumps together three very different kinds of deaths: homicides, suicides, and accidents. And while it is true that “deaths from accidents” are the leading category of deaths for children, this also lumps together many different kinds of deaths: not only car crashes, but also poisonings, drownings, or accidental firearm deaths.
For Arkansas in 2022, here are the leading categories of deaths for children and teens (ages 1-19) if we break down the categories a bit:
Homicides: 66
Non-transport accidents: 58 (largest subcategories: poisonings/ODs and drowning)
Transport accidents: 52 (almost all car crashes)
Suicides: 24
Birth defects: 16
Cancers: 14
Cardiovascular diseases: 13
And no other categories are reported, because CDC WONDER won’t show you anything smaller than 10 deaths.
We might also ask what caused the increase since 2019, especially since this a report on child health and possible solutions. The death rate increased by 9 deaths per 100,000, and over 80% of the increase is accounted for by just two categories: homicides and non-transport accidents. Car crashes actually fell slightly (though the rate increased a bit, since the denominator was also smaller). Deaths from suicides, cancer, and heart diseases also declined from 2019 to 2022 among children in Arkansas, and these are the three on the list above that we would probably consider the “health” categories. Things actually got better!
But the really big increase, and very bad social trend, is the category of homicides. Among children and teens in Arkansas, it rose from 35 deaths in 2019 to 66 deaths in 2022. It almost doubled. That’s bad! But homicides are not mentioned anywhere in the article on this topic that I read (“firearm-related deaths” is the closest they get). And while car accidents are definitely a major problem, they didn’t really increase from 2019 to 2022 (among kids in Arkansas).
One more thing we can do with CDC WONDER is break down the homicides by age. The numbers so far are looking at a very broad range of children and teens, from ages 1-19. As I’ve written about before, the is a huge difference between homicide rates for older teens versus all of the kids. Indeed for Arkansas we see the same pattern, such as when I run a CDC WONDER query for single-years of age: only the ages 17, 18, and 19 show up (remember, anything less than 10 deaths won’t register in the query).
Breaking it down by five-year age groups, we see that 53 of the 66 homicides (in Arkansas among kids and teens) were for ages 15-19, that is 80% of the total. And further if we run the query by race, we see that 40 of the 66 homicides were for African Americans age 15-19. This is clearly a social problem, but it’s an extremely concentrated social problem. And the increase for older teen Blacks has been large too: it was just 17 deaths in 2019, more than doubling to 40 homicides in 2022.
Now, small numbers can jump around a bit, so just looking at 2019 and 2022 might be deceptive. What if we had a longer annual series to look at? Again, CDC WONDER allows us to do this. Here is the chart for homicides among older Black teens in Arkansas:
This is a dramatic chart. The steady rise in homicides among this demographic since 2019 is staggering. Not only the dramatic increase, but notice that 2021 and 2022 are much worse than the crime wave of the early 1990s, which also jump out in this chart. The homicide rate for older Black teens in 2022 was almost 50 percent higher than 1995, the prior worst year on record.
So is there a problem with child and teen deaths in Arkansas? Yes! But with just a few minutes of searching on CDC WONDER, I think we can get a much better picture of what is causing it than the article I read summarizing the report. Indeed, if we read the full national report, the word “homicide” is only mentioned once in a laundry list of many causes of death.
The most important part of addressing a social problem, such as “deadliest year on record for child deaths in Arkansas” is to know some basic details about what is causing a bad social indicator to worsen. Hopefully after reading this blog post you know a little bit more. If you want to read my summary of the research on how to reduce deaths from firearms, see this June 2022 post.
Financial markets have sustained themselves for nearly two years now on the hope that within 1-2 quarters, the Fed will finally relent and start lowering interest rates. This hope gets dashed again and again by data showing stubbornly persistent high employment, high GDP growth, and high inflation, but the hope refuses to die.
Long-term interest rates had been falling nicely for the last month, based on expectations of rate cuts in the fall. Then came Friday’s jobs report, and, blam, up went 10-year rates again. The Bureau of Labor Statistics (BLS) published its “Establishment” survey of data gleaned from employers. Non-farm payrolls rose by US 272k. This was appreciably higher than the 180k consensus expectation.
The plot below indicates that this number fits into a trend of essentially steady, fairly high employment gains (suggesting ongoing inflationary pressures):
There are fundamental reasons to take the BLS Establishment figures with a grain of salt. They have a history of significant revisions some months after first publication. Also, BLS uses a “birth/death” model for small businesses, which can account for some 50% (!) of the job gains they report. [1]
Another factor is that all of the net “jobs” created in recent quarters are reported to be part-time. According to Bret Jensen at Seeking Alpha, “Part-time jobs rose 286,000 during the quarter, while full-time jobs fell by just over 600,000. This is a continuation of a concerning trend where over the past year, roughly 1.5 million part-time positions were created while approximately one million full-time jobs were lost. This difference is that the BLS survey does not account for people working two or three jobs, which are now at a record as many Americans have struggled to maintain their standard of living during the inflationary environment of the past couple of years.”
It seems, then, that this week’s huge “jobs added” figure is not to be taken as indicating that the economy is overheated. However, it is still warm enough that rate cuts will be postponed yet again. A different BLS survey (“Household”) showed unemployment creeping up from 4.0% to 4.1%, which again suggests a more or less steady and fairly robust employment picture.
As far as drivers of inflation, I would look especially at wage growth. That is fitfully slowing, but not nearly enough to get us to the Fed’s 2% annual inflation target. My sense is that ongoing enormous federal deficit spending will keep pumping money into the economy fast enough to keep inflation high. High inflation will prevent significant interest rate cuts, assuming the Fed remains responsible. The interest payments on the federal debt will balloon due to the high rates, leading to even more deficit spending. If we actually get an economic downturn, leading to job insecurity and a willingness of workers to accept slower wage growth in the private sector, the federal spending floodgates will open even wider.
This makes hard assets like gold look attractive, to hedge against inflating U.S. dollars. This is one reason China has been quietly selling off its dollar hoard, and buying gold instead.
[1] For more in-depth treatments of employment statistics, see posts by fellow blogger Jeremy Horpedahl, e.g. here.
When I give talks about AI, I often present my own research on ChatGPT muffing academic references. By the end I make sure that I present some evidence of how good ChatGPT can be, to make sure the audience walks away with the correct overall impression of where technology is heading. On the topic of rapid advances in LLMs, interesting new claims from a person on the inside can by found from Leopold Aschenbrenner in his new article (book?) called “Situational Awareness.” https://situational-awareness.ai/ PDF: https://situational-awareness.ai/wp-content/uploads/2024/06/situationalawareness.pdf
He argues that AGI is near and LLMs will surpass the smartest humans soon.
AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year. We would rapidly go from human-level to vastly superhuman AI systems. The power—and the peril—of superintelligence would be dramatic.
Based on this assumption that AIs will surpass humans soon, he draws conclusions for national security and how we should conduct AI research. (No, I have not read all if it.)
Isn't it true that the "smart high schooler" can just repeat what they learned in a textbook? Why is it a linear progression from there to an AI researcher who is producing novel brilliant papers?
I might offer to contract out my services in the future based on my human instincts shaped by growing up on internet culture (i.e. I know when they are joking) and having an acute sense of irony. How is Artificial General Irony coming along?
You know how perfect diversification means that one bears no idiosyncratic risk? That means that one is willing to pay more for some given return, driving up the price of assets included in such a diversified portfolio. That means that, without an informational advantage, index funds should place upward pressure on the price of assets that compose them. Anyone who invests in individual stocks, again without an informational advantage, would be priced out of the market because they bear idiosyncratic risk and would need to enjoy a risk premium that lowers the maximum price that they are willing to pay.
This week the University of the Arts in Philadelphia announced they were closing effective immediately, leaving students scrambling to transfer and faculty desperate for jobs. U Arts now joins Cabrini University and Birmingham-Southern as some the 20 US colleges closing or being forced to merge so far this year. This trend of closures is likely to accelerate given falling birth rates that mean the number of college-age Americans is set to decline for decades; short-term issues like the FAFSA snafu and rising interest rates aren’t helping either.
All this makes it more important for potential students and employees to consider the financial health of colleges they might join, lest they find themselves in a UArts type situation. But how do you predict which colleges are at significant risk of closing? One thing that jumps out from this year’s list of closures is that essentially every one is a very small (fewer than 2000 undergrad) private school. Rural schools seem especially vulnerable, though this year has also seen plenty of closures in major cities.
There appear to be a number of sourcestracking the financial health of colleges, though most are not kept up to date well. Forbes seems to be the best, with 2023 ratings here; UArts, Cabrini, and Birmingham-Southern all had “C” grades. If you have access to them, credit ratings would also be good to check out; Fitch offers a generally negative take on higher ed here.
In a 2020 Brookings paper, Robert Kelchen identified several statistically significant predictors of college closures:
I used publicly available data compiled by the federal government to examine factors associated with college closures within the following two to four years. I found several factors, such as sharp declines in enrollment and total revenue, that were reasonably strong predictors of closure. Poor performances on federal accountability measures, such as the cohort default rate, financial responsibility metric, and being placed on the most stringent level of Heightened Cash Monitoring, were frequently associated with a higher likelihood of closure. My resulting models were generally able to place a majority of colleges that closed into a high-risk category
The Higher Learning Commission reached similar conclusions. Of course, there is a danger in identifying at-risk colleges too publicly:
Since a majority of colleges identified of being at the highest risk of closure remained open even four years later, there are practical and ethical concerns with using these results in the policy process. The greatest concern is that these results become a self-fulfilling prophecy— being identified as at risk of closure could hasten a struggling college’s demise.
Still, would-be students, staff and faculty should do some basic research to protect themselves as they considering enrolling or accepting a job at a college. College employees would also do well to save money and keep their resumes ready; some of these closures are so sudden that employees find out they are out of a job effective immediately and no paycheck is coming next month.
While many data watchers eagerly anticipate the monthly jobs report coming out this Friday, today the Bureau of Labor Statistics released another set of jobs data, and arguably a much better and more complete set of jobs data for 2023. It’s called the Quarterly Census of Employment and Wages, and I have written about this data before.
The QCEW data is better because, as the name implies, it is a census of employment, rather than just a survey, meaning it is an attempt to measure the universe of employment (or at least, the universe of employment covered by unemployment insurance, which is something like 95% of the workforce). Surveys are nice, because they can provide us more timely information — notice that the QCEW is 5-6 months out of date. It is also useful to have this complete data to check on the monthly data and see if it was mostly accurate — indeed, the data is updated through a process called “benchmarking” on a regular basis.
What do the latest QCEW show us? The headline number is that total employment grew by 2.3 million jobs from December 2022 to December 2023, which is 1.5% job growth (if we use annual averages, growth is a little stronger at 2%). That’s a healthy rate of job growth, but it’s less than the familiar Nonfarm Payroll series (CES) shows from December to December: about 3 million jobs added, or a growth rate of 1.8% If we focus just on private-sector employment, we see again that the monthly series is running faster than the more comprehensive QCEW: 2.3 million jobs in the monthly report added versus 1.7 million.
Does all this mean that the monthly jobs numbers are “fake”? Of course not. Surveys will always be imperfect, but they are still useful. But it does mean that you might want to discount them by about 25 percent.
