Are smartphones bad for kids? Cal walks through the data on this question, including how researchers came to be worried, their findings, critiques of their findings, and where we are today. He then gives recommendations for how to think about technology when it comes to your kids.
In May of 2023, Cal Newport shared well-informed opinions about whether smartphones harm young people. In the first half of the podcast, he talks about depression and loneliness data.
Minute 30 of the podcast: Screentime harms teenagers because they inhibit the development of critical thinking skills. Deep critical thinking skills require training. Reading an analog book is better than screens (see my review of Tyler’s AI generative book and poastmodernism).
See my summary of Emily Oster on video games for kids. She does not clutch her pearls over violent video games. However, she is concerned about what activities get crowded out by screentime. She is especially worried about sleep, because on that topic the data are clear.
Minute 31, Call Newport: Tweens and teens scroll on their phones for too long instead of going to sleep. A 13-year-old boy with a smart phone will “be up until 4 in the morning.” A tween told him that middle school girls arrive at school too exhausted to function because they have been on their phones all night.
FYI, if you are the parent in an Apple device network, you can set time limits on the devices in your family. I filed this report about smart watches last year, incidentally in the same week as the release of Newport’s podcast episode.
One mom told me that Apple Watch is considerably more durable than a Gizmo (I wouldn't know from experience). And, my son described "Watch Jail" where his teacher keeps smart watches overnight if there are usage infractions during the school day.
There is a new generative AI product called Phind. Does it make up fake citations like ChatGPT? I will report experimentation with the free-tier product from January 7, 2023.
Phind is an intelligent search engine and assistant for programmers.
Prompt: “In less than 10 sentences, explain how the ideas of Hayek are connected to bailing out failed banks, and include citations from published papers as a list. include author, year in parentheses, and journal for the citations.”
Friedrich Hayek, a prominent Austrian economist, advocated for a free market system where businesses and individuals make decisions based on individual knowledge and self-interest, rather than central planning by government or bureaucrats. His ideas suggest that interventions, such as bailing out failed banks, should be minimal and temporary, allowing the market to correct itself.
However, the financial crisis of 2008, including the failure of many banks, led to significant government intervention, including the bailout of banks. This action can be seen as a departure from Hayek’s principles, as it involves direct government intervention to stabilize the financial system.
According to a paper by Todd Zywicki (2009), the Federal Reserve’s artificial lowering of short-term interest rates and the resulting shift by consumers to adjustable rate mortgages (ARMs) triggered the housing bubble and subsequent crisis [Source 1](https://reason.org/wp-content/uploads/files/federal_reserve_monetary_policy_hayek_rule.pdf). This aligns with Hayek’s critique of central banking’s manipulation of interest rates.
Moreover, the Basel Accords, international capital adequacy and liquidity requirements for banks, were designed to encourage banks to hold securitized loans in their portfolios, thereby concentrating risk and leveraging up, thereby increasing their exposure to risk. This approach, contrary to Hayek’s preference for decentralization and competition, may have contributed to the financial crisis [Source 1](https://reason.org/wp-content/uploads/files/federal_reserve_monetary_policy_hayek_rule.pdf). In conclusion, while Hayek’s ideas of minimal government intervention and self-regulating markets provide a useful theoretical framework, practical implementation in complex and interconnected modern economies may require different strategies, including bank bailouts.
Are the citations real and correct? Phind simply is not trying to write with in-text citations and academic references. There is a mention of “Todd Zywicki (2009)” and also a weblink. The weblink is to a policy paper published by Reason. This policy paper is on the correct topic and the paper does exist. The Zywicki citations also exists, in the sense that if you go into the Reason paper it cites a real Wall Street Journal op-ed. So, if you follow the trail, you can see what document Phind learned to cite Zywicki from. It seems like it is stealing the citation more than reading the source documents. To be fair, sometimes human academics work that way, too.
What I was hoping for was in-text citations with a compiled bibliography at the end. Phind did not give me that.
I’ll try another test that I ran on ChatGPT and Bing in 2023.
The prompt is “In what paper did John Duffy propose that economists use experiments to study equilibrium selection?”
Phind does not attempt an in-text citation in the response. The link in the paragraph (link number 2) points to John Duffy’s list of papers, which is better than just making up fake citations but also does not create a literature review for me. Phind is, as with the Hayek test above, providing breadcrumbs of links through which I can discover existent papers.
Is there a paper called “The Transition from Stagnation to Growth: An Adaptive Learning Approach”? Yes. And it is by Duffy.
Phind lists weblinks to sources. Has Phind done more for me than Google, on this search? Not much, in terms of finding and synthesizing references.
According to a chart circulating this week, attendance at the AEA winter meeting exceeded 13,000 economists at the peak. I had some job interviews at Chicago in 2017, and I spoke on a panel at San Diego in 2020. Chicago was freezing cold. San Diego was lovely in January. Cold or not, probably every economist on the job market showed up in person to those conferences. There are harrowing stories of people who missed job interviews because of snow and cancelled flights.
A privilege of being in this profession is the chance to meet people from around the world. At my lunch table currently there are people from Chile and Southern China to my right and Kansas and Hong Kong to my left. #ASSA2020
A privilege of being in this profession is the chance to meet people from around the world. At my lunch table currently there are people from Chile and Southern China to my right and Kansas and Hong Kong to my left. #ASSA2020
It is (or was) a huge win for the US to be the meeting place. Now that job interviews have moved to Zoom, attendance has fallen by more than half. I assume that the attendees here in 2024 skew more America-based.
A European economist predicts attendance in 2024 will fall even lower because “the interview stage of the market has moved online, and at a random date between October and January. No point in flying to ASSA, especially if you are a candidate based in Europe hoping for a job in Europe”
Lower, the interview stage of the market has moved online, and at a random date between October and January. No point in flying to ASSA, especially if you are a candidate based in Europe hoping for a job in Europe
Instead of asking why the economics job market unraveled, maybe we should be surprised that it was ever so centralized. People really did travel from all over the world to an American city in January to interview for almost every serious job available in the economics profession.
Small points:
I’ve seen several people complain that it’s a bad time of year. For me, it’s a good time of year. Since I do a lot of teaching, it’s good to have a conference that happens before my teaching starts.
I was happy to see that childcare is available here (for a fee).
I talked to a job market candidate who was trying to weigh a tenure-track job offer from a teaching school against the possible offer of a postdoc. He would prefer the postdoc. Because of the new system, job offers come in sequence. He risks losing the teaching offer before he finds out if he will get the postdoc offer in writing. In the old days, candidates would have been more likely to know what their best option was going to be.
The online JOE job board still serves for coordination. It is still administered by the AEA for jobs globally.
James provided evidence that the AEA will start letting more people get on the program to present research. (By contrast, the Southern Economic Association meeting has actually increased in attendance since Covid. They allow accept more people to present research on the program.)
When my grad school research group went out for lunch, I was sometimes the only American at the table. I loved it. I’m grateful that those brilliant people came here to start their careers.
Back in August I listed the most-read posts of 2023. Here I will finish out the year by listing a few more highlights. This has been another big year for our website.
Zachary has been giving out good advice for economics teachers, backed up by his data. All professors can read: 5 Easy Steps to Improve Your Course Evals. Econ professors check: Update on Game Theory Teaching. It’s about how to teach Game Theory, but I also see it as a testament to how much a course can improve if you allow a teacher to iterate multiple times at the same school. Administrators, take note.
What We Are Learning about Paper Books is jointly my reflection on AI generative books and a review of Tyler Cowen’s new book GOAT. I’m a techno-optimist, but I think there is value in an old-fashioned paper book, mostly from a behavioral or neuro perspective.
I have a paper that emphasizes ChatGPT errors. It is important to recognize that LLMs can make mistakes. However, someone could look at our data and emphasize the opposite potential interpretation. On many points, and even when coming up with citations, the LLM generated correct sentences. More than half of the content was good.
Apparently, LLMs just solved an unsolvable math problem. Is there anything they can’t do? Considering how much of human expression and culture revolves around religion, we can expect AI’s to get involved in that aspect of life.
Alex thinks it will be a short hop from Personal Jesus Chatbot to a whole new AI religion. We’ll see. People have had “LLMs” in the form of human pastors, shaman, or rabbis for a long time, and yet sticking to one sacred text for reference has been stable. I think people might feel the same way in the AI era – stick to the canon for a common point of reference. Text written before the AI era will be considered special for a long time, I predict. Even AI’s ought to be suspicious of AI-generated content, just in the way that humans are now (or are they?).
Many religious traditions have lots of training literature. (In our ChatGPT errors paper, we expect LLMs to produce reliable content on topics for which there is plentiful training literature.)
I gave ChatGPT this prompt:
Can you write a Bible study? I’d like this to be appropriate for the season of Advent, but I’d like most of the Bible readings to be from the book of Job. I’d like to consider what Job was going through, because he was trying to understand the human condition and our relationship to God before the idea of Jesus. Job had a conception of the goodness of God, but he didn’t have the hope of the Gospel. Can you work with that?
Back in June, I watched the livestream of the Chapman Economic Forecast with Dr. Jim Doti (who was president when I was a student at Chapman). Typically, this is a valuable informative event, and the team has an excellent record of performance. They have often outdone other forecasters in predicting the future.
That is why I feel a little bad for making this post in the summer and tweeting out Doti’s prediction that we would have a recession by now.
To be fair to Doti, there has been a lot of uproar over this issue. Lots of people thought the economy would be bad. And lots of people feel like the economy is bad (the “vibecession”) even though it is objectively not. Many tweets have gone by about it.
Doti opened by saying his prediction had turned out to be wrong. He had an explanation for it (pictured below). You can watch it free here (recorded on Dec 14).
Doti said that he had expected a large fiscal stimulus in the form of deficit spending, however he had not expected the deficit to be so large. Debt-financed spending propped up an economy that was otherwise poised to contract. At least, that is a plausible story.
Looking forward, Doti does not predict a recession next year, but he does predict weak growth and possibly one quarter of GPD decline (not two).
The next part of talk was about the long-term consequences of deficit spending. Nothing is free. TANSTAAFL
In addition to vibecession, anyone following economics in 2023 needs to know what a “soft landing” is.
Fed Chair Powell: "There's little basis for thinking the economy is in a recession now."
The latest Fed projections show a soft landing in 2024.
Growth slowing, but positive +1.4% Inflation cooling to ~2.4% Unemployment rising to 4.1% (so just below recession trigger) pic.twitter.com/6cvgC6l5Nd
We study whether people will pay for a fact-check on AI writing. ChatGPT can be very useful, but human readers should not trust every fact that it reports. Yesterday’s post was about ChatGPT writing false things that look real.
The reason participants in our experiment might pay for a fact-check is that they earn bonus payments based on whether they correctly identify errors in a paragraph. If participants believe that the paragraph does not contain any errors, they should not pay for a fact-check. However, if they have doubts, it is rational to pay for a fact-check and earn a smaller bonus, for certain.
Abstract: We explore whether people trust the accuracy of statements produced by large language models (LLMs) versus those written by humans. While LLMs have showcased impressive capabilities in generating text, concerns have been raised regarding the potential for misinformation, bias, or false responses. In this experiment, participants rate the accuracy of statements under different information conditions. Participants who are not explicitly informed of authorship tend to trust statements they believe are human-written more than those attributed to ChatGPT. However, when informed about authorship, participants show equal skepticism towards both human and AI writers. There is an increase in the rate of costly fact-checking by participants who are explicitly informed. These outcomes suggest that trust in AI-generated content is context-dependent.
Our original hypothesis was that people would be more trusting of human writers. That turned out to be only partially true. Participants who are not explicitly informed of authorship tend to trust statements they believe are human-written more than those attributed to ChatGPT.
We presented information to participants in different ways. Sometimes we explicitly told them about authorship (informed treatment) and sometimes we asked them to guess about authorship (uninformed treatment).
This graph (figure 5 in our paper) shows that the overall rate of fact-checking increased when subjects were given more explicit information. Something about being told that a paragraph was written by a human might have aroused suspicion in our participants. (The kids today would say it is “sus.”) They became less confident in their own ability to rate accuracy and therefore more willing to pay for a fact-check. This effect is independent of whether participants trust humans more than AI.
We are thinking of fact-checking as often a good thing, in the context of our previous work on ChatGPT hallucinations. So, one policy implication is that certain types of labels can cause readers to think critically. For example, Twitter labels automated accounts so that readers know when content has been chosen or created by a bot.
Suggested Citation: Buchanan, Joy and Hickman, William, Do People Trust Humans More Than ChatGPT? (November 16, 2023). GMU Working Paper in Economics No. 23-38, Available at SSRN: https://ssrn.com/abstract=4635674
Citation: Buchanan, J., Hill, S., & Shapoval, O. (2024). ChatGPT Hallucinates Non-existent Citations: Evidence from Economics. The American Economist. 69(1), 80-87 https://doi.org/10.1177/05694345231218454
Blog followers will know that we reported this issue earlier with the free version of ChatGPT using GPT-3.5 (covered in the WSJ). We have updated this new article by running the same prompts through the paid version using GPT-4. Did the problems go away with the more powerful LLM?
The error rate went down slightly, but our two main results held up. It’s important that any fake citations at all are being presented as real. The proportion of nonexistent citations was over 30% with GPT-3.5, and it is over 20% with our trial of GPT-4 several months later. See figure 2 from our paper below for the average accuracy rates. The proportion of real citations is always under 90%. GPT-4, when asked about a very specific narrow topic, hallucinates almost half of the citations (57% are real for level 3, as shown in the graph).
The second result from our study is that the error rate of the LLM increases significantly when the prompt is more specific. If you ask GPT-4 about a niche topic for which there is less training data, then a higher proportion of the citations it produces are false. (This has been replicated in different domains, such as knowledge of geography.)
What does Joy Buchanan really think?: I expect that this problem with the fake citations will be solved quickly. It’s very brazen. When people understand this problem, they are shocked. Just… fake citations? Like… it printed out reference for papers that do not actually exist? Yes, it really did that. We were the only ones who quantified and reported it, but the phenomenon was noticed by millions of researchers around the world who experimented with ChatGPT in 2023. These errors are so easy to catch that I expect ChatGPT will clean up its own mess on this particular issue quickly. However, that does not mean that the more general issue of hallucinations is going away.
Not only can ChatGPT make mistakes, as any human worker can mess up, but it can make a different kind of mistake without meaning to. Hallucinations are not intentional lies (which is not to say that an LLM cannot lie). This paper will serve as bright clear evidence that GPT can hallucinate in ways that detract from the quality of the output or even pose safety concerns in some use cases. This generalizes far beyond academic citations. The error rate might decrease to the point where hallucinations are less of a problem than the errors that humans are prone to make; however, the errors made by LLMs will always be of a different quality than the errors made by a human. A human research assistant would not cite nonexistent citations. LLM doctors are going to make a type of mistake that would not be made by human doctors. We should be on the lookout for those mistakes.
ChatGPT is great for some of the inputs to research, but it is not as helpful for original scientific writing. As prolific writer Noah Smith says, “I still can’t use ChatGPT for writing, even with GPT-4, because the risk of inserting even a small number of fake facts… “
I still can't use ChatGPT for writing, even with GPT-4, because the risk of inserting even a small number of fake facts or bad interpretations into a blog post is unacceptable, meaning that it requires so much time to fact-check that it doesn't save effort.
Imagine trying to explain the world today to a person who time traveled forward from 300 years ago. How could someone who lived in France in the year 1600 understand our modern problems?
Person from the Past: So, how is it with 8 billion people?
Me Today: It’s bad. We have too many clothes.
PftP: Right. With 8 billion you wouldn’t have enough clothes for everyone.
MT: Too many.
PftP: Not enough?
MT: I said we have TOO MANY clothes. Not even the poorest people in the world want them. Shirts pile up on the beaches and pollute the ocean.
PftP: …
My article highlights the fact that we live in an era of unprecedented clothing abundance. First, that was not always true.
Most of human history has been characterized by privation and low‐productivity toil. As one American sharecropper exclaimed in John Steinbeck’s Depression‐era novel The Grapes of Wrath, “We got no clothes, torn an’ ragged. If all the neighbors weren’t the same, we’d be ashamed to go to meeting.”
The United Nations Economic Commission for Europe called the fashion industry an “environmental and social emergency” because clothing production has roughly doubled since the year 2000. Their main concerns are fast fashion’s environmental impact and working conditions.
Some of my article is a response to the critics of modern low-cost mass production.
Thirdly, I explain how we could keep most of the benefits of cheap clothes with less litter in the environment. The item I am most optimistic about is using our new artificial intelligence tools to re-sort the world’s junk. We would produce and throw away fewer clothes if we had a better system for rearranging the stock of goods that we already have. The problem I see today is that I have “perfectly good” clothes in my house that I don’t really want; however, attention and time are so scarce that no one will pay me for them. Even if I donate them, I worry that half will end up in the trash. Someone on this earth could use them but identifying that someone and making the trade still has high prohibitively high transaction costs. Very smart AI could come to my house and scan my stuff and pay me for it because very smart AI could get it to someone with a positive value for it.
Asked for its methodology, the White House pointed us to a Nov. 15 blog post by Jeremy Horpedahl, an associate economics professor at the University of Central Arkansas.
“When people ask what Civilization VI’s “cultural victory” condition would look like in the real world, this is it. Write it in the manual.”
When people ask what Civilization VI’s “cultural victory” condition would look like in the real world, this is it. Write it in the manual. https://t.co/i88RmJF7qr
Economist Writing Every Day 