Author: Joy Buchanan

Trusting ChatGPT at JBEE

Joy BuchananLeave a comment

You can find my paper with Will Hickman “Do people trust humans more than ChatGPT?” at the Journal of Behavioral and Experimental Economics (JBEE) online, and you can download it free before July 30, 2024 (temporarily ungated*).

*Find a previous ungated draft at SSRN.

Did we find that people trust humans more than the bots? It’s complicated. Or, as we say in the paper, it’s context-dependent.

When participants saw labels informing them (e.g. “The following paragraph was written by a human.”) about authorship, readers were more likely to purchase a fact-check (the orange bar).

Informed subjects were not more trusting of human authors versus ChatGPT (so we couldn’t reject the null hypothesis about trusting humans, in that sense). However, Informed subjects were significantly less likely to trust their own judgement of the factual accuracy of the paragraph in the experiment, relative to readers who saw no authorship labels.

Some regulations would make the internet more like our Informed treatment. The EU may mandate that ChatGPT comply with the obligation of: “Disclosing that the content was generated by AI.” Our results indicate that this policy would affect behavior because people read differently when they are forced to think up front about how the text was generated.

Inspiration for this article on trust began with observing the serious errors that can be produced by LLMS (e.g. make up fake citations). Our hypothesis was that readers are more trusting of human authors, because of these known mistakes by ChatGPT. This graph shows that participants trust (left blue bar = “High Trust”) statements *believed* to have been written by a human (so, in that sense, our main hypothesis has some confirmation).

Conversely, in the Informed treatment, readers are equally uncertain about text written either by humans or bots. Informed readers are suspicious, so they buy a fact-check. “High Trust” (the blue bar) is the option that maximizes expected value if the reader thinks the author has not made factual errors.

So, in conclusion, we find that human readers can be made more suspicious by framing. In this case, we are thinking of being cautious and doing a fact-check as a good thing. The reason is that, increasingly, the new texts of society are being written by LLMs. Evidence of this fact has been presented by Andrew Gray in a 2023 working paper: “ChatGPT “contamination”: estimating the prevalence of LLMs in the scholarly literature” Note that is the scholarly literature, not just the sports blogs or the Harry Potter – Taylor Swift- crossover fanfics.

What about the medical doctors? What is the authority on whether you are getting surgery or not? See: “Delving into PubMed Records: Some Terms in Medical Writing Have Drastically Changed after the Arrival of ChatGPT”

Continue reading →

Latest from Leopold on AGI

Joy BuchananLeave a comment

When I give talks about AI, I often present my own research on ChatGPT muffing academic references. By the end I make sure that I present some evidence of how good ChatGPT can be, to make sure the audience walks away with the correct overall impression of where technology is heading. On the topic of rapid advances in LLMs, interesting new claims from a person on the inside can by found from Leopold Aschenbrenner in his new article (book?) called “Situational Awareness.”
https://situational-awareness.ai/
PDF: https://situational-awareness.ai/wp-content/uploads/2024/06/situationalawareness.pdf

He argues that AGI is near and LLMs will surpass the smartest humans soon.

AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year. We would rapidly go from human-level to vastly superhuman AI systems. The power—and the peril—of superintelligence would be dramatic.

Based on this assumption that AIs will surpass humans soon, he draws conclusions for national security and how we should conduct AI research. (No, I have not read all if it.)

I dropped in that question and I’m not sure if anyone has, per se, an answer.

You can also get the talking version of Leopold’s paper in his podcast with Dwarkesh.

I’m also not sure if anyone is going to answer this one:

I might offer to contract out my services in the future based on my human instincts shaped by growing up on internet culture (i.e. I know when they are joking) and having an acute sense of irony. How is Artificial General Irony coming along?

Gear Swaps are Happening

Joy BuchananLeave a comment

Everyone feels like we throw away too much stuff. One small way to help is to try to find someone who can use the items before you toss them.

I’m happy to say that one of my economic ideas got to the policy implementation stage. I was staring at the Scout gear my son had grown out of and dreading the thought of throwing it away. I could donate it to Good Will, but I thought that the chances it would get to someone who wants it are very low. What parent wants exactly that stuff? So, I emailed our Pack leader and asked if we could start doing a gear swap.

Parents can bring any scout-related items that they do not want anymore to a pack meeting. It is organized on one table with clear information. Anyone can take anything for free if they can use it and store it.  

This works better than posting to internet Buy Nothing groups because the scout parents are right there. No one has to drive across town for a “porch pick up.”

More sports teams or clubs should do this. Seize the moments when like-minded people are already together in one place.

Previously from me on Fast Fashion:

Secondhand for AdamSmithWorks

Is the repair revolution coming?

Joy’s Fashion Globalization Article with Cato

Do Less for Preschool

Joy Buchanan7 Comments

Today I will write about something I care deeply about: the wellbeing of the moms of young children.

I can remember having a child enrolled in preschool. It was expensive but it was worth it, for us. What follows will be most relevant to readers who are working full-time and have children enrolled in full-time daycare/preschool. That is not the right choice for every family. If it’s the choice you made, then read on.

Do less for preschool. Save your energy and money for the years when your child will actually remember.

Continue reading →

Humanity’s Childhood and Chiefs

Joy Buchanan2 Comments

I’m going to explore a passage from The Dawn of Everything about whether humans reject Western civilization.

The introductory chapter of The Dawn of Everything is called “Farewell to Humanity’s Childhood.” The authors are idealists wrestling with big questions.

We can take [Steven] Pinker as our quintessential Hobbesian. (page 13)

For instance, if Pinker is correct, then any sane person who had to choose between (a) the violent chaos and abject poverty of the ‘tribal’ stage in human development and (b) the relative security and prosperity of Western civilization would not hesitate to leap for safety. (page 18)

Over the last several centuries, there have been numerous occasions when individuals found themselves in a position to make precisely this choice – and they almost never go the way Pinker would have predicted.

Continue reading →

Accounting Appears Before Literature

Joy Buchanan1 Comment

For a current research project on institutions, I skimmed The Dawn of Everything (2021).

I liked this passage about an archaeological site in Syria. The following items were found in a destroyed village where people are estimated to have lived 8,000 years ago:

These devices included economic archives, which were miniature precursors to the temple archives at Uruk and other later Mesopotamian cities.

These were not written archives: writing, as such, would not appear for another 3,000 years. What did exist were geometric tokens made of clay, of a sort that appear to have been used in many similar Neolithic villages, most likely to keep track of the allocation of particular resources.

In chunks, the book has fascinating stuff like the quote above. However, D-o-E is the second book I have read this year that tries to do too much. A book on “everything” sounds incredibly fun to write, and I’m the type who would try, so I take these as a warning.

What is more intriguing than history? Emily Wilson said it well, concerning some of the oldest records we have of human words:

I think we should stop selling classics as, “These are the societies that formed modern America, or that formed the Western canon” — which is a really bogus kind of argument — and instead start saying, “We should learn about ancient societies because they’re different from modern societies.” That means that we can learn things by learning about alterity. We can learn about what would it be to be just as human as we are, and yet be living in a very, very different society.

Sugar Fast Blog

Joy Buchanan5 Comments

Why do Americans eat a lot of junk food? Because it’s the easy way out.

Unhappy? Open a candy bar. You’ll feel happy again in seconds. Kid crying? Hand them a fun-sized candy bar. They will be quiet.  

If you are struggling with paying bills or health (I know, the health one is ironic here), then you’ll tend to reach for anything that is fast and easy to deal with immediate problems.

For me, I decided to wait until my semester is over, so I won’t be attempting this while teaching or traveling. A 40-day sugar fast for the whole family technically began on May 1, but the grocery shopping changed earlier. The idea was to eat down junk and not buy new for over a week.

Forty days isn’t much in the big scheme. The idea is to make a deposit on health. Possibly, I’ll break a mild sugar addiction to the point where the body doesn’t expect it so much. Maybe something that we end up doing to meet this artifactual goal will end up getting into the routine on a regular part of the year when there is more travel and work. Part of the problem I identify is that there are points throughout the day where people feel unhappy. If sugar is on hand, then there is a tendency to reach for it. Part of what I’m going to do is insert more healthy food and activities, but of course that is a lot more work. If it’s just not there, people barely miss it.

I’m already so much happier at home. There is barely any sugary junk food left in the house. Now if the kids circulate the kitchen, I don’t have to stress out and yell at them to not eat cookies before dinner or whatnot.

Internet: So, you’re going to meal plan and not eat dessert for a month? This was worth telling everyone?

Me: I’ve been thinking about it constantly since Christmas.

Internet: Wasn’t this the site where we get more optimistic about the world?

Me: There are some things I read about and decided against. I will not worry about sugar in sauces (e.g. Chicken teriyaki bowl). I will not cut out bread or pasta. There is a sliding scale of how healthy you can be and how much time you are willing to put in. I have decided on a level of effort and a fixed amount of time. I’m not even going to turn down cookies if they are offered to us for free. The most important thing is to stop buying junk from the grocery store. It’s financially very cheap, but actually very costly.

P.S. It’s a small step toward getting my personal chef, but I saw an ad for Walmart “emeals” which is more intelligent grocery delivery plus recipes. I haven’t tried it myself, but it seemed like an update on What the Superintelligence Can Do For Us. When I have the equivalent of “former restaurateur, Frances,” in my house, then I just won’t need anything else and innovation can stop there, thanks.

Zuckerberg wants to solve general intelligence

Joy Buchanan2 Comments

Why does Mark Zuckerberg want to solve general intelligence? Well, for one thing, if he doesn’t, one of his competitors will have a better chatbot. Zuckerberg wants to be the best (and good for him). At his core, he wants to build the best stuff (even the world’s best cattle on his ranch).

If AGI is possible, it will get built. I’m not the first person to point out that this is a new space race. If America takes a pause, then someone else will get there first. However, I thought the Zuck interview was an interesting microcosm for why AGI, if possible, will get made.

… We started FAIR about 10 years ago. The idea was that, along the way to general intelligence or whatever you wanna call it, there are going to be all these different innovations and that’s going to just improve everything that we do. So we didn’t conceive of it as a product. It was more of a research group. Over the last 10 years it has created a lot of different things that have improved all of our products. …
There’s obviously a big change in the last few years with ChatGPT and the diffusion models around image creation coming out. This is some pretty wild stuff that is pretty clearly going to affect how people interact with every app that’s out there. At that point we started a second group, the gen AI group, with the goal of bringing that stuff into our products and building leading foundation models that would power all these different products.
… There’s also basic assistant functionality, whether it’s for our apps or the smart glasses or VR. So it wasn’t completely clear at first that you were going to need full AGI to be able to support those use cases. But in all these subtle ways, through working on them, I think it’s actually become clear that you do. …
Reasoning is another example. Maybe you want to chat with a creator or you’re a business and you’re trying to interact with a customer. That interaction is not just like “okay, the person sends you a message and you just reply.” It’s a multi-step interaction where you’re trying to think through “how do I accomplish the person’s goals?” A lot of times when a customer comes, they don’t necessarily know exactly what they’re looking for or how to ask their questions. So it’s not really the job of the AI to just respond to the question.
You need to kind of think about it more holistically. It really becomes a reasoning problem. So if someone else solves reasoning, or makes good advances on reasoning, and we’re sitting here with a basic chat bot, then our product is lame compared to what other people are building. At the end of the day, we basically realized we’ve got to solve general intelligence… (emphasis mine)

Credit to Dwarkesh Patel for this excellent interview. Credit to M.Z. for sharing his thoughts on topics that affect the world.

“we’ve got to solve general intelligence” If a competitor solves AGI first, then you are left behind. No one would not want general intelligence on their team, on the assumption that it can be controlled.

I would like the AGI to do my chores for me, please. Unfortunately, it’s more likely to be able to write my blog posts first.

Joy on The Inductive Economy podcast

Joy Buchanan1 Comment

I got to be a guest of Vignesh Swaminathan who is based in Mumbai. It’s fun to have a deep conversation with someone on the other side of the world and share it with the whole internet (and the AI’s).

Apple podcast link: https://podcasts.apple.com/us/podcast/dr-joy-buchanan-on-understanding-economics-through/id1719744197?i=1000652541934

Blogpost with links and timestamps: https://www.inductive.in/p/dr-joy-buchanan-on-understanding

The first 10 minutes are about Tyler’s GOAT book. Vignesh asked me to name some influential economists who did not make Tyler’s list.

Around minute 12 we talk about the experimental economics methodology.

The middle (minute 15-42) is a discussion of the pipeline into tech and my Willingness to be Paid paper. He adds his perspective on tech jobs in India.

Around minute 42, Vignesh makes a switch over to the Barbie movie and then Oppenheimer. He observes that Oppenheimer is a “brand.” I speculate on careers in Barbieland. We recorded this before Christmas of ’23, right after everyone had seen these summer movies. Both movies ended up in the 2024 Oscars awards ceremony.

I predicted that people will eventually be able to create a custom movie from a verbal prompt, because of the AI content revolution. Here in Spring of ’24 that has already come true. Sora is shocking everyone and even caused Tyler Perry to halt a physical film studio expansion.

Around minute 55, we pivot to Hayek and competition, which leads to a postmortem on Google Plus (RIP).

1:05-1:16 features intellectual property and my IP experiment with Bart Wilson

Ended with rapid-fire and personal questions.

Skimming back through this conversation has me thinking about tech work. The market for IT workers and programmers has evolved since I first started the project that became “Willingness to be Paid: Who Trains for Tech Jobs?”

I like pointing people all the way back to this report on jobs from 1958. Learn to Code has been good advice for a long time, for the people who can tolerate the work. That does not mean it will be true forever, but I would argue that it is still true today.

Silicon Valley as a career might have peaked around 2021. It’s not going away, but it might not be growing anymore in terms of the number of talented people who can be absorbed there. (Might I suggest Huntsville instead?)

The WSJ recently ran a story “Tech Job Seekers Without AI Skills Face a New Reality: Lower Salaries and Fewer Roles”

The rise of artificial intelligence is affecting job seekers in tech who, accustomed to high paychecks and robust demand for their skills, are facing a new reality: Learn AI and don’t expect the same pay packages you were getting a few years ago.

Jobs in areas like telecommunications, corporate systems management and entry-level IT have declined in recent months, while roles in cybersecurity, AI and data science continue to rise, according to Janco’s data. The average total compensation for IT workers is about $100,000, making the position a target for continued cost-cutting.

One reason tech jobs are less attractive than some other professional paths is that the skillset changes. We mentioned this as a drawback in our policy paper. Computers are constantly changing. Vignesh and I discuss the issue of risk. I suggested that companies could pay less for talent if they were willing to offer packages that carry less risk of getting fired.

Nevertheless, tech still has decent job prospects. An unemployment rate of about 5% is about normal for work, even though tech had seen lower rates at the peak of demand. I do not know what programming as a career will look like in 10 years, but I’d say the same about screenwriting and live sports commentary. The LLMs are coming for everything or nothing or something in between.

I’ve been on tour (regionally) with our ChatGPT paper and getting opportunities to query different audiences about their LLM use. Last week I talked to a young man in our business school who is using ChatGPT to write SQL code at his job. I said in the podcast that I would still advise young people in Alabama to learn to code, even if they are not going to move to Silicon Valley. I think coding is more fun in the LLM-age or at least less miserable.

Hanging the curtains back up

Joy Buchanan3 Comments

There were not a lot of successful female writers and academics in the 1970’s. Maybe I underestimate how many there were, but obviously they would have been in the minority. I’m reading a chapter on the anthropologist Mary Douglas who somehow combined raising three children with remaining active in academia. I read a few pages while helping at the Cub Scout camping trip.

In one of her books, Douglas added an apology for professional duties eclipsing domestic ones: ‘All our things have fallen into neglect while I have been writing, floors unpolished, curtains falling off hooks. I am grateful to my family for their patience.’

page 130 of The Slain God by Timothy Larsen

It is irksome to hear this woman apologizing for working
what is essentially two jobs and performing so well at each one. (I wouldn’t want
to put anyone off reading Larsen, who admires her very much.)

I had planned to do this a year ago, but then I ended up
writing papers on artificial intelligence and doing a bunch of related speaking
engagements. (I love it – anyone who wants a speaker on ChatGPT should invite
me out.) Anyway, I’m going to try to do the equivalent of fixing the
“curtains falling off hooks.” The curtains really do fall down. You
could have a well-functioning household and drawers full of clothes that fit
your children… and then if someone is not engaged in constant warfare… it
will all fall apart in about 6 months.