Year: 2022

Message To My Students: Don’t Use AI to Cheat (at least not yet)

Jeremy Horpedahl4 Comments

If you have spent any time on social media in the past week, you’ve probably noticed a lot of people using the new AI program called ChatGPT. Joy blogged about it recently too. It’s a fun thing to play with and often gives you very good (or at least interesting) responses to questions you ask. And it’s blown up on social media, probably because it’s free, responds instantly, and is easy to screenshot.

But as with all things AI, there are numerous concerns that come up, both theoretical and immediately real. One immediately real concern among academics is the possibility of cheating by students on homework, short writing assignments, or take-home exams. I don’t want to diminish these concerns, but I think for now they are overblown. Let me demonstrate by example.

This semester I am teaching an undergraduate course in Economic History. Two of the big topics we cover are the Industrial Revolution and the Great Depression. Specifically, we spend a lot of time discussing the various theories of the causes of these two events. On the exams, students are asked to, more or less, summarize these potential causes and discuss them.

How does ChatGPT do?

On the Industrial Revolution:

And on the Great Depression:

Now, it’s not that these answers are flat out wrong. The answers certainly list theories that have been discussed by at various times, including in the academic literature. But these answers just wouldn’t be very good for my class, primarily because they miss almost all of the theories that we have discussed in class as being likely causes. Moreover, the answers also list theories that we have discussed in class as probably not being correct.

These kinds of errors are especially true of the answer about the Great Depression, which reads like it was taken straight from a high school history textbook, ignoring almost everything economists have said about the topic. The answer for the Industrial Revolution doesn’t make this mistake as much as it misses most of the theories discussed by Koyama and Rubin, which was the main book we used to work through the literature. If a student gave an answer like the AI, it suggests to me that they didn’t even look at the chapter titles in K&R, which provide a roadmap of the main theories.

So, my message to students: don’t try to use this to answer questions in class, at least not right now. The program will certainly improve in the future, and perhaps it will eventually get very good at answering these kinds of academic questions.

But I also have a message to fellow academics: make sure that you are writing questions that aren’t easily answered by an AI. This can be hard to do, especially if you haven’t thought about it deeply, but ultimately thinking in this way should help you to write better exam and homework questions. This approach seems far superior to the one that the AI suggests.

Gambler Ruined: Sam Bankman-Fried’s Bizarre Notions of Risk and the Blow-Up of FTX

Scott Buchanan3 Comments

The drama continues for Sam Bankman-Fried (SBF), the former head of now-bankrupt crypto exchange FTX. This past week has been giving a series of interviews, in which he (the brilliant master, the White Knight, of the crypto world a mere month ago) is trying to convince us (potential jurors?) that he is too dim-witted to have masterminded a shell game of international wire transfers, and that he had no idea what was happening in the closely-held company of which he was Chief Executive Officer. (For an entertaining take on what We The People think of SBF’s disclaimers, see responses in this thread ttps://twitter.com/SBF_FTX/status/1591989554881658880, especially the video posted by “Not Jim Cramer”). 

The word on the street is that his former partner Caroline Ellison (who he has been implicitly throwing under the bus with his disclaimers of responsibility for the multi-billion dollar transfers from his FTX to her Alameda company) may well be cutting a deal with prosecutors to testify against SBF.  It remains to be seen whether SBF’s monumental political donations will suffice to keep him from doing hard time.

But all that legal drama aside, the SBF saga brings up some interesting issues on risk management. Earlier here on EWED James Bailey  highlighted a revealing exchange between SBF and Tyler Cowen, in which SBF displayed a heedless neglect of the risk of catastrophic outcomes, as long as there is a reasonable chance of great gain:

TC: Ok, but let’s say there’s a game: 51% you double the Earth out somewhere else, 49% it all disappears. And would you keep on playing that game, double or nothing?

SBF: Yeah…take the pure hypothetical… yeah.

TC: So then you keep on playing the game. What’s the chance we’re left with anything? Don’t I just St. Petersburg Paradox you into non-existence?

SBF: No, not necessarily – maybe [we’re] St. Petersburg-paradoxed into an enormously valuable existence. That’s the other option.

Boiled down, the St Petersburg Paradox involves a scenario where you have a 50% chance of winning $2.00, a 25% (1/4) chance of winning $4.00, a 1/8 chance of winning $8.00, and so on without limit. If you add up all the probabilities multiplied by the amount won for each probability, the Expected Value for this scenario is infinite. Therefore it seems like it would be rational, if you were offered a chance to play this game, to stake 100% of your net worth in one shot. However, almost nobody would actually do that; most folks might spend something like $20 or maybe 0.1% of their net worth for a shot at this, since the likely prospect of losing a large amount does not psychologically compensate for the smaller chance of gaining a much, much larger amount. But SBF is not “most folks”.

Victor Haghani recently authored an article on risk management and on SBF’s approach:

Most people derive less and less incremental satisfaction from progressive increases in wealth – or, as economists like to say: most people exhibit diminishing marginal utility of wealth. This naturally leads to risk aversion because a loss hurts more than the equivalent gain feels good. The classic Theory of Choice Under Uncertainty recommends making decisions that maximize Expected Utility, which is the probability-weighted average of all possible utility outcomes.

SBF explained on multiple occasions that his level of risk-aversion was so low that he didn’t need to think about maximizing Expected Utility, but could instead just make his decisions based on maximizing the Expected Value of his wealth directly. So what does this mean in practice? Let’s say you find an investment which has a 1% chance of a 10,000x payoff, but a 99% chance of winding up worth zero. It has a very high expected return, but it’s also very risky. How much of your total wealth would you want to invest in it?

There’s no right or wrong answer; it’s down to your own personal preferences. However, we think most affluent people would invest somewhere between 0.1% and 1% of their wealth in this investment, based on observing other risky choices such people make and surveys we’ve conducted…

SBF on the other hand, making his decision strictly according to his stated preferences, would choose to invest 100% of his wealth in this investment, because it maximizes the Expected Value of his wealth.

Even in a game with a fair 50/50 outcome, a player with finite resources will eventually go broke. This is the “Gambler’s Ruin” concept in statistics. SBF’s outsized penchant for risk took his net worth to something like $30 billion earlier this year, something we more-timid souls will never achieve, but it eventually proved to be his undoing.

Most people have a more or less logarithmic sense of the utility of money – if you only have $1000, the gain or loss of $100 is significant, whereas $100 is lost in the noise for someone whose net worth is over a million dollars. SBF apparently felt that he was playing with such big numbers, that he did not need to worry about big losses, as long as there was a chance at a big, big win. Here is a Twitter Thread  by SBF, from  Dec 10, 2020:

SBF: …What about a wackier bet? How about you only win 10% of the time, but if you do you get paid out 10,000x your bet size?

[So, if you have $100k,] Kelly* suggests you only bet $10k: you’ll almost certainly lose. And if you kept doing this much more than $10k at a time, you’d probably blow out.

…this bet is great Expected Value; you win [more precisely, your Expected Value is] 1,000x your bet size.

…In many cases I think $10k is a reasonable bet. But I, personally, would do more. I’d probably do more like $50k.

Why? Because ultimately my utility function isn’t really logarithmic. It’s closer to linear.

…Kelly tells you that when the backdrop is trillions of dollars, there’s essentially no risk aversion on the scale of thousands or millions.

Put another way: if you’re maximizing EV(log(W+$1,000,000,000,000)) and W is much less than a trillion, this is very similar to just maximizing EV(W).

Does this mean you should be willing to accept a significant chance of failing to do much good sometimes?

Yes, it does. And that’s ok. If it was the right play in EV, sometimes you win and sometimes you lose.

(*The Kelly criterion is a formula that determines the optimal theoretical size for a bet.)

Haghani concludes, “It seems like SBF was essentially telling anyone who was listening that he’d either wind up with all the money in the world, which he’d then redistribute according to his Effective Altruist principles – or, much more likely, he’d die trying.”

( Full disclosure: I have lost an irritating amount of money thanks to SBF’s shenanigans. My BlockFi crypto account is frozen due to fallout from the FTX collapse, with no word on if/when I might see my funds again. )

Ideological overconfidence is a trojan horse for entitlement

mdmakowsky1 Comment

I re-watched “The Big Lebowski”. I have watched this movie many times, but like all great films, you often find yourself appreciating new things with each re-watch. Usually these are simply small, but interesting details that simply slipped by your attention previously, but in this case it was a broad theme that smacked me in the face as if I stepped on a rake. The Dude is the Buddha.

Spoilers, both narrative and spiritual, are ahead

Let me back up. For those unfamiliar with the film, there are four parties relevant to this story. Jeffrey Lebowski (old rich guy), three nameless German nihilists, Walter Sobchak, and The Dude. Jeffrey Lebowski’s trophy wife, Bunny, runs off to Vegas unannounced. Hi-jinks ensue.

The Nihilists falsely claim to have kidnapped her and demand ransom. Jeffrey wants Bunny to go away so he can better hang on to the money she is frittering away, so he fills a briefcase with deadweight and gives it to the Dude to deliver in the hopes the kidnappers will be angered and make his problem go away. Walter becomes privy to this exchange and swaps out “a ringer for a ringer”, hoping to keep the ransom money for himself. A perfect triangle of scams.

All three scamming parties are driven by selfish entitlement and all three rationalize it behind blustery, paper thin philosophies. The Nihislists believe in a chaotic universe absent responsibility, morality, or ethics. They simply want a thing so they try to take it. Jeffrey Lebowski casually gives in to his own avarice and greed to push Bunny out, rationalized by the belief that he has earned his wealth through “achievement”. Walter Sobchak wants to take the money because he has been wronged by fate and society, an unloved Veteran whose reactionary ethos assures him that he has earned anything he subsequently takes through past valor. Over the course of the film all three rationalizations are found to be more or less fabricated, their philosophical fascades disintegrating as soon as the objects of their desire prove out of reach.

Only the Dude remains whole and unblemished. His innocence isn’t shielded by superior philosophy, morality, or intelligence. He remains above the destruction and greed simply because of his lack of desire. A burned out hippie of nearly zero consequence, his lines are the only reliable words spoken throughout the story simply because he doesn’t want anything more than to go bowling and, if possible, recover a lost rug. It is the emptiness of The Dude that preserves his soul.

Too much unbending confidence in your own philosophy often betrays a far starker reality. You don’t just want something, you want a reason that you deserve it.

Anyway. Go a little easier on each other. Make yourself a White Russian tonight. Take a moment to reflect on my remedial cinema film analysis and how sure you really are. About any of this.

Introducing Students to Text Mining II

Joy Buchanan6 Comments

In the Fall of 2020, I blogged about how I introduce students to text mining, as part of a data analytics class.

Could Turing ever have imagined that a human seeking customer service from a bank could chat with a bot? Maybe text mining is a big advance over chess, but it only took about one decade longer for a computer (developed by IBM) to beat a human in Jeopardy. Winning Jeopardy requires the computer to get meaning from a sentence of words. Computers have already moved way beyond playing a game show to natural language processing.

https://economistwritingeveryday.com/2020/11/07/introducing-students-to-text-mining/

I told the students that “chat bots” are getting better and NLP is advancing. By July 2020, OpenAI had released a beta API playground to external developers to play with GPT-3, but I did not sign up to use it myself.

In April of 2022, I added some slides inspired by Alex’s post about the Turing Test that included output from Google’s Pathway Languages Model. According to Alex, “It seems obvious that the computer is reasoning.”

This week in class, I did something that few people could have imagined 5 years ago. I signed into the free new GPTChat function in class and typed in questions from my students.

We started with questions that we assumed would be easy to answer:

Then we were surprised that it answered a question we had thought would be difficult:

And then we asked two questions that prompted the program to hedge, although for different reasons.

It seems like the model is smarter than it lets on. For now, the creators are trying hard not to offend anyone or get in the way of Google’s advertising business. Overall, the quality of the answers are high.

Because of when I was born, I believe that something I have published will make it into the training data for these models. Will that turn out to be more significant than any human readers we can attract?

Of course, GPT can still make mistakes. I’m horrified by this mischaracterization of my tweets:

The Imperfection of Subgame Perfection

Zachary Bartsch1 Comment

I’ve written previously about Pure Strategy Nash Equilibria (PSNE). They are the set of strategies that players can adopt in equilibrium – with no incentive to change their strategy. Students have an intuition that PSNE aren’t great because some outcomes that they identify depend on players making silly decisions in the past. In jargon, we can say that some PSNE depend on players choosing irrationally in a subgame while still reaching a PSNE.

See the extensive form game (below right). There are two players, each with two strategies per information set, and player two has two information sets. All PSNE will include a strategy for each information set. We can present the same game in normal form in order to make it easier to identify the PSNE (below left).

Player 1 (P1) can choose the row (B or C) and Player 2 (P2) can choose the column. Importantly, whether P1 might want to change his mind depends on P2’s strategy at the decision node in the alternative information set. Therefore, P2 must have two strategies, one per information set.

The four PSNE strategies and payoffs are underlined in the above table and they are noted in red on the below extensive form games. Again, the logic of PSNE states that no player can improve their payoff by changing only their own strategy, given the opposing player’s strategy. After all, a player can control their own strategy, but not that of their opponent. For example, note PSNE II. In the left subgame, P2 chooses M. His payoff would be unchanged if he changed his strategy, given the strategy of P1.

Continue reading →

Ban, Subsidize, Mandate: Ethics and US Healthcare Policy

James BaileyLeave a comment

Tomorrow (Friday 12/2) I’ll be speaking at the Fall Ethics Forum at Sacramento State. The Center for Practical and Professional Ethics there does a forum every year on a different field of practical ethics, and this year they chose healthcare (some previous iterations look quite interesting, like Bryan Caplan on education and Lyman Stone on population). The event is open to the public if you happen to live near Sacramento, and I hope to be able to post a recording later. But for now, here’s a short preview of what I plan to say:

In many key respects, US health policy is about restricting the choices available to patients and health care providers: banning things the government doesn’t want, while mandating or subsidizing things they want. These restrictions on autonomy are typically justified by the idea that they lead to superior health or economic outcomes. In some cases this tradeoff between freedom and efficient utilitarian outcomes is real, but I highlight some policies such as Certificate of Need laws that appear to harm both freedom and efficiency. I argue that the overarching US approach to health policy is to subsidize demand while restricting supply, which together lead to exceptionally high prices but mediocre health outcomes.

I’ll also take on some classic questions like: when are free lunches truly free? And when is moral hazard really immoral?

Fight for $15? $25? $40?

Jeremy Horpedahl2 Comments

Remember the “Fight for $15”? It’s a 10-year-old movement to raise the federal minimum wage to $15 per hour. While there hasn’t been any increase in the federal minimum wage since the movement began in 2012, plenty of states and localities have done so.

I won’t rehash the entire debate on the minimum wage here, but I will point you to this post from Joy on large minimum wage changes, and here are several other posts on this blog on the same topic. But lately I have seen an increasing call for even larger minimum wage increases, well beyond $15.

A prominent recent call for a higher wage comes from the SEIU, the second largest labor union in the nation. They are calling for a $25 minimum wage in Chicago, where the legal minimum wage just recently crossed $15 last year. Again, without getting into the detailed debates about the economics of the minimum wage, we can recognize that this would be a massively high minimum wage, given that median hourly wage for the Chicago MSA was $22.74 in May 2021. It’s certainly a bit higher in 2022, and the city of Chicago is probably a bit higher than the entire MSA. Still, we are talking about a minimum wage that would cover roughly half the workforce. Well, at least half the current workforce. The negative employment effects would potentially be large.

Here I will dabble a little bit in the minimum wage literature. One of the most famous recent papers that suggests increasing the minimum wage doesn’t have large negative employment effects is a 2019 paper by Cengiz, et al. This paper only looks at legal minimum wages that go up to 59% of the median market wage, which is the highest wages have been pushed up so far. By contrast, that $25 minimum wage in Chicago would be somewhere around 100% (!) of the local median market wage. That’s huge, and goes far beyond what even the most sympathetic-to-the-minimum-wage research has looked at.

But here’s the most recent minimum wage call that really takes the cake: over $40 per hour in Hawaii. That comes from, in a way, a Tweet from Hal Singer:

Now in fairness, he doesn’t exactly call for a $40 minimum wage in Hawaii, but he does say we should use the minimum wage as a tool to address homelessness, and then points to a study showing that you would need to earn $40/hour in Hawaii to afford a two-bedroom apartment. That’s pretty close. The median wage in Hawaii? About $23 in May 2021. In fact, the 75th percentile wage in Hawaii was $36.50 in 2021! So, depending on exactly how much wage growth there has been in Hawaii since May 2021, we are likely talking about a $40 minimum wage covering 75% of the workforce! That would likely have some “bite,” as economists say.

Protests Erupt Across China Over COVID Policy But Lockdowns Continue: Why?

Scott Buchanan1 Comment

Headlines in today’s financial news include items like “Clashes in Shanghai as COVID protests flare across China“ from Yahoo Finance. There have been widespread protests this week, which are normally a rarity under the authoritarian regime, and are being suppressed by any means necessary. Apple stock is down about 4% in the past two trading days on fears that iPhone shortages will get worse due to worker unrest in the giant Foxconn factory in Zhengzhou. Wall Street keeps hoping the China will loosen up, since the lockdowns on the world’s second-largest economy are a drag on global markets.

China has pursued a “zero-COVID” policy, of strict mass lockdowns to halt any spread of the virus. Residents have been confined to their apartments for over 3 months in some cases. Whole cities with tens of millions of people have been locked down for months at a time whenever a number of cases are spotted. China’s economic growth has stagnated, and unemployment among young people has risen to 20%, which has helped fuel unrest.  Chinese people are aware that the rest of the world has moved on from mass lockdowns, and may be realizing the futility of thinking that lockdowns can stave off the virus indefinitely.

Given its discomfort with widespread discontent and protests, why does the Chinese government persist in this policy? An article in the Atlantic by Michael Shuman answers that question: “Zero COVID Has Outlived Its Usefulness. Here’s Why China Is Still Enforcing It. “  Back in 2020 when COVID first swept through the world, the strict lockdowns (readily enforced in an authoritarian regime) seemed like a big win for the Chinese leadership:

When the outbreak began in Wuhan in early 2020, the virus was unknown, vaccines were unavailable, and China’s poorly equipped health system could have quickly become overwhelmed by a sweeping pandemic. Yet the policy had a political element from the very beginning as well. The Communist Party is adept at sniffing out threats to its rule, and it quickly identified COVID as one of them. A major public-health crisis, with millions dying, would have raised serious doubts about the regime’s competence, which is, in effect, its sole claim to legitimacy.

Worse, the party could have faced a populace that directly blamed it for the outbreak—with good reason. The Chinese authorities at both the national and local levels botched their initial response to the novel coronavirus, suppressing information about its discovery by a Wuhan doctor and acting far too slowly to contain the initial spread. Sensing its potential vulnerability, the party shifted into anti-COVID overdrive, shutting down large swaths of the country, with the result that it did succeed in snuffing out an epidemic in a matter of weeks, even as it spread to the rest of the world.

That success allowed the Communist Party to transform a potential tragedy into a public-relations triumph. Within weeks of the Wuhan outbreak, China’s propaganda machine was touting the wonders of its virus-busting methods. And as the U.S. and other Western countries struggled to contain the disease, Beijing’s big win became even more valuable as evidence that its authoritarian system was more capable and caring than any democratic one. Beijing and its advocates pointed to rising case and death counts in the U.S. as proof of China’s superiority and American decline.

A number of other countries including Australia and New Zealand also implemented strict (stricter than in the U.S.) lockdown measures in 2020, and, like China, experienced far less impact from the virus in that timeframe than seen in the U.S. However, most of these measures were lifted in 2021. The widespread application of mRNA vaccines like those from Pfizer and Moderna in the West has served to mitigate the severity of the viral infection. Also, some measure of herd immunity has been achieved by the widespread exposure to COVID in the population; antibodies persist for at least eight months after contracting the disease. So, what’s up with China?

China has resisted using Western vaccines, relying instead on homegrown vaccines which are less effective, though they do give some measure of protection.  Also, “The additional layers of high-tech surveillance adopted in the name of pandemic prevention can be used to enhance the tracking and monitoring of the populace more generally,” which is another win for the government. However, the major factor is that the Party, and especially President Xi, cannot afford to loosen up now and risk an embarrassing explosion of cases that would overburden the healthcare system and probably lead to millions of deaths:

The victory of zero COVID was claimed not just as the party’s but as Xi Jinping’s in particular. The State Council, China’s highest governing body, declared in a 2020 white paper that Xi had “taken personal command, planned the response, overseen the general situation and acted decisively, pointing the way forward in the fight against the epidemic.”

This narrative became entrenched. If Beijing loosened up and allowed COVID to run amok, the Chinese system would appear no better than those of loser democracies, and Xi would seem like another failing politician, a mere mortal, not the virus-fighting superhero he was painted as. Zero COVID’s failure would be a disaster for the Communist Party’s veneer of infallibility.

So the leadership insists on zero COVID and damn the consequences.

Maybe authoritarianism doesn’t scale

mdmakowsky1 Comment

Many of us, myself included, were fretting about role that surveillance technologies would play in creating more resilient authoritarian regimes, but we’re starting to see the cracks forming in multiple regimes all at once.

I don’t want to try to share in not-as-yet-achieved victories half way around the globe as innocent civilians are still being killed and imprisoned while bearing no personal risk myself. That said, I can’t help but observe that surveillance technology seems to offer relatively little for maintaining a regime. In fact, it may be the case that all it serves to do is accelerate government overeach and eventual public resistance.

Sureveillance doesn’t seem to actually grant any additional scale to authorities, the increasing the population a smaller group in power might control. It may, however, accelerate the rate at which the public achieves a critical mass of awareness of not just transgressions by authorities, but the true beliefs of their peers regarding those transgressions. Social media and other technologies seem to be accelerating the process through which preference falsification is being unraveled.

The simpler observation, however, is that modern information technology appears to have given some authoritarian regimes the misguided belief that simply being aware of resistance earlier, and reacting ruthlessly, would be sufficient to maintain greater control and reach. Instead, this unwarranted confidence has simply greased slide to overreach. When a bunch of stormtroopers show up to force people to work, the broader public arrives at a dark calculus all the faster: resisting is no riskier than not resisting in the short run, resisting is likely less risky in the long run.

There’s no shortage of fiction reminding us that authority is brittle. Even in a world of imagined sci-fi technology, absent a dystopian robot apocalypse, controlling humans using other humans remains an analog mechanism where the returns to scale remain gratefully constant. Sometimes a small number of people can achieve control over a larger number, but their days are always numbered. The technology that we feared might tilt the balance of this equation may have, in fact, merely sped up the timeline.

Thankful List in 2022

Joy BuchananLeave a comment
  1. I was able to get a free Covid-19 booster shot reformulated to fight new Omicron strains. This was easy to schedule at Walgreens, and I got a flu shot at the same time to save time. Vaccines for all types of diseases are advancing.  
  2. I didn’t lose money on crypto.
  3. When Russia invaded Ukraine in February of this year, I ordered Potassium Iodide tablets. I have not needed them.
  4. “The shrinking ozone hole shows that the world can actually solve an environmental crisis”
  5. NASA can save us from an incoming asteroid
  6. Bringing one over from Dynomight (HT: Tyler) “That there’s been a 93% decline in stomach cancer deaths over the past 100 years—from by far the biggest killer among cancers to one of the smaller ones—and mostly this was an accident, it happened because better food refrigeration reduced infections of H. pylori, a bacterium that wasn’t even identified until 1982 after most of the decline had already happened.”