What the Superintelligence can do for us

These days, when I blog-rant about my everyday life, I have increasingly ended on the thought “AGI fixes this.”

Yesterday, I mused whether AGI would be my personal chef? : Where Can You Still Buy a Great Dinner in the US?

Would AGI help me match my clothes that I no longer want to humans who can use them, to cut down on pollution?: Joy’s Fashion Globalization Article with Cato

Would AGI make no mistakes about weather-related school closure?: Intelligence for School Closing

Can AGI book summer camp for me?

As a millennial woman working through my 30’s, I increasingly see social media posts from my friends like this one:

One of the difficult things about infertility, for my friends going through it, is the uncertainty. Modern medicine seems legitimately short on information and predictive analytics for this issue. So… AGI to the rescue, someday?

All I’m writing about tonight is that I have created a growing to-do list, over roughly the past year, for the AGI. Would something smart enough to do all of the above be dangerous? I wouldn’t rule it out. As pure speculation, it feels safer to have an AI that is specifically devoted to being a personal chef but which strictly cannot do anything else beside manage food. An AI that could actually do all of those things… would be quite powerful.

Here’s me musing about the AGI rising up against us, written after watching the TV show Severance: Artificial Intelligence in the Basement of Lumon Industries

Videos for Teaching Inflation in 2024

I’m teaching principles of macro this semester. Making macroeconomics sound important to students is partly about explaining that recessions are painful and significant.

As Alex Tabarrok says, “The Great Depression is Over!”  Maybe Gen Z can appreciate the significance of the Great Depression, but it is history. Gen Z has heard of the Great Recession, but keep in mind that a student who is 20-y-o in 2024 was 4 in 2008. It’s a weird one, but there has been a recession more recently. The Covid Recession is what I like to link to, when possible, in class.

To teach the inflation chapter this week, I’m using video clips that I’ll put up here as resources for others.

To start off the inflation chapter and bring in a more global perspective, I show: “Zimbabwe’s inflation rate hits triple digits”  This 2-minute news clip was produced by Al Jazeera. They talk about lending and policy in addition to retail price increases.

After we have gone through some definitions, I show two clips of an economic forecast that was recorded in 2021. I don’t usually show such long clips in class, but I’m relying on dramatic irony to make it interesting. The students know the path that inflation took from 2020 to 2024, but Dr. Doti in the video does not. I stop the video occasionally to point out connections to our textbook.

Chapman University’s 2021 Economic Forecast Update was presented virtually on Wednesday, June 16, 2021.

Dr. Jim Doti predicts that an unprecedented increase in the money supply after Covid will lead to inflation. He’s not right about everything, but that’s what makes it so interesting. Right after showing students the quantity theory of money equation, I can show them someone trying to apply it from about minute 25 to about minute 35. (don’t start the video from minute 1)

Then, I go back to my lecture and introduce the Fisher effect. Next, we watch about minute 38 to minute 43 of the 2021 forecast because of the direct connection of inflation to interest rates. Partly this just helps illustrate how messy the real world is.

Also, I pull from one of Jeremy’s 2023 posts to illustrate the long run neutrality of money. “The Rate of Inflation is Falling, But Prices are Still Rising (And So are Wages)

Covid Death Structural Breaks

xtbreak (STATA)

I found a new time series and panel data tool that I want to share. What does it do? It’s called xtbreak and it finds what are known as ‘structural breaks’ in the data. What does that mean? It means that the determinants of a dependent variable matter differently at different periods of time. In statistics we’d say that the regression coefficients are different during different periods of time. To elaborate, I’ll walk through the same example that the authors of the command use.

You can download the time series data from here: https://github.com/JanDitzen/xtbreak/blob/main/data/US.dta

The data contains weekly US covid cases and deaths for 2020-2021. Here’s what it looks like:

So, what’s the data generating process? It stands to reason that the number of deaths is related to the number of cases one week prior. So, we can adopt the following model:

That seems reasonable. However, we suspect that δ is not the same across the entire sample period. Why not? Medical professionals learned how to better treat covid, and the public changed their behavior so that different types of people contracted covid. Further, once they contracted it, the public’s criteria for visiting the doctor changed. So, while the lagged number of cases is a reasonable determinant of deaths across the entire sample, we would expect it to predict a different number deaths at different times. In the model above, we are saying that δ changes over time and maybe at discrete points.

First, xtbreak allows us to test whether there are any structural breaks. Specifically, it can test whether there are S breaks rather than S-1 breaks. If the test statistic is greater than the critical statistics, then we can conclude that there are some number of breaks. Note that there being 5 breaks given that there are 4 depends on there also be at least 4 breaks. And since we can’t say that there are certainly 4 breaks rather than 3, it would be inappropriate to say that there are 4 or 5 breaks.

Great, so if there are three structural breaks, then when do they occur? xbtreak can answer that too (below). The three structural breaks are noted as the 20th  week of 2020, the 51st week of 2020, and the 11th week of 2021. Conveniently, there is also a confidence interval. Note that the confidence intervals for 2020w11 and 2021w11 breaks are nice and precise with a 1-week confidence interval. The 2nd break, however, has a big 30-week confidence interval (nearly 7 months). So, while we suspect that there is a 3rd  structural break, we don’t know as precisely where it is.

Regardless, if there are three structural breaks, then that means that there are four time periods with different relationships between lagged covid cases and covid deaths. We can create a scatter plot of the raw data and run a regression to see the different slopes. Below we can see the different slopes that describe the impact of lagged covid cases on deaths. Sensibly, covid cases resulted in more deaths earlier during the pandemic. As time passed, the proportion of cases which resulted in death declined (as seen in the falling slope of the dots). It’s no wonder that people were freaking out at the start of the pandemic.

What’s nice about this method for finding breaks is that it is statistically determined. Of course, it’s important to have a theoretical motivation for why any breaks would occur in the first place. This method is more rigorous than eye-balling the data and provides opportunities to hypothesis test the number of breaks and their location. If you read the documentation, then there are other tests, such as breaks in the constant, that are also possible.


See this ppt by the authors for more: https://www.stata.com/meeting/germany21/slides/Germany21_Ditzen.pdf

See this Stata Journal article for more still: https://repec.cal.bham.ac.uk/pdf/21-14.pdf

Does GPT-4 Know How High the Alps Are?

I’m getting ready to give some public local talks about AI. Last week I shared some pictures that I think might help people understand ChatGPT, specifically:

My first thought is that GPT-4 was giving incorrect estimates of the heights of these mountains because it does not actually “know” the correct elevations. But then a nagging question came to mind.

GPT has a “creativity parameter.” Sometimes, it intentionally does not select the top-rated next word in a sentence, for example, in order to avoid being stiff and boring. Could GPT-4 know the exact elevation of these mountains, and it is just intentionally being “creative,” in this case?

I do not want to stand up in front of the local Rotary Club and say something wrong. So, I went to a true expert, Lenny Bogdonoff, to ask for help. Here is his reply:

Not quite. It’s not that it knows or doesn’t know, but based on the prompt, it’s likely unable to parse the specific details and is outputting results respectively. There is a component of stochastic behavior based on what part of the model weights are activated.

One common practice to help avoid this and see what the model does grasp, is to ask it to think step by step, and explain its reasoning. When doing this, you can see the fault in logic.

All that being said, the vision model is actually faulty in being able to grasp the relative position of information, so this kind of task will be more likely to hallucinate.

There are better vision models, that aren’t OpenAI based. For example Qwen-VL-Max is very good, from the Chinese company Alibaba. Another is LLaVA which uses different baselines of open source language models to add vision capabilities

Depending on what you are needing vision for, models can be spiky in capability. Good at OCR but bad at relative positioning. Good at classifying a specific UI element, but bad at detecting plants, etc etc. 

Joy: So, I think I can tell the Rotary Club that GPT was “wrong” as opposed to “intentionally creative.” I think, as I originally concluded, you should not make ChatGPT the pilot of your airplane and go to sleep when approaching the Alps. ChatGPT should be used for what it is good at, such as writing the rough draft of a cover letter. (We have great “autopilot” software for flying planes, already, without involving large language models.)

Another expert, Gavin Leech, also weighed in with some helpful background information:

  • the creativity parameter is known as temperature. But you can actually radically change the output (intelligence, style, creativity) by using more complicated sampling schemes. The best analogy for changing the sampling scheme is that you’re giving it a psychiatric drug. Changing the prompt, conversely, is like CBT or one of those cute mindset interventions.
  • For each real-name model (e.g. “gpt-4-0613”), there’s 3 versions: the base model (which now no one except highly vetted researchers have access to), the instruction-tuned model, and the RLHF (or rather RLAIF) model. The base model is wildly creative, unhinged, but the RLHF one (which the linked researchers use) is heavily electroshocked into not intentionally making things up (as Lenny says).
  • It’s currently not usually possible to diagnose an error – the proverbial black box. My friends are working on this though
  • For more, note OpenAI admitting the “laziness” of their own models. the Turbo model line is intended to fix this.

Thank you, Lenny and Gavin, for donating your insights.

How ChatGPT works from geography and Stephen Wolfram

By now, everyone should consider using ChatGPT and be familiar with how it works. I’m going to highlight resources for that.

My paper about how ChatGPT generates academic citations should be useful to academics as a way to quickly grasp the strengths and weakness of ChatGPT. ChatGPT often works well, but sometimes fails. It’s important to anticipate how it fails. Our paper is so short and simple that your undergraduates could read it before using ChatGPT for their writing assignments.

A paper that does this in a different domain is “GPT4GEO: How a Language Model Sees the World’s Geography” (Again, consider showing it to your undergrads because of the neat pictures, but probably walk through it together in class instead of assigning it as reading.) They describe their project: “To characterise what GPT-4 knows about the world, we devise a set of progressively more challenging experiments… “

For example, they asked ChatGPT about the populations of countries and found that: “For populations, GPT-4 performs relatively well with a mean relative error (MRE) of 3.61%. However, significantly higher errors [occur] … for less populated countries.”

ChatGPT will often say SOMETHING, if prompted correctly. It is often, at least slightly, wrong. This graph shows that most estimates of national populations were not correct and the performance was worse on countries that are less well-known. That’s exactly what we found in our paper on citations. We found that very famous books are often cited correctly, because ChatGPT is mimicking other documents that correctly cite those books. However, if there are not many documents to train on, then ChatGPT will make things up.

I love this figure from the geography paper showing how ChatGPT estimates the elevations of mountains. This visual should be all over Twitter.

There are 3 lines because they did the prompt three times. ChatGPT threw out three different wrong mountains. Is that kind of work good enough for your tasks? Often it is. The shaded area in the graph is the actual topography of the earth in those places. ChatGPT “knows” that this area of the world is a mountain. But it will just put out incorrect estimates of the exact elevation, instead of stating that it does not know the exact elevation of those areas of the world.

Another free (long, advanced) resource with great pictures is Stephen Wolfram’s 2023 blog article “What Is ChatGPT Doing … and Why Does It Work?” (YouTube version)

The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.

If you feel like you already are proficient with using ChatGPT, then I would recommend Wolfram’s blog because you will learn a lot about math and computers.

Scott wrote “Generative AI Nano-Tutorial” here, which has the advantage of being much shorter than Wolfram’s blog.

EDIT: New 2023 overview paper (link from Lenny): “A Survey of Large Language Models

Hazards of the Internet of Things 2. Big Brother Is Watching Your Every Breath

There seems to be something of a generational divide as to how important is your personal privacy. Folks under, say, age 40, have lived such a large fraction of their lives with Facebook and Amazon and Google and Twitter logging and analyzing and reselling information on what they view and listen to and say and buy, that they seem rather numb to the issue of internet privacy. Install an Alexa that ships out every sound in your home and a smart doorbell that transmits every coming and going to some corporate server, fine, what could possibly be the objection?  So what if your automobile, in addition to tracking and reporting your location, feeds all your  personal phone text messages to the vehicle manufacturer?

For us older folks whose brain pathways were largely shaped in a time when communication meant talking in person or on a (presumably untapped) phone, this seems just creepy. Polls show that a majority of Americans are uneasy about the amount of data on them being collected, but “do not think it is possible to go about daily life without corporate and government entities collecting data about them.”

There are substantive concerns that can be raised about the uses to which all this information may be put, and about its security. Per VPNOverview:

Over 1,800 data leaks took place last year in the US alone, according to Statista. These breaches compromised the records of over 420 million people.” . With smartwatches having access to so much sensitive information, here’s what kind of data can fall into the wrong hands in case of a data leak:

  • Your personal information, including name, address, and sometimes even Social Security Number
  • Sensitive health information collected by the smartwatch
  • Login credentials to all the online platforms connected to your smartwatch
  • Credit card and other payment information
  • Digital identifiers like your IP address, device ID, or browser fingerprint
  • Remote access information to smart home devices

Several times a year now, I get notices from a doctor’s office or finance company or on-line business noting blandly that their computer systems have been hacked and bad guys now have my name, address, birthdate, social security number, medical records, etc., etc. (They generously offer me a year of free ID fraud monitoring. )

The Internet of Things (IoT) promises to ramp up the snooping to a whole new level. I took note four years ago when Google acquired Fitbit. At one gulp, the internet giant gained access to a whole world of activity and health data on, well, you. The use of medical and other sensors, routed through the internet, keeps growing. One family member uses a CPAP machine for breathing (avoid sleep apnea) at night; the company wanted the machine to be connected on the internet for them to monitor and presumably profit from tracking your sleep habits and your very breath. And of course when you don a smart watch, your every movement, as well as your heartbeat, are being sent off into the ether. (I wonder if the next sensor to be put into a smart watch will be galvanic skin response, so Big Tech can log when you are lying).

According to a senior systems architect: “The IoT is inevitable, like getting to the Pacific Ocean was inevitable. It’s manifest destiny. Ninety eight percent of the things in the world are not connected. So we’re gonna connect them. It could be a moisture sensor that sits in the ground. It could be your liver. That’s your IoT. The next step is what we do with the data. We’ll visualize it, make sense of it, and monetize it. That’s our IoT.”

When my kids were little, we let them use cassette tape players to play Winnie the Pooh stories. With my grandkids, the comparable device is a Yoto player. This also plays stories (which is good, better than screens), but it only operates in connection with the internet. The default is that the Yoto makers collect and sell personal information on usage by you and your child (which would include time of day as well as choice of stories). You can opt out, if you are willing to take the trouble to write to their legal team (thanks, guys).

There are cities in the world, in China but also some European cities, where there are monitoring cameras (IoT) everywhere. Individuals can be recognized by facial features and even by the way they walk; governmental authorities compile and track this information. These surveillance systems are being sold to the public with the promise of increased “security.” Whether it really makes we the people more secure is heavily dependent on the benevolence and impartiality of the state powers. Supposing a department of the federal government with access to surveillance data became politicized and then harassed members of the opposing party?

I’ll conclude with several slides from  Timothy Wallace’s 2023 presentation on the Internet of things:

The dystopian  novel 1984 by George Orwell was published in 1949.  It describes a repressive totalitarian state, headed by Big Brother, which was characterized by pervasive surveillance. Ubiquitous posters reminded citizens, “Big Brother is watching you.” Presumably the various cameras and microphones used in the mass surveillance there were paid for and installed by the eavesdropping authorities. It is perhaps ironic that so many Americans now purchase and install devices that allow some corporate or governmental entity to snoop them more intimately than Orwell could have imagined.

Intelligence for School Closing

I don’t have much time to write this week because I lost so many work hours to schools closing for “weather.”

Tyler has been saying that we should welcome more intelligence (in the form of LLMs – I’m not getting any smarter). What would we want intelligence for? How about reducing the error rate on school closing?

First, I will recognize that things are already getting better due to computers. The internet and texting and radar help. Compared to when I was a child in New Jersey, it’s more efficient to text all the parents the night before, as opposed to having people get up at 6am to scan the radio for news. Weather forecasting has presumably gotten better.

Now my rant: Right around what was already a three-day official weekend, school was closed three times. Even my kids were irate when that last day was announced. In my opinion, only one of those closures was justified for extreme weather.

There is a lot of dumb in a city. People complain about routine processes being suboptimal. It would be great if we humans could figure out ways to apply more intelligence to these local problems and make less mistakes.

This is a joke for any readers in cold climates. My Alabama kids thought it was fun to collect icicles because they have almost never seen them before.

Hazards of the Internet of Things 1. Hacking of Devices (Baby Monitors, Freezers, Hospital Ventilators) in Homes and Institutions

For my birthday this year, someone gave me a “smart” plug-in power socket. You plug it into the wall, and then can plug in something, say a lamp, into the smart socket, which you can then control via the internet. Yay, I am now a part of the Internet of Things (IoT). What could possibly go wrong?

However, my Spidey-sense started to tingle, and I chose to give this device away.  At that point, I was thinking mainly of the potential for such devices to get hacked and then recruited to be part of a vast bot-net which can then (under the control of bad actors) conduct massive attacks on crucial internet components. For instance,

Mirai [way back in 2016] infected IoT devices from routers to video cameras and video recorders by successfully attempting to log in using a table of 61 common hard-coded default usernames and passwords.

The malware created a vast botnet. It “enslaved” a string of 400,000 connected devices. In September 2016, Mirai-infected devices (who became “zombies”) were used to launch the world’s first 1Tbps Distributed Denial-of-Service (DDoS) attack on servers at the heart of internet services.  It took down parts of Amazon Web Services and its clients, including GitHub, Netflix, Twitter, and Airbnb.

But it turns out the hazards with smart devices are widespread indeed. IoT devices are so useful for bad guys that that they are attacked more than either mobile devices or computers. One layer of hazard is the hacking of specific, poorly-secured devices in a home or institution, with subsequent control of devices and infiltration of broader computing systems. This will be the focus of today’s blog post. Another layer of hazard is the use to which masses of (sometimes private and personal) data snooped from “unhacked” smart devices are put by large corporations and state actors; that will be considered in a part 2 post.

Here are results from one study from nearly three years ago:

https://www.thalesgroup.com/en/markets/digital-identity-and-security/iot/magazine/internet-threats

A study published in July 2020 analyzed over 5 million IoT, IoMT (Internet of Medical Things), and unmanaged connected devices in healthcare, retail, manufacturing, and life sciences. It reveals an astonishing number of vulnerabilities and risks across a stunningly diverse set of connected objects….

The report brings to light disturbing facts and trends:

  • Up to 15% of devices were unknown or unauthorized.
  • 5 to 19% were using unsupported legacy operating systems.
  • 49% of IT teams were guessing or had tinkered with their existing IT solutions to get visibility.
  • 51% of them were unaware of what types of smart objects were active in their network.
  • 75% of deployments had VLAN violations
  • 86% of healthcare deployments included more than ten FDA-recalled devices.
  • 95% of healthcare networks integrated Amazon Alexa and Echo devices alongside hospital surveillance equipment.

…Ransomware gangs specifically target healthcare more than any other domain in the United States. It’s now, by far, the #1 healthcare breach root cause in the country. …The mix of old legacy systems and connected devices like patient monitors, ventilators, infusion pumps, lights, and thermostats with very poor security features are sometimes especially prone to attacks.

So, these criminals understand that stopping critical applications and holding patient data can put lives at risk and that these organizations are more likely to pay a ransom.

I know people in organizations which have been brought to their knees by ransomware attacks. And I have read of the dilemma of the guy who was on vacation in the Caribbean or whatever, and got a text from a hacker instructing him to deposit several hundred dollars in a Bitcoin account, or else his “smart” refrigerator/freezer would be turned off and he would come home to a spoiled, moldy mess.

What brought all this IoT stuff to my attention this week was a talk I ran across from retired MIT researcher Timothy Wallace, titled “Effects, Side Effects and Risks of the Internet of Things”, presented at the 2023 American Scientific Affiliation meeting. The slides for his talk are here. I will paste in a few snipped excerpts from his talk, that are fairly self-explanatory:

(My comment: 10 billion is a really, really big number…)

(My comment: this type of catastrophic compromise of computer systems being enabled by hacking some piddling little IoT device that happens to be in the home or institution local network is not uncommon. Which is why I am reluctant to put IoT devices, especially from no-name foreign manufacturers, on my home wireless network).

Many of these vulnerabilities could in theory be addressed by better practices like always resetting factory passwords on your smart devices, but it is easy for forget to do that.

And just to end on a light note (this cartoon also lifted from Wallace’s slides):

Cal Newport on Smartphones for Kids

EP. 246: KIDS AND PHONES

Are smartphones bad for kids? Cal walks through the data on this question, including how researchers came to be worried, their findings, critiques of their findings, and where we are today. He then gives recommendations for how to think about technology when it comes to your kids.

In May of 2023, Cal Newport shared well-informed opinions about whether smartphones harm young people. In the first half of the podcast, he talks about depression and loneliness data.

Minute 30 of the podcast: Screentime harms teenagers because they inhibit the development of critical thinking skills. Deep critical thinking skills require training. Reading an analog book is better than screens (see my review of Tyler’s AI generative book and poastmodernism).

See my summary of Emily Oster on video games for kids. She does not clutch her pearls over violent video games. However, she is concerned about what activities get crowded out by screentime. She is especially worried about sleep, because on that topic the data are clear.

Minute 31, Call Newport: Tweens and teens scroll on their phones for too long instead of going to sleep. A 13-year-old boy with a smart phone will “be up until 4 in the morning.” A tween told him that middle school girls arrive at school too exhausted to function because they have been on their phones all night.

FYI, if you are the parent in an Apple device network, you can set time limits on the devices in your family. I filed this report about smart watches last year, incidentally in the same week as the release of Newport’s podcast episode.