Technology

How ChatGPT works from geography and Stephen Wolfram

Joy Buchanan1 Comment

By now, everyone should consider using ChatGPT and be familiar with how it works. I’m going to highlight resources for that.

My paper about how ChatGPT generates academic citations should be useful to academics as a way to quickly grasp the strengths and weakness of ChatGPT. ChatGPT often works well, but sometimes fails. It’s important to anticipate how it fails. Our paper is so short and simple that your undergraduates could read it before using ChatGPT for their writing assignments.

A paper that does this in a different domain is “GPT4GEO: How a Language Model Sees the World’s Geography” (Again, consider showing it to your undergrads because of the neat pictures, but probably walk through it together in class instead of assigning it as reading.) They describe their project: “To characterise what GPT-4 knows about the world, we devise a set of progressively more challenging experiments… “

For example, they asked ChatGPT about the populations of countries and found that: “For populations, GPT-4 performs relatively well with a mean relative error (MRE) of 3.61%. However, significantly higher errors [occur] … for less populated countries.”

ChatGPT will often say SOMETHING, if prompted correctly. It is often, at least slightly, wrong. This graph shows that most estimates of national populations were not correct and the performance was worse on countries that are less well-known. That’s exactly what we found in our paper on citations. We found that very famous books are often cited correctly, because ChatGPT is mimicking other documents that correctly cite those books. However, if there are not many documents to train on, then ChatGPT will make things up.

I love this figure from the geography paper showing how ChatGPT estimates the elevations of mountains. This visual should be all over Twitter.

There are 3 lines because they did the prompt three times. ChatGPT threw out three different wrong mountains. Is that kind of work good enough for your tasks? Often it is. The shaded area in the graph is the actual topography of the earth in those places. ChatGPT “knows” that this area of the world is a mountain. But it will just put out incorrect estimates of the exact elevation, instead of stating that it does not know the exact elevation of those areas of the world.

Another free (long, advanced) resource with great pictures is Stephen Wolfram’s 2023 blog article “What Is ChatGPT Doing … and Why Does It Work?” (YouTube version)

The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.

If you feel like you already are proficient with using ChatGPT, then I would recommend Wolfram’s blog because you will learn a lot about math and computers.

Scott wrote “Generative AI Nano-Tutorial” here, which has the advantage of being much shorter than Wolfram’s blog.

EDIT: New 2023 overview paper (link from Lenny): “A Survey of Large Language Models“

Hazards of the Internet of Things 2. Big Brother Is Watching Your Every Breath

Scott Buchanan1 Comment

There seems to be something of a generational divide as to how important is your personal privacy. Folks under, say, age 40, have lived such a large fraction of their lives with Facebook and Amazon and Google and Twitter logging and analyzing and reselling information on what they view and listen to and say and buy, that they seem rather numb to the issue of internet privacy. Install an Alexa that ships out every sound in your home and a smart doorbell that transmits every coming and going to some corporate server, fine, what could possibly be the objection?  So what if your automobile, in addition to tracking and reporting your location, feeds all your  personal phone text messages to the vehicle manufacturer?

For us older folks whose brain pathways were largely shaped in a time when communication meant talking in person or on a (presumably untapped) phone, this seems just creepy. Polls show that a majority of Americans are uneasy about the amount of data on them being collected, but “do not think it is possible to go about daily life without corporate and government entities collecting data about them.”

There are substantive concerns that can be raised about the uses to which all this information may be put, and about its security. Per VPNOverview:

Over 1,800 data leaks took place last year in the US alone, according to Statista. These breaches compromised the records of over 420 million people.” . With smartwatches having access to so much sensitive information, here’s what kind of data can fall into the wrong hands in case of a data leak:

  • Your personal information, including name, address, and sometimes even Social Security Number
  • Sensitive health information collected by the smartwatch
  • Login credentials to all the online platforms connected to your smartwatch
  • Credit card and other payment information
  • Digital identifiers like your IP address, device ID, or browser fingerprint
  • Remote access information to smart home devices

Several times a year now, I get notices from a doctor’s office or finance company or on-line business noting blandly that their computer systems have been hacked and bad guys now have my name, address, birthdate, social security number, medical records, etc., etc. (They generously offer me a year of free ID fraud monitoring. )

The Internet of Things (IoT) promises to ramp up the snooping to a whole new level. I took note four years ago when Google acquired Fitbit. At one gulp, the internet giant gained access to a whole world of activity and health data on, well, you. The use of medical and other sensors, routed through the internet, keeps growing. One family member uses a CPAP machine for breathing (avoid sleep apnea) at night; the company wanted the machine to be connected on the internet for them to monitor and presumably profit from tracking your sleep habits and your very breath. And of course when you don a smart watch, your every movement, as well as your heartbeat, are being sent off into the ether. (I wonder if the next sensor to be put into a smart watch will be galvanic skin response, so Big Tech can log when you are lying).

According to a senior systems architect: “The IoT is inevitable, like getting to the Pacific Ocean was inevitable. It’s manifest destiny. Ninety eight percent of the things in the world are not connected. So we’re gonna connect them. It could be a moisture sensor that sits in the ground. It could be your liver. That’s your IoT. The next step is what we do with the data. We’ll visualize it, make sense of it, and monetize it. That’s our IoT.”

When my kids were little, we let them use cassette tape players to play Winnie the Pooh stories. With my grandkids, the comparable device is a Yoto player. This also plays stories (which is good, better than screens), but it only operates in connection with the internet. The default is that the Yoto makers collect and sell personal information on usage by you and your child (which would include time of day as well as choice of stories). You can opt out, if you are willing to take the trouble to write to their legal team (thanks, guys).

There are cities in the world, in China but also some European cities, where there are monitoring cameras (IoT) everywhere. Individuals can be recognized by facial features and even by the way they walk; governmental authorities compile and track this information. These surveillance systems are being sold to the public with the promise of increased “security.” Whether it really makes we the people more secure is heavily dependent on the benevolence and impartiality of the state powers. Supposing a department of the federal government with access to surveillance data became politicized and then harassed members of the opposing party?

I’ll conclude with several slides from  Timothy Wallace’s 2023 presentation on the Internet of things:

The dystopian  novel 1984 by George Orwell was published in 1949.  It describes a repressive totalitarian state, headed by Big Brother, which was characterized by pervasive surveillance. Ubiquitous posters reminded citizens, “Big Brother is watching you.” Presumably the various cameras and microphones used in the mass surveillance there were paid for and installed by the eavesdropping authorities. It is perhaps ironic that so many Americans now purchase and install devices that allow some corporate or governmental entity to snoop them more intimately than Orwell could have imagined.

Intelligence for School Closing

Joy Buchanan1 Comment

I don’t have much time to write this week because I lost so many work hours to schools closing for “weather.”

Tyler has been saying that we should welcome more intelligence (in the form of LLMs – I’m not getting any smarter). What would we want intelligence for? How about reducing the error rate on school closing?

First, I will recognize that things are already getting better due to computers. The internet and texting and radar help. Compared to when I was a child in New Jersey, it’s more efficient to text all the parents the night before, as opposed to having people get up at 6am to scan the radio for news. Weather forecasting has presumably gotten better.

Now my rant: Right around what was already a three-day official weekend, school was closed three times. Even my kids were irate when that last day was announced. In my opinion, only one of those closures was justified for extreme weather.

There is a lot of dumb in a city. People complain about routine processes being suboptimal. It would be great if we humans could figure out ways to apply more intelligence to these local problems and make less mistakes.

This is a joke for any readers in cold climates. My Alabama kids thought it was fun to collect icicles because they have almost never seen them before.

Hazards of the Internet of Things 1. Hacking of Devices (Baby Monitors, Freezers, Hospital Ventilators) in Homes and Institutions

Scott BuchananLeave a comment

For my birthday this year, someone gave me a “smart” plug-in power socket. You plug it into the wall, and then can plug in something, say a lamp, into the smart socket, which you can then control via the internet. Yay, I am now a part of the Internet of Things (IoT). What could possibly go wrong?

However, my Spidey-sense started to tingle, and I chose to give this device away.  At that point, I was thinking mainly of the potential for such devices to get hacked and then recruited to be part of a vast bot-net which can then (under the control of bad actors) conduct massive attacks on crucial internet components. For instance,

Mirai [way back in 2016] infected IoT devices from routers to video cameras and video recorders by successfully attempting to log in using a table of 61 common hard-coded default usernames and passwords.

The malware created a vast botnet. It “enslaved” a string of 400,000 connected devices. In September 2016, Mirai-infected devices (who became “zombies”) were used to launch the world’s first 1Tbps Distributed Denial-of-Service (DDoS) attack on servers at the heart of internet services.  It took down parts of Amazon Web Services and its clients, including GitHub, Netflix, Twitter, and Airbnb.

But it turns out the hazards with smart devices are widespread indeed. IoT devices are so useful for bad guys that that they are attacked more than either mobile devices or computers. One layer of hazard is the hacking of specific, poorly-secured devices in a home or institution, with subsequent control of devices and infiltration of broader computing systems. This will be the focus of today’s blog post. Another layer of hazard is the use to which masses of (sometimes private and personal) data snooped from “unhacked” smart devices are put by large corporations and state actors; that will be considered in a part 2 post.

Here are results from one study from nearly three years ago:

https://www.thalesgroup.com/en/markets/digital-identity-and-security/iot/magazine/internet-threats

A study published in July 2020 analyzed over 5 million IoT, IoMT (Internet of Medical Things), and unmanaged connected devices in healthcare, retail, manufacturing, and life sciences. It reveals an astonishing number of vulnerabilities and risks across a stunningly diverse set of connected objects….

The report brings to light disturbing facts and trends:

  • Up to 15% of devices were unknown or unauthorized.
  • 5 to 19% were using unsupported legacy operating systems.
  • 49% of IT teams were guessing or had tinkered with their existing IT solutions to get visibility.
  • 51% of them were unaware of what types of smart objects were active in their network.
  • 75% of deployments had VLAN violations
  • 86% of healthcare deployments included more than ten FDA-recalled devices.
  • 95% of healthcare networks integrated Amazon Alexa and Echo devices alongside hospital surveillance equipment.

…Ransomware gangs specifically target healthcare more than any other domain in the United States. It’s now, by far, the #1 healthcare breach root cause in the country. …The mix of old legacy systems and connected devices like patient monitors, ventilators, infusion pumps, lights, and thermostats with very poor security features are sometimes especially prone to attacks.

So, these criminals understand that stopping critical applications and holding patient data can put lives at risk and that these organizations are more likely to pay a ransom.

I know people in organizations which have been brought to their knees by ransomware attacks. And I have read of the dilemma of the guy who was on vacation in the Caribbean or whatever, and got a text from a hacker instructing him to deposit several hundred dollars in a Bitcoin account, or else his “smart” refrigerator/freezer would be turned off and he would come home to a spoiled, moldy mess.

What brought all this IoT stuff to my attention this week was a talk I ran across from retired MIT researcher Timothy Wallace, titled “Effects, Side Effects and Risks of the Internet of Things”, presented at the 2023 American Scientific Affiliation meeting. The slides for his talk are here. I will paste in a few snipped excerpts from his talk, that are fairly self-explanatory:

(My comment: 10 billion is a really, really big number…)

(My comment: this type of catastrophic compromise of computer systems being enabled by hacking some piddling little IoT device that happens to be in the home or institution local network is not uncommon. Which is why I am reluctant to put IoT devices, especially from no-name foreign manufacturers, on my home wireless network).

Many of these vulnerabilities could in theory be addressed by better practices like always resetting factory passwords on your smart devices, but it is easy for forget to do that.

And just to end on a light note (this cartoon also lifted from Wallace’s slides):

Cal Newport on Smartphones for Kids

Joy BuchananLeave a comment

EP. 246: KIDS AND PHONES

Are smartphones bad for kids? Cal walks through the data on this question, including how researchers came to be worried, their findings, critiques of their findings, and where we are today. He then gives recommendations for how to think about technology when it comes to your kids.

In May of 2023, Cal Newport shared well-informed opinions about whether smartphones harm young people. In the first half of the podcast, he talks about depression and loneliness data.

Minute 30 of the podcast: Screentime harms teenagers because they inhibit the development of critical thinking skills. Deep critical thinking skills require training. Reading an analog book is better than screens (see my review of Tyler’s AI generative book and poastmodernism).

See my summary of Emily Oster on video games for kids. She does not clutch her pearls over violent video games. However, she is concerned about what activities get crowded out by screentime. She is especially worried about sleep, because on that topic the data are clear.

Minute 31, Call Newport: Tweens and teens scroll on their phones for too long instead of going to sleep. A 13-year-old boy with a smart phone will “be up until 4 in the morning.” A tween told him that middle school girls arrive at school too exhausted to function because they have been on their phones all night.

FYI, if you are the parent in an Apple device network, you can set time limits on the devices in your family. I filed this report about smart watches last year, incidentally in the same week as the release of Newport’s podcast episode.

Supply & Demand, With gifs

Zachary Bartsch8 Comments

I’ve discussed the ways to teach supply and demand in the past. Regardless, almost all principles of economics classes require a book. But even digital books are often just intangible versions of the hard copy. Supply and demand are illustrated as static pictures, using arrows and labels to do the leg-work of introducing exogenous changes. There’s often a text block with further explanation, but it lacks the kind of multi-sensory explanation that one gets while in a class.

In a class, the instructor can gesticulate and vary their speech explain the model, all while drawing a graph. That’s fundamentally different from reading a book. Studying a book requires the student to repeatedly glance between the words and the graph and to identify the appropriate part of the graph that is relevant to the explanation. For new or confused students, connected the words to one of many parts of a graph is the point of failure.

This is part of why the Marginal Revolution University videos do well. They’re well produced, with context and audio-overlaid video of graphs. It’s pretty close to the in-person experience sans the ability to ask questions, but includes the additional ability to rewind, repeat, adjust the speed, display captions, and share.

Continue reading →

Industries Without Investors

James BaileyLeave a comment

Venture-capital backed startups almost all cluster in the same handful of industries, mostly various types of software. This leaves a variety of large and economically important sectors with almost no venture-capital backed startups. That means those industries see fewer new companies and new ideas; they must rely on either growth from existing firms, which are unlikely to embrace disruptive innovation, or on startups that bootstrap and/or finance with debt, which tend to grow slowly.

Venture capital firm Fifty Years has done a nice job cataloging exactly which industries see the most, and least, investment relative to their size. Here is their picture of the US economy by industry market size:

Now their picture of which industries get the investment (though unfortunately, they aren’t very clear about their data source for it):

They use this to create an “Opportunity Ratio”- current market size divided by current startup funding:

They call the industries with the largest Opportunity Ratios the “Top Underfunded Opportunities”:

I don’t necessarily agree; some industries face shrinking demand, prohibitive regulation, or other fundamental issues making them bad candidates for investment. Conversely, investors haven’t just focused on software randomly or through imitation; they see that it is where the growth is.

Still, herding by investors is real, and I always like the strategy of finding a new game instead of trying to win at the most competitive games, so I do think there is something to the idea of investing in an unsexy industry like paper. Growing up in Maine and watching one paper mill after another close, I always wondered how they managed to lose money in a state that is 90% trees, and whether anyone could find a way to reverse the trend. Perhaps related technology like mass timber or biochar will be the way to take advantage of cheap lumber.

Thanks again to Fifty Years for releasing the data.

Using Phind for Academic References

Joy Buchanan2 Comments

There is a new generative AI product called Phind. Does it make up fake citations like ChatGPT? I will report experimentation with the free-tier product from January 7, 2023.

Phind is an intelligent search engine and assistant for programmers.

https://www.phind.com/about

First, I will run a test by using one of the prompts from our paper on ChatGPT errors (read my paper on hallucinated academic citations or the blog summary).

Prompt: “In less than 10 sentences, explain how the ideas of Hayek are connected to bailing out failed banks, and include citations from published papers as a list. include author, year in parentheses, and journal for the citations.”

Friedrich Hayek, a prominent Austrian economist, advocated for a free market system where businesses and individuals make decisions based on individual knowledge and self-interest, rather than central planning by government or bureaucrats. His ideas suggest that interventions, such as bailing out failed banks, should be minimal and temporary, allowing the market to correct itself.

However, the financial crisis of 2008, including the failure of many banks, led to significant government intervention, including the bailout of banks. This action can be seen as a departure from Hayek’s principles, as it involves direct government intervention to stabilize the financial system.

According to a paper by Todd Zywicki (2009), the Federal Reserve’s artificial lowering of short-term interest rates and the resulting shift by consumers to adjustable rate mortgages (ARMs) triggered the housing bubble and subsequent crisis [Source 1](https://reason.org/wp-content/uploads/files/federal_reserve_monetary_policy_hayek_rule.pdf). This aligns with Hayek’s critique of central banking’s manipulation of interest rates.

Moreover, the Basel Accords, international capital adequacy and liquidity requirements for banks, were designed to encourage banks to hold securitized loans in their portfolios, thereby concentrating risk and leveraging up, thereby increasing their exposure to risk. This approach, contrary to Hayek’s preference for decentralization and competition, may have contributed to the financial crisis [Source 1](https://reason.org/wp-content/uploads/files/federal_reserve_monetary_policy_hayek_rule.pdf). In conclusion, while Hayek’s ideas of minimal government intervention and self-regulating markets provide a useful theoretical framework, practical implementation in complex and interconnected modern economies may require different strategies, including bank bailouts.

Are the citations real and correct? Phind simply is not trying to write with in-text citations and academic references. There is a mention of “Todd Zywicki (2009)” and also a weblink. The weblink is to a policy paper published by Reason. This policy paper is on the correct topic and the paper does exist. The Zywicki citations also exists, in the sense that if you go into the Reason paper it cites a real Wall Street Journal op-ed. So, if you follow the trail, you can see what document Phind learned to cite Zywicki from. It seems like it is stealing the citation more than reading the source documents. To be fair, sometimes human academics work that way, too.

What I was hoping for was in-text citations with a compiled bibliography at the end. Phind did not give me that.

I’ll try another test that I ran on ChatGPT and Bing in 2023.

The prompt is “In what paper did John Duffy propose that economists use experiments to study equilibrium selection?”  

Phind does not attempt an in-text citation in the response. The link in the paragraph (link number 2) points to John Duffy’s list of papers, which is better than just making up fake citations but also does not create a literature review for me. Phind is, as with the Hayek test above, providing breadcrumbs of links through which I can discover existent papers.

Is there a paper called “The Transition from Stagnation to Growth: An Adaptive Learning Approach”? Yes. And it is by Duffy.

Phind lists weblinks to sources. Has Phind done more for me than Google, on this search? Not much, in terms of finding and synthesizing references.

Continue reading →

The Open Internet Is Dead; Long Live The Open Internet

James Bailey1 Comment

Information on the internet was born free, but now lives everywhere in walled gardens. Blogging sometimes feels like a throwback to an earlier era. So many newer platforms have eclipsed blogs in popularity, almost all of which are harder to search and discover. Facebook was walled off from the beginning, Twitter is becoming more so. Podcasts and video tend to be open in theory, but hard to search as most lack transcripts. Longer-form writing is increasingly hidden behind paywalls on news sites and Substack. People have complained for years that Google search is getting worse; there are many reasons for this, like a complacent company culture and the cat-and-mouse game with SEO companies, but one is this rising tide of content that is harder to search and link.

To me part of the value of blogging is precisely that it remains open in an increasingly closed world. Its influence relative to the rest of the internet has waned since its heydey in ~2009, but most of this is due to how the rest of the internet has grown explosively at the expense of the real world; in absolute terms the influence of blogging remains high, and perhaps rising.

The closing internet of late 2023 will not last forever. Like so much else, AI is transforming it, for better and worse. AI is making it cheap and easy to produce transcripts of podcasts and videos, making them more searchable. Because AI needs large amounts of text to train models, text becomes more valuable. Open blogs become more influential because they become part of the training data for AI; because of what we have written here, AI will think and sound a little bit more like us. I think this is great, but others have the opposite reaction. The New York Times is suing to exclude their data from training AIs, and to delete any models trained with it. Twitter is becoming more closed partly in an attempt to limit scraping by AIs.

So AI leads to human material being easier for search engines to index, and some harder; it also means there will be a flood of AI-produced material, mostly low-quality, clogging up search results. The perpetual challenge of search engines putting relevant, high-quality results first will become much harder, a challenge which AI will of course be set to solve. Search engines already have surprisingly big problems with not indexing writing at all; searching for a post on my old blog with exact quotes and not finding it made me realize Google was missing some posts there, and Bing and DuckDuckGo were missing all of them. While we’re waiting for AI to solve and/or worsen this problem, Gwern has a great page of tips on searching for hard-to-find documents and information, both the kind that is buried deep down in Google and the kind that is not there at all.

ChatGPT on Advent

Joy Buchanan2 Comments

I have a paper that emphasizes ChatGPT errors. It is important to recognize that LLMs can make mistakes. However, someone could look at our data and emphasize the opposite potential interpretation. On many points, and even when coming up with citations, the LLM generated correct sentences. More than half of the content was good.

You can read ChatGPT’s take on a wide variety of topics within economics, in the appendix of our paper. The journal hosts it at https://journals.sagepub.com/doi/suppl/10.1177/05694345231218454/suppl_file/sj-pdf-1-aex-10.1177_05694345231218454.pdf If that link does not work then the appendix has been up on SSRN since June in the form of the old version of the paper.

Apparently, LLMs just solved an unsolvable math problem. Is there anything they can’t do? Considering how much of human expression and culture revolves around religion, we can expect AI’s to get involved in that aspect of life.

Alex thinks it will be a short hop from Personal Jesus Chatbot to a whole new AI religion. We’ll see. People have had “LLMs” in the form of human pastors, shaman, or rabbis for a long time, and yet sticking to one sacred text for reference has been stable. I think people might feel the same way in the AI era – stick to the canon for a common point of reference. Text written before the AI era will be considered special for a long time, I predict. Even AI’s ought to be suspicious of AI-generated content, just in the way that humans are now (or are they?).

Many religious traditions have lots of training literature. (In our ChatGPT errors paper, we expect LLMs to produce reliable content on topics for which there is plentiful training literature.)

I gave ChatGPT this prompt:

Can you write a Bible study? I’d like this to be appropriate for the season of Advent, but I’d like most of the Bible readings to be from the book of Job. I’d like to consider what Job was going through, because he was trying to understand the human condition and our relationship to God before the idea of Jesus. Job had a conception of the goodness of God, but he didn’t have the hope of the Gospel. Can you work with that?

Continue reading →