These next two are STEM-friendly goofy books with lots of pictures to break up the text. Currently, if he has to do independent reading time, these are first choices.
Ender’s Game – Still incredible. Still feels futuristic, although the emphasis on “smart desks” instead of “smart phones” is funny. It was published in 1985.
The 5-year-old picked out “Woodpecker Wants a Waffle” at our local library and has been delighted by it. It’s a funny picture book that holds up to re-reads. See if your library has it. Public libraries are especially great for 5-year-olds who are ready to explore beyond the Dr. Suess classics at home but also not able to commit to any books worth buying.
I found that enjoyment or subjective preferences are underrated in the policy literature on the skills gap and promoting STEM in America. In a presentation in the Spring of ’23, I speculated that ChatGPT or other AI-assisted coding tools would make coding less tedious and therefore more fun.
I observe in this set of memes (posted in February 2024) that ChatGPT is already embedded in coding life, and yet it does not feel like anything fundamental has changed. Workers still Google their own way to solutions (although surely that has diminished somewhat due to LLMs). The work still feels hard and the workers still feel undervalued.
Senior programmers today would have grown up working very closely with search engines, largely to harvest the vast knowledge contained in tech message boards. I myself use that tool often when I have to program. Part of learning to code is just learning how to get help. This requires a certain mindset that is different from what is traditionally taught in school.Many such cases.Many people who end up as programmers want to do better. They are driven to write clean sensible code. A common theme is frustration that the product does not match the vision. This sentiment comes up more frequently than I have seen in other professions. The work they do is truly hard, and they are rarely afforded enough time to do it “right.” Continue reading →
Ladies, Tyler Cowen has done us a solid. He included John Stuart Mill as a contender for the greatest economist of all time in large part because of his insights on gender equality.
I’m short on time at the moment. I’d like to do a better job than this, with more nuance about Hayek, but here’s the most I can do this week:
Have you been told that economists only care about money? If anyone would tell Taylor Swift to focus on her own career, you might think it would be the most famous economist of all, Adam Smith. But AdamSmithWorks fans already know that Adam Smith was concerned with the whole person. So, would Adam Smith advise Taylor Swift to rush back to America after an exhausting concert just to cheer on a man in a football game?
I’m getting ready to give some public local talks about AI. Last week I shared some pictures that I think might help people understand ChatGPT, specifically:
My first thought is that GPT-4 was giving incorrect estimates of the heights of these mountains because it does not actually “know” the correct elevations. But then a nagging question came to mind.
GPT has a “creativity parameter.” Sometimes, it intentionally does not select the top-rated next word in a sentence, for example, in order to avoid being stiff and boring. Could GPT-4 know the exact elevation of these mountains, and it is just intentionally being “creative,” in this case?
I do not want to stand up in front of the local Rotary Club and say something wrong. So, I went to a true expert, Lenny Bogdonoff, to ask for help. Here is his reply:
Not quite. It’s not that it knows or doesn’t know, but based on the prompt, it’s likely unable to parse the specific details and is outputting results respectively. There is a component of stochastic behavior based on what part of the model weights are activated.
One common practice to help avoid this and see what the model does grasp, is to ask it to think step by step, and explain its reasoning. When doing this, you can see the fault in logic.
All that being said, the vision model is actually faulty in being able to grasp the relative position of information, so this kind of task will be more likely to hallucinate.
There are better vision models, that aren’t OpenAI based. For example Qwen-VL-Max is very good, from the Chinese company Alibaba. Another is LLaVA which uses different baselines of open source language models to add vision capabilities
Depending on what you are needing vision for, models can be spiky in capability. Good at OCR but bad at relative positioning. Good at classifying a specific UI element, but bad at detecting plants, etc etc.
Joy: So, I think I can tell the Rotary Club that GPT was “wrong” as opposed to “intentionally creative.” I think, as I originally concluded, you should not make ChatGPT the pilot of your airplane and go to sleep when approaching the Alps. ChatGPT should be used for what it is good at, such as writing the rough draft of a cover letter. (We have great “autopilot” software for flying planes, already, without involving large language models.)
Another expert, Gavin Leech, also weighed in with some helpful background information:
the creativity parameter is known as temperature. But you can actually radically change the output (intelligence, style, creativity) by using more complicated sampling schemes. The best analogy for changing the sampling scheme is that you’re giving it a psychiatric drug. Changing the prompt, conversely, is like CBT or one of those cute mindset interventions.
For each real-name model (e.g. “gpt-4-0613”), there’s 3 versions: the base model (which now no one except highly vetted researchers have access to), the instruction-tuned model, and the RLHF (or rather RLAIF) model. The base model is wildly creative, unhinged, but the RLHF one (which the linked researchers use) is heavily electroshocked into not intentionally making things up (as Lenny says).
It’s currently not usually possible to diagnose an error – the proverbial black box. My friends are working on this though
For more, note OpenAI admitting the “laziness” of their own models. the Turbo model line is intended to fix this.
Thank you, Lenny and Gavin, for donating your insights.
By now, everyone should consider using ChatGPT and be familiar with how it works. I’m going to highlight resources for that.
My paper about how ChatGPT generates academic citations should be useful to academics as a way to quickly grasp the strengths and weakness of ChatGPT. ChatGPT often works well, but sometimes fails. It’s important to anticipate how it fails. Our paper is so short and simple that your undergraduates could read it before using ChatGPT for their writing assignments.
A paper that does this in a different domain is “GPT4GEO: How a Language Model Sees the World’s Geography” (Again, consider showing it to your undergrads because of the neat pictures, but probably walk through it together in class instead of assigning it as reading.) They describe their project: “To characterise what GPT-4 knows about the world, we devise a set of progressively more challenging experiments… “
For example, they asked ChatGPT about the populations of countries and found that: “For populations, GPT-4 performs relatively well with a mean relative error (MRE) of 3.61%. However, significantly higher errors [occur] … for less populated countries.”
ChatGPT will often say SOMETHING, if prompted correctly. It is often, at least slightly, wrong. This graph shows that most estimates of national populations were not correct and the performance was worse on countries that are less well-known. That’s exactly what we found in our paper on citations. We found that very famous books are often cited correctly, because ChatGPT is mimicking other documents that correctly cite those books. However, if there are not many documents to train on, then ChatGPT will make things up.
I love this figure from the geography paper showing how ChatGPT estimates the elevations of mountains. This visual should be all over Twitter.
There are 3 lines because they did the prompt three times. ChatGPT threw out three different wrong mountains. Is that kind of work good enough for your tasks? Often it is. The shaded area in the graph is the actual topography of the earth in those places. ChatGPT “knows” that this area of the world is a mountain. But it will just put out incorrect estimates of the exact elevation, instead of stating that it does not know the exact elevation of those areas of the world.
The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.
If you feel like you already are proficient with using ChatGPT, then I would recommend Wolfram’s blog because you will learn a lot about math and computers.
Scott wrote “Generative AI Nano-Tutorial” here, which has the advantage of being much shorter than Wolfram’s blog.
I don’t have much time to write this week because I lost so many work hours to schools closing for “weather.”
Tyler has been saying that we should welcome more intelligence (in the form of LLMs – I’m not getting any smarter). What would we want intelligence for? How about reducing the error rate on school closing?
First, I will recognize that things are already getting better due to computers. The internet and texting and radar help. Compared to when I was a child in New Jersey, it’s more efficient to text all the parents the night before, as opposed to having people get up at 6am to scan the radio for news. Weather forecasting has presumably gotten better.
Now my rant: Right around what was already a three-day official weekend, school was closed three times. Even my kids were irate when that last day was announced. In my opinion, only one of those closures was justified for extreme weather.
There is a lot of dumb in a city. People complain about routine processes being suboptimal. It would be great if we humans could figure out ways to apply more intelligence to these local problems and make less mistakes.
This is a joke for any readers in cold climates. My Alabama kids thought it was fun to collect icicles because they have almost never seen them before.
In addition to all the usual items for a principles of macroeconomics class, I’m asking my students to listen to one podcast episode this semester. They have to write a short summary on a discussion board for credit.
It took me a bit of time to collect this list of links. I also give them some discretion to find their own episode, but I’m not posting my rules on that point here. This list is something you can copy, paste, and modify. The point is to have all the web links in one place so that students can just click around. There have been many great podcasts over past 2 decades, but I list relatively new content so that we get a bit of “current events” thrown in. So, even if you’ve assigned podcasts before, this new list might be helpful.
Are smartphones bad for kids? Cal walks through the data on this question, including how researchers came to be worried, their findings, critiques of their findings, and where we are today. He then gives recommendations for how to think about technology when it comes to your kids.
In May of 2023, Cal Newport shared well-informed opinions about whether smartphones harm young people. In the first half of the podcast, he talks about depression and loneliness data.
Minute 30 of the podcast: Screentime harms teenagers because they inhibit the development of critical thinking skills. Deep critical thinking skills require training. Reading an analog book is better than screens (see my review of Tyler’s AI generative book and poastmodernism).
See my summary of Emily Oster on video games for kids. She does not clutch her pearls over violent video games. However, she is concerned about what activities get crowded out by screentime. She is especially worried about sleep, because on that topic the data are clear.
Minute 31, Call Newport: Tweens and teens scroll on their phones for too long instead of going to sleep. A 13-year-old boy with a smart phone will “be up until 4 in the morning.” A tween told him that middle school girls arrive at school too exhausted to function because they have been on their phones all night.
FYI, if you are the parent in an Apple device network, you can set time limits on the devices in your family. I filed this report about smart watches last year, incidentally in the same week as the release of Newport’s podcast episode.
One mom told me that Apple Watch is considerably more durable than a Gizmo (I wouldn't know from experience). And, my son described "Watch Jail" where his teacher keeps smart watches overnight if there are usage infractions during the school day.
There is a new generative AI product called Phind. Does it make up fake citations like ChatGPT? I will report experimentation with the free-tier product from January 7, 2023.
Phind is an intelligent search engine and assistant for programmers.
Prompt: “In less than 10 sentences, explain how the ideas of Hayek are connected to bailing out failed banks, and include citations from published papers as a list. include author, year in parentheses, and journal for the citations.”
Friedrich Hayek, a prominent Austrian economist, advocated for a free market system where businesses and individuals make decisions based on individual knowledge and self-interest, rather than central planning by government or bureaucrats. His ideas suggest that interventions, such as bailing out failed banks, should be minimal and temporary, allowing the market to correct itself.
However, the financial crisis of 2008, including the failure of many banks, led to significant government intervention, including the bailout of banks. This action can be seen as a departure from Hayek’s principles, as it involves direct government intervention to stabilize the financial system.
According to a paper by Todd Zywicki (2009), the Federal Reserve’s artificial lowering of short-term interest rates and the resulting shift by consumers to adjustable rate mortgages (ARMs) triggered the housing bubble and subsequent crisis [Source 1](https://reason.org/wp-content/uploads/files/federal_reserve_monetary_policy_hayek_rule.pdf). This aligns with Hayek’s critique of central banking’s manipulation of interest rates.
Moreover, the Basel Accords, international capital adequacy and liquidity requirements for banks, were designed to encourage banks to hold securitized loans in their portfolios, thereby concentrating risk and leveraging up, thereby increasing their exposure to risk. This approach, contrary to Hayek’s preference for decentralization and competition, may have contributed to the financial crisis [Source 1](https://reason.org/wp-content/uploads/files/federal_reserve_monetary_policy_hayek_rule.pdf). In conclusion, while Hayek’s ideas of minimal government intervention and self-regulating markets provide a useful theoretical framework, practical implementation in complex and interconnected modern economies may require different strategies, including bank bailouts.
Are the citations real and correct? Phind simply is not trying to write with in-text citations and academic references. There is a mention of “Todd Zywicki (2009)” and also a weblink. The weblink is to a policy paper published by Reason. This policy paper is on the correct topic and the paper does exist. The Zywicki citations also exists, in the sense that if you go into the Reason paper it cites a real Wall Street Journal op-ed. So, if you follow the trail, you can see what document Phind learned to cite Zywicki from. It seems like it is stealing the citation more than reading the source documents. To be fair, sometimes human academics work that way, too.
What I was hoping for was in-text citations with a compiled bibliography at the end. Phind did not give me that.
I’ll try another test that I ran on ChatGPT and Bing in 2023.
The prompt is “In what paper did John Duffy propose that economists use experiments to study equilibrium selection?”
Phind does not attempt an in-text citation in the response. The link in the paragraph (link number 2) points to John Duffy’s list of papers, which is better than just making up fake citations but also does not create a literature review for me. Phind is, as with the Hayek test above, providing breadcrumbs of links through which I can discover existent papers.
Is there a paper called “The Transition from Stagnation to Growth: An Adaptive Learning Approach”? Yes. And it is by Duffy.
Phind lists weblinks to sources. Has Phind done more for me than Google, on this search? Not much, in terms of finding and synthesizing references.
Economist Writing Every Day 