Once undergraduates have learned the basics of interpreting regression results, we would like to introduce them to the world of economics research papers. Reading these papers will help reinforce the statistical concepts, and also we want them to get access to the insights in the literature.
Many empirical papers in economics are too long or too difficult to assign to undergraduates, especially if the course is focused more on analytics than economics specifically. Here I provide materials and instructions for teaching two published econ articles to undergraduates. Assume the students have learned the basics of interpreting a regression model (perhaps from a course textbook) but have had few opportunities to apply theses skills or engage in scientific literature.
“The Effects of Attendance on Student Learning in Principles of Economics” is only 4 pages long! Students do not need to read past page 7 of “My Reference Point, Not Yours” to answer the reading guide questions. So, these readings can be assigned outside of class, but I did some of the reading during our class period.
Handing out printed copies of at least one of the papers and my guided questions can make a good classroom activity. If students do not have experience reading tables of regression results, it can be useful to do it together in person.
The questions in the reading guide help students to identify the main variables and hypotheses. Then, students are asked to pull specific results from the tables in the papers. You can customize this list of questions by deleting lines if you do not want to discuss issues like non-linear effects or the null hypothesis.
I provide links below. First is the reading guide with about 30 short-answer questions about the two articles.
Link to download the reading guide that goes with both papers, starting with the shorter one.
3. Two web sources for “My Reference Point, Not Yours” (15 pages in total in the JEBO manuscript, but students do not need to read past page 7 for this exercise, and they can skip the Literature Review section)
The United States technology industry continues to struggle to recruit new talent. According to the US Bureau of Labor Statistics, the number of people employed in technology is not increasing quickly.
Tech jobs pay well and don’t have the drawbacks of some other in-demand jobs, such as the travel schedule of a truck driver or the physically taxing labor required in oil fields.
Tech jobs are sometimes touted as a guarantee of having a comfortable and rewarding career, but the reality is not that simple.
Economics suggests that high wages would eliminate labor shortages, but that’s not the case in tech work. Why?
In this paper, authors Joy Buchanan and Henry Kronk propose a set of factors that have been overlooked and apply broadly to the tech sector.
Individuals with high-status tech jobs report burnout, anxiety, depression, and other mental health issues at higher rates than the general population. They also have to deal with the constant threat of becoming obsolete. Because technology changes so quickly, they must constantly work to update their skills in order to remain competitive.
The authors offer several recommendations for tech companies, educators, and policymakers:
Political and community leaders can provide more accurate messaging such as communicating clearer expectations about the difficulties of entering the tech workforce.
The tech industry could benefit from improvements in computer education. The authors cite a need for more pre-college exposure to computer occupations as well as a need to add communication skills to computer science curriculums.
Teachers, parents, and tech companies can all find ways to inform young people at an age-appropriate level about opportunities. Computer science is abstract and hard to understand. Young people who have some exposure to computer science through a class or camp are more likely to become CS majors in college.
Company leaders can improve their recruitment and development strategies to reflect the labor market realities including paying enough to compensate employees for the mental challenges of demanding technical work and alleviating their own talent shortages by investing in training and education.
Tech companies may be able to attract more women and minorities by improving their scheduling and management practices.
Henry and I examined public data and the existing literature to get a better understanding of the current state of knowledge on this issue. I hope our paper can be helpful, however we partly just highlight how many questions still exist about tech and talent.
In the Fall of 2020, I blogged about how I introduce students to text mining, as part of a data analytics class.
Could Turing ever have imagined that a human seeking customer service from a bank could chat with a bot? Maybe text mining is a big advance over chess, but it only took about one decade longer for a computer (developed by IBM) to beat a human in Jeopardy. Winning Jeopardy requires the computer to get meaning from a sentence of words. Computers have already moved way beyond playing a game show to natural language processing.
I told the students that “chat bots” are getting better and NLP is advancing. By July 2020, OpenAI had released a beta API playground to external developers to play with GPT-3, but I did not sign up to use it myself.
In April of 2022, I added some slides inspired by Alex’s post about the Turing Test that included output from Google’s Pathway Languages Model. According to Alex, “It seems obvious that the computer is reasoning.”
This week in class, I did something that few people could have imagined 5 years ago. I signed into the free new GPTChat function in class and typed in questions from my students.
We started with questions that we assumed would be easy to answer:
Then we were surprised that it answered a question we had thought would be difficult:
And then we asked two questions that prompted the program to hedge, although for different reasons.
It seems like the model is smarter than it lets on. For now, the creators are trying hard not to offend anyone or get in the way of Google’s advertising business. Overall, the quality of the answers are high.
Because of when I was born, I believe that something I have published will make it into the training data for these models. Will that turn out to be more significant than any human readers we can attract?
This isn't even GPT-4, these are just the breadcrumbs, just wait for dessert: https://t.co/Avwdz0ZhfN
I was able to get a free Covid-19 booster shot reformulated to fight new Omicron strains. This was easy to schedule at Walgreens, and I got a flu shot at the same time to save time. Vaccines for all types of diseases are advancing.
I didn’t lose money on crypto.
When Russia invaded Ukraine in February of this year, I ordered Potassium Iodide tablets. I have not needed them.
Bringing one over from Dynomight (HT: Tyler) “That there’s been a 93% decline in stomach cancer deaths over the past 100 years—from by far the biggest killer among cancers to one of the smaller ones—and mostly this was an accident, it happened because better food refrigeration reduced infections of H. pylori, a bacterium that wasn’t even identified until 1982 after most of the decline had already happened.”
The community I follow was speculating, half-seriously, that Musk has abruptly fired so many employees that the entire site would just crash.
If Twitter goes down, you can find me during office hours, in my office: MW 12-3pm, TTh 10am-Noon, and by appointment
— Jeremy 'adjusted for inflation' Horpedahl 📈 (@jmhorp) November 18, 2022
My prediction is that many economists will stay on Twitter because it is such a useful venue for sharing work and ideas. There was also a sizable migration to Mastodon this week where a critical mass of economists will likely remain for a while. I made an account there @JoyBuchanan@econtwitter.net I don’t think my habits are going to change much right now, meaning I will not add being active on Mastodon to my current slate of activities. Personally, I find Facebook to be a place for meaningful connection in my professional community, as well, but that might be much harder for younger researchers to break into.
Here’s my “What if it all goes down?” moment. I’ll document a great tweet thread by Dennie van Dolder, who is an inspiration for the art form of announcing a working paper on Twitter. (Mastodon does not currently have the thread format, which might point people back to blogging on the margin?)
Dennie van Dolder of the University of Essex provides a tweet thread
🚨NEW PAPER🚨 We examine high-stakes strategic choice using more than 40 (!) years of data from the American TV game show "The Price Is Right". Want to know what we find? Read the thread below!👇 [or the paper here: https://t.co/AKfYK3SDaC] #EconTwitterpic.twitter.com/TIhdE6QFqg
He also provides a blog post with high production quality and the more traditional SSRN working paper. It’s 2022 and that’s the trifecta for putting work out online, or at least it was. I have a “News” and a “History” category for classifying these blog posts. Where does this post belong? I settled on “Technology”.
Most of the economists in this group use experiments as part of their empirical research. In this post I will highlight some recently published work that is in the tradition of Vernon Smith, who influenced all of us so much.
Abstract: We study minimal conditions for competitive behavior with few agents. We adapt a price-quantity strategic market game to the indivisible commodity environment commonly used in double auction experiments, and show that all Nash equilibrium outcomes with active trading are competitive if and only if there are at least two buyers and two sellers willing to trade at every competitive price. Unlike previous formulations, this condition can be verified directly by checking the set of competitive equilibria. In laboratory experiments, the condition we provide turns out to be enough to induce competitive results, and the Nash equilibrium appears to be a good approximation for market outcomes. Subjects, although possessing limited information, are able to act as if complete information were available in the market.
This small excerpt from their results shows a market converging toward equilibrium over time, under different treatment conditions. With some opportunities for practice and feedback, agents create surplus value by trading.
Figure 4 plots the average efficiency in each round in the four treatments. Efficiency is defined as the percentage of the maximum social surplus realized. … learning takes longer under the clearing house institution; hence, average efficiency under the clearing house institution presents a stronger upward trend over time. Under the clearing house institution, the average efficiencies start at levels lower than under the double auction institution, and remain statistically lower in the second half of the experiment. Nevertheless, we can observe from Fig. 4 that the upward trend of the efficiencies in clearing house treatments persist over time, and at the end of the experiment, the efficiency levels from the two institutions are close.
Every year I request posts about stuff the writers actually use. My logic is that a great wave of stuff-buying is coming, so let’s try to highlight the good items and reduce holiday waste.
For Children
James recommends buying a whole bounce house. It might seem like something you could only afford to rent once a year, but the price of buying one you can use at home is now less than $300. In a big room, you can even do this indoors. Be the Christmas hero. Check on the space requirements.
I recommend two games that help kids learn to read. These are a great complement to Kindergarten or 1st-grade reading assignments. With enough confidence, you can convince kids that these games are toys and not “a book?”.
Velcro Cut to Length – Zachary suggests: “Do you have a phone charger beside your bed that keeps falling on the ground? Just Velcro it to the nightstand lamp and it will stay exactly where you want it.”
Minute Soil is better than the dirt you have. This makes growing plants more fun and easier. Sounds like a great gift to wrap up for someone who likes gardening.
Barge All Purpose TF Cement Rubber – Praise from Scott: “Unlike most “superglues”, it will work on rough or porous surfaces, including situations like leather where flexibility is needed.”
Qwix Mix windshield – Windshield wiper fluid concentrate that is easy to store at home for when you need it.
Lastly, Mike has some correct life advice. Give yourself what your future self would want. For example, if you enjoy video games but don’t exercise enough, then try setting up an exercise bike right in front of your video games. That way you’ll get your cardio in and not have regrets the next week.
Grosch, Kerstin, Simone Haeckl, and Martin G. Kocher. “Closing the gender STEM gap-A large-scale randomized-controlled trial in elementary schools.” (2022).
These authors were thinking about the same problem at the same time, unbeknownst to me. In their introduction they write, “We currently know surprisingly little about why women still remain underrepresented in STEM fields and which interventions might work to close the gender STEM gap.”
My conclusion from my paper is that, by college age, subjective attitudes toward tech are very important. This leads to the questions of whether those subjective attitudes are shaped at younger ages. Grosch et al. have run an experiment to target 3rd-graders with a STEM-themed game. I’ll quote their description:
The treatment web application (treatment app) intends to increase interest in STEM directly by increasing knowledge and awareness about STEM professions and indirectly by addressing the underlying behavioral mechanisms that could interfere with the development of interest in STEM. The treatment app presents both fictitious and real STEM professionals, such as engineers and programmers, on fantasy planets. Accompanied by the professionals, the children playfully learn more about various societal challenges, such as threats from climate change and to public health, and how STEM skills can contribute to combating them. The storyline of the app comprises exercises, videos, and texts. The app also informs children about STEM-related content in general. To address the behavioral mechanisms, the app uses tutorials, exercises, and (non-monetary) rewards that teach children a growth mindset and improve their self-confidence and competitive aptitude. Moreover, the app introduces female STEM role models to overcome stereotypical beliefs. To test the app’s effect, we recruited 39 elementary schools in Vienna (an urban area) and Upper Austria (a predominantly rural area).
This is a preview of their results, although I recommend reading their paper to understand how these measurements were made:
Girls’ STEM confidence increases significantly in the treatment group (difference: 0.047 points or 0.28 standard deviations, p = 0.002, Wald test), and the effect for girls is significantly larger than the effect for boys.
Result 2: Children’s competitiveness is positively associated with children’s interest in STEM. We do not find evidence that stereotypical thinking and a growth mindset is associated with STEM interest.
Lastly, my kids play STEM-themed tablet games. PBS Kids has a great suite of games that are free and educational. Unfortunately, I have not tried to treat one kid while giving the other kid a placebo app, so my ability to do causal inference is limited.
I recommend two games for teaching kids to read their “sight words”. In early school grades, learning sight words can mean doing boring homework or rote memorization of flash cards. Instead use
These are both fun interactive games that will get kids reading and talking about sight words. Zingo Sight Words is easier, so I recommend starting there. It’s a lot like bingo with a fun plastic dispenser. Kids can do the matching task to win the game even if they are not yet confident with reading.
Sight Word Swat is a little more advanced but good for expanding vocabulary past the first 50 words. It’s fast paced and fun. Someone yells out a word and then two players compete to “swat” with a plastic mallet the correct “fly” that has the word. Also, if the kid isn’t competitive, they could swat the correct word without time pressure.
Next, I’ll recommend a game that will not remotely feel like an educational exercise. “Spot It” is a genius card game. The tin is small, so you can store it easily and travel with it. The game is easy to teach to new friends because it’s just matching visual patterns. Spot It requires zero reading – not even reading numbers. So, a kid as young as 4 could potentially jump in and start trying to get matches. One of the great things about Spot It is that you play a series of mini games. It’s not the nightmare of a Monopoly game that could take multiple days to finish. So, if you are a parent with limited time to spend on card games, you can parachute in and out quickly.
All of these items are under $20 and potentially all of them could make fun holiday gifts, although your mileage may vary for gifting books and getting smiles. Personally, I bought the sight word games when we needed them for learning instead of trying to make them Christmas gifts.
I had been looking forward to reading the Phantom Tollbooth with my kids for a long time. This is the kind of book that you should read as soon as they are ready to understand most of the action, but not before. If too much is going over their heads, then it isn’t fun. In my case, this book prompted a lot of questions and great conversations with the 7-year-old. The book will teach kids a lot, but if you keep your tone light it feels like just another adventure story.
This is the second of two blog posts on my paper “Willingness to be Paid: Who Trains for Tech Jobs”. Follow this link to download the paper from Labour Economics (free until November 27, 2022).
Women did not reject a short-term computer programming job at a higher rate than men.
For the incentivized portions of the experiment, women had the same reservation wage to program. Women also seemed equally confident in their ability after a belief elicitation.
The main gender-related outcomes were, surprisingly, null results. I ran the experiment three times with slightly different subject pools.
However, I did find that women might be less likely to pursue programming outside of the experiment based on their self-reported survey answers. Women are more likely to say they are “not confident” and more likely to say that they expect harassment in a tech career.
In all three experiments, the attribute that best predicted whether someone would program is if they say they enjoy programming. This subjective attitude appears more important even than having taken classes previously.
Along with “enjoy programming” or “like math”, subjects who have a high opportunity cost of time were less willing to return to the experiment to do programming at a given wage level.
I wrote this paper partly written to understand why more people are not attracted to the tech sector where wages are high. This recent tweet indicates that, although perhaps more young people are training for tech than ever before, the market price for labor is still quite high.
The tech wages are too damn high.
— Antonio García Martínez (agm.eth) (@antoniogm) October 7, 2022
The neat thing about controlled experiments is that you can randomly assign treatment conditions to subjects. This post is about what happened after adding either extra information or providing encouragement to some subjects.
Informed by reading the policy literature, I assumed that a lack of confidence was a barrier to pursuing tech. A large study done by Google in 2013 suggested that women who major in computer science were influenced by encouragement.
I provided an encouraging message to two treatment groups. The long version of this encouraging message was:
If you have never done computer programming before, don’t worry. Other students with no experience have been able to complete the training and pass the quiz.
Not only did this not have a significant positive effect on willingness to program, but there is some indication that it made subjects less confident and less willing to program. For example, in the “High Stakes” experiment, the reservation wage for subjects who had seen the encouraging message was $13 more than for the control subjects.
My experiment does not prove that encouragement never matters, of course. Most people think that a certain type of encouragement nudges behavior. My results could serve as a cautionary tale for policy makers who would like to scale up encouragement. John List’s latest book The Voltage Effect discusses the difficulty of delivering effective interventions at scale.
The other randomly assigned intervention was extra information, called INFO. Subjects in the INFO treatment saw a sample programming quiz question. Instead of just knowing that they would be doing “computer programming,” they saw some chunks of R code with an explanation. In theory, someone who is not familiar with computer programming could be reassured by this excerpt. My results show that INFO did not affect behavior. Today, most people know what programming is already. About half of subjects said that they had already taken a class that taught programming. Perhaps, if there are opportunities for educating young adults, it would be in career paths rather than just the technical basics.
Since the differences between treatments turned out to be negligible, I pooled all of my data (686 subjects total) for certain types of analysis. In the graph below, I group every subject as either someone who accepted the programming follow-up job or as someone who refused to return to program at any wage. Recall that the highest wage level I offered was considerably higher on a per-hour basis than what I expect their outside earning option to be.
Fig. 5. Characteristics of subjects who do not ask for a follow-up invitation, pooling all treatments and sample
I’ll discuss the three features in this graph in what appear to be the order of importance for predicting whether someone wants to program. There was an enormous difference in the percent of people who were willing to return for an easy tedious task that I call Counting. By inviting all of these subjects to return to count at the same hourly rate as the programming job, I got a rough measure of their opportunity cost of time. Someone with a high opportunity cost of time is less likely to take me up on the programming job. This might seem very predictable, but this is a large part of the reason why more Americans are not going into tech.
Considering the first batch of 310 subjects, I have a very clean comparison between the programming reservation wage and the reservation wage for counting. People who do not enjoy programming require a higher payment to program than they do to return for the counting job. Self-reported enjoyment is a very significant factor. The orange bar in the graph shows that the majority of people who accepted the programming job say that they enjoy programming.
Lastly, the blue bar shows the percent of female subjects in each group. The gender split is nearly the same. As I show several ways in the paper, there is a surprising lack of a gender gap for incentivized decisions.
I hope that my experiment will inspire more work in this area. Experiments are neat because this is something that someone could try to replicate with a different group of subjects or with a change to the design. Interesting gaps could open up between subject types under new circumstances.
The topic of skill problems in the US represents something reasonably new for labor market and public policy discussions. It is difficult to think of a labor market issue where academic research or even research using standard academic techniques has played such a small role, where parties with a material interest in the outcomes have so dominated the discussion, where the quality of evidence and discussion has been so poor, and where the stakes are potentially so large.
Cappelli, PH, 2015. Skill gaps, skill shortages, and skill mismatches: evidence and arguments for the United States. ILR Rev. 68 (2), 251–290.
