Solving the Participation Pickle with Pick.al

Joy: This post was written by my friend and fellow econ professor Cameron Hardwick.

One of my biggest ongoing teaching challenges is keeping students engaged during lectures.

Sure, there are ways to add interactivity here and there, but sometimes there’s just no way around an old-fashioned lecture.

There are a few ways of dealing with this, and I haven’t been satisfied with any.

  1. It’s their grade, if they zone out that’s on them. In terms of the incentives, sure, the externalities are all internalized. But as a macroeconomist, I also know: if time-inconsistency problems are hard for policymakers, how much more for students! We shouldn’t be surprised when students do poorly if the main feedback they get from paying attention or not comes a week later with the homework grade.
  2. Posing questions and waiting for answers. Either you get a minute of awkward silence, or you get the same two engaged students answering everything while everyone else keeps zoning out.
  3. Cold calling. I started doing this a few years into teaching. The advantage is that it keeps students on their toes and paying attention. But a few problems left me unsatisfied:
    1. “How about you in the red shirt”. Hard to catch a student’s attention that way, and in a class of 40 or more, learning names takes a good chunk of the semester.
    1. I had no systematic way of keeping track of participation. Every semester I’d look at the roster and still have a few names I couldn’t put a face to.
    1. Humans are really bad at making random choices! Much as I tried, I couldn’t guarantee I wasn’t biased toward or against (say) the corners of the room, or students whose names I knew.
  4. LMS software. These can offer a lot of great student participation tools. But students have to pay for them – which isn’t worth it if you’re just looking for one feature. On top of that, then you’re locked into an ecosystem.

So, I made an app myself. It does one thing and does it well.

Pick.al (pronounced Pickle) picks students at random from a roster and keeps track of participation points. I can now pose a question in class, ask “what do you think…”, pull out my phone and hit a button, and have a name.

I can also record the quality of their answers:

  • ✓: 1 point, good attempt! (Since this is for participation points, I record ✓ whether right or wrong, as long as they give it a good shot)
  • ?: 0.5 points, if they ask “wait, what was the question?”
  • ×: 0 points, if they’re not there or don’t respond at all.

There’s also a 1-5 scale option, for those who want a more fine-grained evaluation.

This has a lot of benefits in the classroom:

  • Since I can call on students by name, I learn names more quickly.
  • Pick.al chooses randomly from the pool of students who have been called on the least so far. So, I know my participation points are as fair as possible.
  • Students know they can get called on at any time, so they pay attention more in class, and then do better on the homeworks and tests.
  • Students appreciate being brought in more frequently. One noted on the evaluations the first semester I piloted it: “something specific I like is he got the class involved by calling people out which forced them to test their knowledge which is something teachers need to do more of.”

Using Pick.al is as simple as registering (with an email address or an OrcID), uploading a roster, and then hitting a button during class. You can also swipe through the history and edit or undo participation events, and go back in the admin interface and add, edit, and remove participation events after the fact if necessary.

Pick.al is secure and password-protected, and has a number of handy features:

  • You can set excused absences if a student lets you know beforehand, so their name doesn’t come up until a certain date.
  • You can select specific students from the roster in a sidebar, if you want to give credit to – say – a student who raises his hand unbidden.
  • If you’d like to use the classroom computer instead of pulling out a phone, you can use it with full keyboard navigation.
  • Scores can be downloaded as a CSV to be put in your own gradebook.
  • Private notes can be added to students to show up when their names are selected, e.g. “sits in the back corner”

If you use it and find a bug or have an idea that would make it more useful to you, feel free to let me know. It’s been a great tool in my own classes, and I hope it’ll be useful for other teachers to keep students engaged too.

Review of Cowen Tabarrok Econ Textbook

It’s been almost a decade since I taught principles of economics classes. One major allocation of my time this semester is course prep, since I am teaching 3 different classes.

For my Principles of Microeconomics course, I chose Modern Principles of Economics, because I figured Tyler and Alex had done a good job and I have heard good reviews from others.

I’m writing a short review of their instructor resources, and then I’ll have to get back to course prep.

  • Like most textbooks, they provide you with slides that you can modify. Not having to start from scratch on lecture slides is great.
  • They also have teacher guides for each chapter. I find these helpful, because I have not taught this class in many years. Even though “I’m an economist,” there is still a technique to presenting these ideas for the first time to undergraduates. No need to re-invent that wheel completely.
  • They have suggestions for in-class activities. For example, to illustrate demand shifts, ask the students about a recent celebrity scandal and how that created a fall in the demand for concert tickets. It works. Everyone loves talking about celebrity scandals. It will be an evergreen idea. There are always new scandals for each semester – the students know more about it than the professors. My students (Fall 2023) informed me that Lizzo got in hot water for fat-shaming.
  • Their online learning platform called Achieve within MacMillan works well. It integrates really well with Canvas, our LMS. One warning I would give you is to make sure that students buy Achieve through an account on their .edu email address. I have headaches over students signing up with a personal email address and then not having their data integrate with Canvas.
  • You can sign up for EconInbox, which will email you topical relevant news stories right before you would want to present them in class. You’ll have to tell them ahead of time what your schedule of topics is, but that is something you ought to have worked out in your syllabus at the beginning of the semester. Obviously, you can’t cover every chapter in one semester. There are far more resources, generally, than you can use. But picking and choosing from a great library is easier than trying to build something from scratch yourself.
  • Lastly, the Cowen Tabarrok textbook integrates nicely with the free Marginal Revolution University video library. MRU is free to all. So, as an instructor you could still use it heavily even if you are not assigning their textbook or even if you are not doing Achieve. Still, I think that making use of the MRU resource is easiest if you are using their textbook. A fun video that might even be worth using class time for is Avengers: The Story of Globalization

Rat to Research Discourse

I made this Decreasing Marginal Utility rat picture when I was an undergraduate, and it caught on. A textbook asked me for permission to print it.

This week on Twitter (X.com), someone said it was their favorite graph. Upon replying I learned that he had used it for teaching. It’s fun when you know one of your ideas is out in the world helping people.

For real? I absolutely HOWLED when I found it on a google image search! Bravo! I taught HS Econ for many years and this was the kind of stuff that kept kids awake!

https://twitter.com/arburnside/status/1702690454884487495

Blogger privilege is to manifest a new conversation on here. If one of my research articles were to achieve the same level of influence as the stuffed rat, then people might tweet something along the following lines:

An Experiment on Protecting Intellectual Property,” (2014), with Bart Wilson. Experimental Economics, 17:4, 691-716.

This original project, both in terms of methodology and subject, is one of the first controlled experiments on intellectual property protection, which has inspired subsequent lab work on this issue. We present a color cube mechanism that provides a creative task for subjects to do in an experiment on creative output. The results indicate that IP protection alone does not cause people to become inventors, although entrepreneurs are encouraged to specialize by IP protection.

Smile, Dictator, You’re On Camera,” (2017), with Matthew McMahon, Matthew Simpson and Bart Wilson. Southern Economic Journal, 84:1, 52-65.

The dictator game (DG) is attractive because of its simplicity. Out of thousands of replications of the DG, ours is probably the controlled experiment that has reduced “social distance” to the farthest extreme possible, while maintaining the key element of anonymity between the dictator and their receiver counterpart. In our experiment the dictator knows they are being watched, which is the opposite of the famous “double-blind” manipulation that removed even the view of the experimenter. As we predicted, people are more generous when they are being watched. Anyone teaching about DGs in the classroom should show our entertaining video of dictators making decisions in public: https://www.youtube.com/watch?v=vZHN8xyp6Y0&t=22s

My Reference Point, Not Yours,” (2020) Journal of Economic Behavior and Organization, 171: 297-311.

There is a lot of talk about reference points. No matter how you feel about “behavioral” economics, I don’t think anyone would deny that reference-dependent behavior explains some choices, even very big ones like when to sell your house. Considering how important reference points are, can people conceive of the fact that different people have different reference points shaped by their different life experiences? Results of this study imply that I tend to assume that everyone else has my own reference point, which biases my beliefs about what others will do. Because this paper is short and simple, it would make a good assignment for students in either an experimental or econometrics class. I have a blog post on how to turn this paper into an assignment for students who are just learning about regression for the first time.

If Wages Fell During a Recession,” (2022) with Daniel Houser, Journal of Economic Behavior and Organization.  Vol. 200, 1141-1159.

The title comes from Truman Bewley’s book Why Wages Don’t Fall during a Recession. First, I’ll take some lines directly from his book summary:

A deep question in economics is why wages and salaries don’t fall during recessions. This is not true of other prices, which adjust relatively quickly to reflect changes in demand and supply. Although economists have posited many theories to account for wage rigidity, none is satisfactory. Eschewing “top-down” theorizing, Truman Bewley explored the puzzle by interviewing—during the recession of the early 1990s—over three hundred business executives and labor leaders as well as professional recruiters and advisors to the unemployed.

By taking this approach, gaining the confidence of his interlocutors and asking them detailed questions in a nonstructured way, he was able to uncover empirically the circumstances that give rise to wage rigidity. He found that the executives were averse to cutting wages of either current employees or new hires, even during the economic downturn when demand for their products fell sharply. They believed that cutting wages would hurt morale, which they felt was critical in gaining the cooperation of their employees and in convincing them to internalize the managers’ objectives for the company.

We are one of the first to take this important question to the laboratory. The nice thing about an experiment is that you can measure shirking precisely and you can get observations on wage cuts, which are rare in the naturally occurring American economy.

We find support for the morale theory, but a new puzzle got introduced along the way. Many of our subjects in the role of the employer cut the wages of their counterpart, which probably lowered their payment. Why didn’t they anticipate the retaliation against wage cuts? That question inspired the paper “My Reference Point, Not Yours.”

Other people’s money: preferences for equality in groups,” (2022) with Gavin Roberts, European Journal of Political Economy, Vol. 73.

Andreoni & Miller (2002) have been cited over 2500 times for their experiment that shows demand curves for altruism slope down. Economic theory is not broken by generosity. We extend their work to show that demand curves for equality slope down. Individuals don’t love inequality, but they also don’t love parting with their own money. There is a higher demand for reducing inequality with other people’s money than with own income.

Willingness to be Paid: Who Trains for Tech Jobs?” (2022), Labour Economics, Vol 79, 102267. 

This is the last paper I’ll do here. At this point, readers probably would like a funny animal picture. Here’s a meme about the difficult life of computer programmers:

For decades, tech skills have had a high return in the labor market. There is very little empirical work on why more people do not try to become computer programmers, although there are policy discussions about confidence and encouragement.

I ran an experiment to measure something that is important and underexplored. One thing I found is that attempts to increase confidence, if not carefully evaluated, might backfire.

Would you predict it’s more important to have taken a class in programming or for a potential worker to report that they enjoy programming? My results imply that we should be doing more to understand both the causes and effects of subjective preferences (enjoyment) for tech work. 

A few more decades to go here… I will try to top the stuffed rat picture.

[Not] Choosing Rationally

I’ve written previously on game theory, about the generality of Pure Strategy Nash Equilibria (PSNE), and the drawbacks of Sub-Game Perfect Nash Equilibria (SGPE). In this post I have another limitation for SGPE.


First, some definitions:
PSNE: “No player can change one of their strategies and improve their payoff, given the strategies of all other players.”
Subgame: “A subset of any extensive-form game that includes an initial node (which doesn’t share an information set with other nodes) and all its successor nodes.”
Subgame Equilibrium (SGE): “The PSNE of the Subgame”
SGPE: “The set of PSNE that are also SGE”


Clearly, there is nothing inconsistent about the above definitions. The reason that SGPE emerged was because some PSNE assert that a player would be willing to choose strategies that do not maximize conditional payoffs in subgames that are off of the equilibrium path. So, people often characterize the SGPE as a player ‘being rational each step of the way in each subgame’.

But, there is a problem. “Each step of the way” and “in each subgame” are not the same thing. Each step of the way implies that a player is rational at each decision – ie, at each information set. But, not every information set is a subgame! So, a SGPE can include rationality at each SGE while also permitting some irrationality at individual information sets. Since economists like to identify the bounds of their claims, let me emphasize the word can. In order to be correct, I need only identify one case in which the claim is true.


Here is that case:

Continue reading

Interpolation Vs Transition

Sometimes you read an academic article and the author fills in the data gaps with interpolation. That is, they assume some functional form of the data and then replace the missing values with the estimated ones. Often, lacking an informed opinion about functional form, authors will just linearly interpolate between the closest known values. Sometimes this method is OK. But sometimes we can do better.

Historical census data provides a good example because the frequency was only every ten years. Say that we want to know more about child migration patterns between 1850 and 1860. What happened in the intervening years? Who knows. Let’s look at the data.

Using data on individuals who have been linked across censuses allows us to fill in the gaps a little bit. For simplicity, let’s just look at whether a child migrant lived in an urban location and whether they lived on a farm. That means that there are 4 possible ways to describe their residence. Below is a summary of where children migrants lived at the age of zero in 1850 and where the same children lived a decade later at the age of ten in 1860 given that they moved counties.

When I’m the mean time did these children move from one place and to the other? We don’t know exactly. The popular answer is to say that they moved uniformly throughout the decade. That’s ‘fine’. But it assumes that the rate at which people departed places was rising and the rate at which they arrived places was falling. Maybe that’s true, but we don’t really know. Below-left is a graph that shows the linear interpolation.

The nice thing about linear interpolation is that everyone is accounted for at each point in time. The total number of people don’t rise or fall in the intervening interpolation period. But if we were to assume that children departed/arrived at each type of place at a constant rate (maybe a more reasonable assumption), then suddenly we lose track of people. That is, the sum of people dips below 100% as people depart faster than they arrive.

What’s the alternative to linear interpolation?

Continue reading

Life Tables are Cool

Demography is cool generally, but life tables are really cool in their elegance. Don’t know what a life table is? Let me ‘splain.

A life table uses data from private or public death registers, or even genealogical records, to identify a variety of survival and death estimates. Briefly, the tables include for each age:

  • Probability of death in the next year
  • Probability of surviving to the age
  • The life expectancy

There is more in the tables, but these are the big items that people often want to know. All of the various table columns can be calculated from survival rates. The US government and the UN each has created many such tables for a variety of time, locations, and development details. For example, the earliest and most dependable one is from 1901 and includes separate tables by race, sex, migrant status, urbanity, and even for some specific states.

Continue reading

Confronting my Macroeconomics Professor

I’m gearing up to teach macroeconomics for the first time. The following is a story that I will keep in mind as I work to make technical material relevant to undergraduates.

Years ago, I was an undergraduate sitting in a macroeconomics class. As it happened, I was in an intermediate-level macro class with no relevant background or context for the material. (If I had taken principles-level econ, then maybe I wouldn’t have been in this situation.)

My instructor was grinding through theory in a methodical way. By the end of the first month, as I remember it, we had covered the short run and the medium-term effects of monetary policy.

For anyone who is not familiar, see these MRU videos on shifting the aggregate supply curve.

The Short-Run Aggregate Supply Curve

Office Hours: Using the AD-AS Model

In summary, the government can inject money into the economy to achieve a short-term increase in output. For a short amount of time, you can help, and that seemed good to me. I had signed up for the course to understand how to reduce poverty and make the world better. I was acing the exams. Things were going well at first.

Then we got bad news. Increasing the money supply does not work for long. Consumers realize that everything is more expensive, so they cut back on real spending. The economy shifts back to where it was before. Nothing actually improves. I had spent a month of my life on this class and we were getting nowhere.

After the lecture on returning to the long-run aggregate supply curve, I went up to the professor after class. I asked him what was going on and when would we learn something that matters. (I was polite. I realized I was going to sound dumb to him, but life is short. I needed to know if this class was going to deliver anything.)

He looked at me, surely confused that I was unsatisfied with the standard progression of material in his course. Then he explained, “Oh. You are talking about the long term, and we will get to that next month.” That’s what I needed. I did not drop the course or the major. I’m an economics professor today because I didn’t mind looking like an idiot if I could get my questions answered.

This story helps me remember what it was like to be an undergrad in an economics class. Tyler says “context is that which is scarce.” Economics teachers need to do two things at once: present technical material and provide context. I will try to get that mix right going forward.

Note to students: Students, don’t be afraid to ask stupid questions. This is your chance. A good teacher will be glad you took the initiative. However, if the question occurs to you right in the middle of a lecture, then it may or may not be the appropriate time for the lecturer to stop and have a conversation with you. Teachers will be most amenable to having a deep conversation after class or during office hours.

My macro-related research:

Published paper: “If Wages Fell During a RecessionYouTube video presentation of this paper from minute 19:00-32:00.

Working paper (no draft yet): “Sticky Prices as Coordination Failure: An Experimental Investigation”

5 Game Theory Course Changes

I want to share some changes that I’ll make to my game theory course, just for the record. It’s an intense course for students. They complete homeworks, midterm exams, they present scholarly articles to the class, and they write and present a term paper that includes many parts. Students have the potential to learn a huge amount, including those more intangible communication skills for which firms pine.

There is a great deal of freedom in the course. Students model circumstances that they choose for the homeworks, and they write the paper on a topic that they choose. The 2nd half of the course is mathematically intensive. When I’ve got a great batch of students, they achieve amazing things. They build models, they ask questions, they work together. BUT, when the students are academically below average, the course much less fun (for them and me). We spend way more time on math and way less time on the theory and why the math works or on the applicable circumstances. All of that time spent and they still can’t perform on the mathematical assignments. To boot, their analytical production suffers because of all that low marginal product time invested in math. It’s a frustrating experience for them, for me, and for the students who are capable of more.

This year, I’m making a few changes that I want to share.

  1. Minimal Understanding Quizzes: All students must complete a weekly quiz for no credit and earn beyond a threshold score in order to proceed to the homework and exams. I’m hoping to stop the coasters from getting ‘too far’ in the course without getting the basics down well enough. The quizzes must strike the balance of being hard enough that students must know the content, and easy enough that they don’t resent the requirement.
Continue reading

5 Easy Steps to Improve Your Course Evals.

Incentives matter. I’ve taught at both public and private universities, and students have given me both great course evaluations and less great student evaluations. The private university cared a lot more about them. Obviously, some parts of student evaluations of their instructors are beyond the instructor’s control. The instructor can’t control inalienables and may not be able to change their charisma. But what about the things that instructors can control? Regardless of your current evals, here are 5 policies that are guaranteed to improve your course evaluations.

1: Very Clear Expectations/Schedule

Have all deadlines determined by the time that the semester starts. Students are busy people and they appreciate the ability to optimally plan their time. Relatedly, students desire respect from their instructor. Having clear rubrics and deadlines helps students know your expectations and how to meet them – or at least understand how they failed to meet them. Students want to feel like they were told the rules of the game ahead of time. This means no arbitrary deductions or deadlines. The syllabus is a contract if you treat it like one.

2: Mid-Semester Evaluations

One of the absolute best ways to improve your evaluation is to ask your evaluators for a performance update. Make a copy of your end-of-semester course evaluation and issue it about halfway through the semester. Then, summarize the feedback and review it with your class. This achieves three goals. (1) It is an opportunity to clarify policy if there are misplaced complaints. You may also wish to explain why policy is what it is. Knowing a good reason makes students more amenable to policies that they otherwise don’t prefer.  (2) It provides voice to students who have things to say. Often, students want to be heard and acknowledged. It’s better that a student vents during the informal mid-semester survey than on the important one at the conclusion of the course. (3) If there are widespread issues with your course, then make changes. If you’re on the fence about something, then take a poll. And if you decide to make changes, then be graciously upfront about it. Unexplained or covert changes violate policy #1.

Continue reading

Supply & Demand, with Tables?

When I was a graduate student, I paid for my tuition by tutoring for the university athletics department. I tutored stat, math, micro, macro, excel, and finance. I tutored the same students each week, so I got to know them pretty well over the course of the semester. I also got to know their strengths and weaknesses. It was at this time that I realized most quantitative or even analytical ideas could be described in 4 potentially equivalent ways:

  1. Mathematically
  2. Using logic in English
  3. Graphically
  4. With a Table

In this post I want to share the Supply & Demand cheat-sheet that I use to help my students learn about the effects of supply and demand.

Continue reading