For all the fuss over whether schools provide actual skills or merely signal underlying ability, I think we may underappreciate the consquences of internalizing education within employment tournaments. “Neurotic parents worry for their kid’s future if they don’t get into the best pre-school” is a fairly trite sitcom premise at this point, but like a lot of tropes it bears enough truth to carry an episode. Parents “red-shirting” their kids to give them a competitive advantage in both academics and sports has received plenty of attention. Researchers have observed that birthdays early in the calendar year are overly predictive of selection into hockey at the advanced amateur and professional levels. These are all products of, and reactions to, the long and grueling tournaments that portend to identify talent.
We are rightfully obsessed with how to best identify talent at the micro level, but I spend more of my time thinking about the broader consequences that emerge from how we sort talent. More specificly, when and how we sort talent out. The filters.
The above is two academics arguing, both quite reasonably, about how a department should hire faculty. What I want to focus on is that last line of the second tweet:
“are you arguing we should hire.. (people) who weren’t competitive for higher-tier PhD programs?”
And there it is. The filter. If you weren’t good enough to get into a good PhD program at 22-25 years old, why should we waste our already thin resources considering you for position when we have plenty of candidates who did get into a higher-tier program? That’s a completely reasonable argument. The candidates at the best programs are, on average, better than those from less prestigious academic pedigrees.
What I want to do now is persuade you that this is a deeply flawed strategy. To overweight entry into your pool of consideration based on these early filters is bad for your academic department, company, or your hockey team. It will not only cause you to miss out on whole swaths of talent, but will have long run consequences upstream as well, as more and more resources will be wasted within an increasingly desperate all-pay auction to avoid being filtered out. In fact, those resources will be worse than wasted, as growing resource demands to survive the filter will result in both a shrinking and homogenizing final talent pool, condemning departments and disciplines to the stultifying malaise of sameness.
(From here on I’m going to overly lean on an academic framing here becuase it’s what I know best, but I think the story and argument are broadly applicable.)
Surviving the academic filter
I myself am part of a generation of tenured academic economists that can’t help but quietly wonder if we would have made it to the other side of what is now less a path to an academic career and more a post-baccalaureate Hunger Games. Two years predoc, six years of graduate school, three years postdoc, six years on the tenure clock at your first job, a-hopefully-shortened-to-four-year clock at your second job. That’s a 17 to 21 years of often brutal competition for publications at journals and grants from institutions with <5% acceptance rates. But now let’s walk backwards through that path.
Getting published in top outlets and getting funding for your research is a lot easier given the resources available to junior faculty at elite research institutions, appointments which are almost exclusively available to graduates of elite PhD programs. Even if one fails to receive an appointment or receive tenure at a top research institution, the networks you build in a top graduate program will bear fruit your entire career. You will get invited to present in seminars, join the right clubs, and disseminate your work through the best working papers series.
So how do you get into a good graduate program? I haven’t inspected the numbers personally, but I have been told by colleagues that the vast majority of students at top programs have at least near-perfect GRE scores and GPAs. These standard performance metrics left insufficient for final decisions, what’s left is the the prestige of your institution, extraordinary non-traditional undergraduate performance (i.e. writing an outstanding senior thesis), or procuring a prestigious post-baccalaurare research experience (a predoc). The net of this is that when you weight faculty appointmentss disproproportionately towards the prestige of a graduate academic program, you are indirectly weighting your decisions towards the selective criteria of undergraduate admisisions and a young person’s ability to independently network with faculty. You are effectively filtering out candidates based on attributes they failed to display when they were teenagers.
The trap of failed forecasting and early filters
When hiring and admitting talent, you are playing a statistical game. You’re not just trying to maximize the expected value (the mean), you’re also pursuring optimal variance, specifically upside variation. An NBA team would rather draft 1 hall of famer, 2 all-stars, and 7 busts that never play a minute than 10 players with average NBA careers. Pursuing upside and the upper tail, pursuing outliers, has a strange alchemy to it. To say that outliers and the rest of extreme upper tail is less understood than average outcomes is almost tautalogical, but in this case it’s important. To pursue upside variance in recruitment is, at least partly, an effort to forecast that upper tail. Forecasting these people is not only hard to do, it’s hard even to tell ex post if someone has been successful at it for reasons attributable to skill or merely luck.
When we filter out candidates based on where they went to graduate school, we are indirectly saying we believe that undergraduate admissions can forecast outcomes of considerable rarity. That’s quite the leap of faith.
I don’t get the impression that venture capitalists are particularly concerned with the prestige of their entrepreneur’s undergraduate education. Some people might use this as a cudgel to demean the value of a college education or an opportunity to extoll the greatness of genius that refuses to be suffocated by a structured education, blah blah blah. You will be shocked to learn that I think both of these strawmen arguments I’ve created are entirely silly. I think venture capitalists place so little weight on educational prestige because they are looking for extreme outliers in dimensions that are not necessarily orthogonal to education, but are simply unforecastable at any sort of scale. They can’t be predicted, only observed retrospectively in evidence of successfull entrepreneurship.
I put it to you, humble department hiring committee, that attributes that make for a high quality researcher are similarly difficult to forecast early in life. Further, the skills that make for great student, I am sorry to say, do not correspond all that well to successful research. While perhaps not as extreme as professional basketball players or billionaire entrepreneurs, top researchers are in many ways outliers. They are strange amalgams of, yes, intelligence, but also conscientousness and rebelliousness. They are creative and pragmatic, capable of working with other outliers and managing teams while also tolerating long periods of loneliness and even professional derision. These combinations of characteristics are not necessarily special, but they certainly are unusual.
Mechanisms that maximize the mean of your recruited talent will not maximize the upper bound on the pool you are choosing from. It will not maximize your upside outliers. Maxmizing the mean, however, is exactly what undergraduate admissions are largely doing. They are maximizing the SAT scores and GPAs of their incoming class and, in doing so, they are maximizing expected 5-year graduatiuon rates, post-graduation employment, medical school admissions, etc. They are maximizing the quality of each admission cohort. They are maximizing the mean and, in the process, minimizing the variance of those cohorts.
When you are recruiting talent to your research faculty you are looking for upside variance, the exact statistical characteristic that undergraduate admissions are often minimizing. When you overweight your hiring decisions based on whether candidates were competitive when applying to graduate school, you are likely increasing the mean of your pool of consideration but also reducing it’s variance. You are indirectly filtering your candidates through a chain of graduate and undergraduate admissions criteria that are minimizing upside variance.
Now, to be clear, this may be exactly what you want. If you’re recruiting to a job or field that is about minimizing downside risk, then this may be the optimal strategy. I’d be thrilled to learn that the mechanism that produced primary care physicians was explicitly designed to maximized the mean and minimize the variance of physician quality. But for a research department recruiting junior faculty (or a basketball team, or entrepreneurial tech incubator,…), downside risk is fairly minimial while upside potential remains enormous.
The cost of early filters
Early sub-optimal talent filters are costly in two ways. First, they filter out talent before its value can be observed. Second, they incentivize individuals and households to commit resources to wasteful tournaments.
The first, micro, costs of early filters are pretty obvious: we miss out on the contributions of potential stars because of decisions overweighted towards criteria that reflected mean cohort maxmization, an attribute at least orthogonal, if not explicitly counter to, the qualities you are recruiting. The second, more macro, costs should be considered as well.
If an individual identifies a career they want, whether its professor, medical doctor, or professional athlete, and observes institutions that filter people out of consideration, they will invest resources to survive the filter. If slots that survive the filter are a fixed quantity, it will quickly devolve into a tournment, where surivival will depend, at least in part, on the resources an individual is willing to commit. These resources are largely unrecoverable. This is an all-pay auction. It’s a trap where social welfare goes to die.
This is a trap in which diversity dies, too. Social and economic diversity for sure, but also diversity of ideas, perspective, patterns of though, and intuition. The fraction of households that can commit significant resources to gain admission to top secondary and undergraduate institutions, that can endure the opportunity cost of unpaid internships and underpaid “predocs” is not representative of the broader population. It is wealthier, Whiter, and far more educated. As Schultz and Stansbury note, 65% of U.S.-born economics PhDs had at least one parent with a graduate degree, while only 14% of those PhDs were first-generation college graduates.
Every academic scholar knows the pressure to conform to trends in models, theories, hypotheses, sometimes even explicit policy prescriptions. This pressure comes in the form of editorial, hiring, and promotion decisions. This pressure is only further augmented by the forces that filter talent into these positions. Filters that may, in fact, increase the how smart and diligent we are on average, but in doing so sand off the fat tails of our distribution of colleagues. The very people perhaps most likely to think of something completely different. The people most likely to change our minds.
To lose them for the possibility of being a little smarter on average? That is costly indeed.
Economist Writing Every Day 
I read this article with two teenagers, so I’m amused(?) by the comparison of academia with sports. Kids and parents are cautioned against putting all their dreams in being/raising the next Lebron, but you make the tenure track seem worse?
There are positive trends in undergraduate admissions moving away from standardized test scores toward “whole applicant” screening (normalizing a student’s performance based on their high school’s ranking, for example.) One hopes that idea will percolate up to grad programs as well. But unlike basketball, research skills are hard to teach, practice, and test without an actual research program behind it. So it will always be a rich person’s game, until high school academic programs are given as much resources and coaching talent as athletic programs are.
LikeLike