Selectivity and Selection Bias: Are Selective Colleges Better?

If you have ever been through the process of applying to colleges, you have almost certainly heard the term “selective colleges.” If you haven’t the basic idea is that some colleges are harder to get into, for example as measured by what percentage of applicants are accepted to the school. The assumption of both applicants and schools is that a more selective college is “better” in some sense than a less selective college. But is it?

In a new working paper, Mountjoy and Hickman explore this question in great detail. The short version of their answer: selective colleges don’t seem to matter much, as measured by either completion rates or earnings in the labor market. That’s an interesting result in itself, but understanding how they get to this result is also interesting and an excellent example of how to do social science correctly.

Here’s the problem: when you just look at outcomes such as graduation rates or earnings, selective colleges seem to do better. But most college freshmen could immediately identify the problem with this result: that’s correlation, not causation (and importantly, they probably knew this before stepping onto a college campus). Students that go to more selective colleges have higher abilities, whether as measured by SAT scores or by other traits such as perseverance. It’s a classic selection bias problem. How much value is the college really adding?

Here’s how this paper addresses the problem: by only looking at students that apply to and are accepted to colleges with different selectivity levels, but some choose to go to the less selective colleges. What if we only compare this students (and of course, control for measurable differences in ability)?

Now this approach is not a perfect experiment. Students are not randomly assigned to different colleges. There is still some choice going on. But are the students who choose to attend a less selective college different in some way? The authors try to convince us in a number of ways that they are not really that different. Here’s one thing they point out: “nearly half of the students in our identifying sample choose a less selective college than their most selective option, suggesting this identifying variation is not merely an odd choice confined to a small faction of quirky students.”

Perhaps that alone doesn’t convince you, but let’s proceed for now to the results. This chart on post-college earnings nicely summarizes the results (see Figure 3 in the paper, which also has a very similar chart for completion rates)

Continue reading