19 January 2011

Learning v. Sorting in Higher Education

No one denies that people who go to college do far better socio-economically than people who do not. The hard question is why that is the case. Is it because what you learn adds value? Or, is it because the people who go to college are smarter and more together than their non-college bound peers on average, and hence more desirable in the world of work?

Prior Indications That Higher Education May Not Be Adding Much Value For Marginal Students

One little factoid that has favored the sorting theory over the learning theory is that the socio-economic benefits of being admitted to college and dropping out are almost indistinguishable from the socio-economic benefits of obtaining an associates degree (i.e. a two year college degree, typically in skill heavy pre-professional field like nursing or engineering technology). An associates degree provides some benefit relative to merely attending and dropping out of college without earning a degree (and interestingly, a huge advantage as measured through likelihood of ending up in prison), but the benefit is stunningly modest.

Another suggestive nugget of data is that the socio-economic benefits of college appear to arise despite the fact that many college graduates work in jobs that don't actually require a college education to perform: "More than one-third of current working graduates are in jobs that do not require a degree, and the proportion appears to be rising rapidly."

The push to increase enrollments has led to a majority of the increment of our stock of college graduates finding employment in relatively low skilled jobs, most of which are not particularly high paying (although there are exceptions). We added roughly 20 million college graduates to the population between 1992 and 2008, for example, but the number of graduates holding jobs requiring less-than-college education skill sets rose during that same period by about 12 million; in other words, 60 percent of the total increase in graduates over the past two decades was underemployed. . . . [This raises] real questions about the desirability of pushing to increase the proportion of Americans attending and graduating from four year colleges and universities.

From a study by Richard Vedder and co-authors on the topic.

A Large Share of College Students Aren't Learning Much In College

A new study adds more direct evidence to the case that a large share of the benefits of a college education for a large share of people who go to college comes from its sorting role, rather than as a result of what is learned while one is in college.

[A study] of more than 2,300 undergraduates . . . [from a representative sample of 24 colleges and universities described in] the book, "Academically Adrift: Limited Learning on College Campuses." . . . found an average-scoring student in fall 2005 scored 7 percentage points higher in the spring of 2007 on the [College Learning] assessment [which measures critical thinking, complex reasoning and writing]. In other words, those who entered college in the 50th percentile would rise to the equivalent of the 57th after their sophomore years. . . . After four years, 36 percent of students did not demonstrate significant improvement, compared with 45 percent after two [years].

Students who studied alone, read and wrote more, attended more selective schools and majored in traditional arts and sciences majors posted greater learning gains. . . .

Students who spent more time studying with peers showed diminishing growth, and students who spent more time in the Greek system had decreased rates of learning . . .

[A]ctivities such as working off campus, participating in campus clubs and volunteering did not affect learning.

Students from families with different levels of parental education enter college with different learning levels but learn at about the same rates while attending college. . . .

Black students improve their assessment scores at lower levels than whites.

The extent to which colleges add value or merely sort students matters a lot from a public policy perspective, because it is far cheaper to establish a system that ranks people by ability (for all involved) than it is to go through the motions of having people go to college for many years if they aren't going to learn much in the process.

Has Higher Education Expanded Too Much?

Put another way, the higher education system does seem to be imparting considerable value to the traditional students at selective institutions in arts and science majors taking rigorous courses and studying a lot who were the only people that higher education served in the pre-World War II era, and is doing so on a far more meritocratic and democratic basis than it did then when wealth, rather than academic ability, was the key to admission.

Since then, there has been a dramatic expansion of higher education: "in 1960, only 7.7% of American adults over the age of 25 possessed college degrees. This proportion has increased in every year since 1960 for which data is available, with the exception of both 1992 and 2005, and by 2008, 29.4% of Americans 25 years of age and older held college degrees." This has happened mostly in dramatically expanding state college systems with less selective admissions, much larger enrollments, and a large share of students taking pre-professional majors like marketing and communication seems to have added less value than one might of hoped, in terms of actual increasing ability as a result of the experience.

The Least Academically Able Students Who Dropping Out Most Learn The Least

The fact that learning is weakest in the least rigorous programs at the least selective institutions suggests that there is a heavy overlap between the students who are learning the least and those who are most likely to drop out of college.

The Associated Press report quoted above doesn't make clear (although perhaps the book it relies upon does), how the change in learning data relate to retention. All but the most selective institutions of higher education have high drop out rates. The most selective public colleges and universities in Colorado graduate two-thirds of their entering students, the least selective four year public colleges and universities graduate one-third of their entering students, and community colleges lose half to 70% of their entering students in the first year. In general, the less selective the institution, the higher the dropout rate. Retention rates are generally even worse for African-American students.

The odds that a study will drop out is very strongly correlated to the quality of that student's academic preparation and academic ability, and with traditional v. non-traditional student status. Students who drop out are very disproportionately those who are least academically able:

[F]ewer than 38 percent of high school students who plan to get a college degree actually do so within 10 years of graduating. Of those with poor high school grades, less than 14 percent achieve their college plans. . . . 92 percent of students with low grades planning to earn an A.A. failed to do so—even higher than the 86 percent of those who abandoned their plans to earn a BA.

Many students starting college these days are not very well qualified to do so. Many Colorado students who started college in the fall of 2008 need remedial work, especially at the community college level, and in math:

Of those starting two year college:
52.7% needed remedial help in math, reading or writing.
40% needed remedial help in math.
17% needed remedial help in math, reading and writing.

Of those starting four year college:
19% needed help in math, reading, or writing.
16% needed help in math

I don't think anyone would be surprised by a study showing that students who need remedial work going into college are less likely to graduate and less likely to learn much while they are in college.

Implications For Admissions

If the students who are prevented from going to college by more selective admissions requirements are students who are at extremely elevated risk of dropping out, and show a high likelihood of achieving no significant learning even if they do manage to graduate, then the social costs of tightening admissions standards (and thereby reducing higher education spending or making the package more generous for those who do attend) may not be as great as they seem.

But, people are not statistics. Selectivity is a good option only to the extent that we can accurately predict in the case of an individual who will learn and graudate, and who will not learn and not graduate. The fact that the retention rate at the most selective institutions in Colorado is still only two-thirds, while a third of students at the least selective institutions graduate suggest that the clear statistical trend may be far less clear at the level of the individual students. Even pretty decent students may have a pretty good chance of learning little and of dropping out, while pretty marginal students may have a not insubstantial change of learning a lot and graduating.

Certainly, there are a lot of students who start college who are doomed from the start. Their need for remedial work in multiple subjects, low high school grades, and poor college admissions test scores make clear that they have an extremely low chance of graduating with any kind of degree.

Mostly, the system does a pretty good job of steering these students to community colleges, where the per student cost of education is the lowest, and overwhelmingly, these students decide for themselves to drop out of college after one or two semesters of trying. Students who are clearly not academically ready and go to less selective four year colleges (which are the least expensive per student in the system), likewise overwhelmingly drop out in the first one or two years.

Indeed, these students may even be subsidizing the educations of students who go to college and do take their studies seriously. At the very least, they are imposing financial burden on the system that is far less than the raw number of unprepared students who enter college each year might imply.

The fact that transfer students from community colleges do just as well in four year colleges as their peers who started there suggests that the quality of the instruction at community colleges isn't materially different for those who can hack it.


The new value added learning report is also notable for being in tension with some prior findings in the field. Group studying has been linked to better academic performance in other studies. Also, social engagement has been fairly strongly linked to retention (i.e. socially unattached students are more likely to drop out), and Greek system participants have had grades comparable, if not better, to their non-Greek system peers.

These findings are suspect unless the contradictions in the findings can be better resolved.

Higher Ed Does Make A Difference For Millions And Can Be Improved

There is also a glass half-empty, or half-full element to these results. Significant contributions to learning in a majority of students after two years -- by seven percentiles of entering student performance -- and two-thirds of students after four years, is nothing to sniff at. The study states that for most students, and in particular, for two-thirds of students who manage to stick it out for four years and graduate, that college really is adding significant value in the form of increased learning. I suspect that by that measure, U.S. institutions of higher education would compare favorably to their peers in Europe and Japan, which are notorious for having rigorous college examinations at the end of high school, but mediocre teaching and a lack of rigor for undergraduates.

The link between rigorous academic classes and learning made by the study makes the case for more stringent curricular requirements at colleges and universities.

If we can predict in advance that curricular changes, like more rigorous expectations from professors in college courses, and admissions changes that make higher education as a whole more selective, can make the proportion of students who are learning and increasing their academic abilities in college greater, then we can take a system that is adding considerable value in a pretty wasteful way into one that is more consistently adding value.

Means Based Financial Support Is Worthwhile

This doesn't mean that there isn't a strong case for government funding of higher education. There is one. Students who can start college right after high school and are academically prepared learn the most and are most likely to graduate. But, a large share of lower income kids who are academically able aren't going to college and aren't graduating because they can't afford it. About one in five low income kids in the top quartile of academic ability don't go to college because they can't afford it. In the next quartile of academic ability, about a quarter of low income kids are prevented from going to college for financial reasons.

Ability to pay is a major barrier to the ability of academically talented low and medium income students to start or finish college:

High academic achievement kids in high income families have a 97% chance of going to college, while it is 78% for low income kids. For those in the second quartile of academic achievement kids in high income families have a 90% chance of going to college, while it is 63% for low income kids. For those in the third quartile of academic achievement 85% chance of going to college, while it is 50% for low income kids. For those in the bottom quartile of academic achievement kids in high income families have a 77% chance of going to college, while it is 36% for low income kids.

Similarly and unsurprisingly, more generous financial aid packages for low income students have been empirically demonstrated to significantly improve retention rates.

Footnote: Are Some Majors Bad Choices?

The study seems to indicate that there is less learning in majors outside the traditional arts and sciences.

I'd be curious to see how the learning rates differ in particular college majors, and also to see information beyond starting salary at graduation to validate the concept that some majors may add considerable value, while others may be a waste of time for their students.

There have been studies looking at the return on investment from different college majors such as this rather ill designed one, and those usually show great benefits from a college education (with the exception of culinary school). But, few of those studies are designed to capture the distinction between value added from the material studies and benefits largely attributable to sorting people by ability.

No comments: