An explanation is considered in the linked post at Marginal Revolution. The gist is that lower student loan amounts are incurred by students who are academically less able and who tend to drop out before receiving the credentials that they spent money trying to earn. But, other specific situations that contribute to the situation, like unemployable recently released felons who enroll in community college to comply with parole requirements because they kind find jobs are also interesting.
In previous posts, I've also noted the dismal situation of students running up non-dischargeable student loan debts at for profit colleges. For example:
While for-profits educate less than 10 percent of students, those colleges' students received close to a quarter of Pell Grant and federal-student-loan dollars in 2008, according to the College Board. And they accounted for 44 percent of defaults among borrowers who entered repayment in 2007.The for profit Western International University is typical of this problem.
The bottom line policy conclusion that this and other data leads me to is that indiscriminate higher education subsidies coupled with open or not very selective admissions policies waste a great deal of scarce higher education resources.
Sending people who aren't academically ready to do college level work only to have them drop out, often in relatively short order, may create jobs for underpaid adjunct professors teaching introductory and remedial classes, but does little to built human capital in these students. Not only do they overwhelmingly drop out early and end up paying for efforts to obtain degrees that they don't attain; they also receive less value added from the time they spend in college, whether or not they graduate.
There are distributive justice issues with a purely merit basis for funding higher education as well, however. It turns out that highly academically able students, particularly those at selective institutions of higher education (public and private alike) tend to be very affluent, often coming from families in the top 5-10% of household income. Taxing less affluent people in order to send the subsidize the college educations of children of affluent people who can afford to forego state higher education subsidies, may have "trickle down" benefits to society, but isn't obviously good policy either. The fact that many resident graduates of state subsidized public colleges leave their home states when they graduate dilutes these benefits further.
Now, this isn't to say that I am against public funding for higher education. Indeed, few opportunities for dramatic economic benefits from public spending are more clear. Opening access to higher education for previously excluded people with academic ability in the civil rights era contributed substantially to U.S. economic growth. Low income students with strong academic ability are less likely to attend and to finish college than high income students with weak academic ability, and this is almost entirely due to insufficient grant based financial aid for these students.
Therefore, rather than subsidizing the higher education expenses of all state residents indiscriminately, we should instead subsidize higher education (as opposed to university based research not intended to have direct educational benefit to college students), almost entirely with scholarships that require both merit and financial need. Ideally, every student with academic merit should receive a scholarship equal to their full financial need determined in an accurate way.
In practice, there is uncertainty and inaccuracy in both academic merit determinations, and in financial need determinations.
On the academic need side, the answer is probably to provide full scholarships to those with clear academic merit, partial scholarships to those whose academic ability is probably sufficient to benefit but marginal, and to deny scholarships to those who lack academic merit. Perhaps students with at least a two-thirds probability of graduating given their grades and test scores, who have no remedial course requirements would get full scholarships, students with a 50% to two-thirds chance of graduating and no more than one remedial course to take would get a two-third grant, one-third loan package, students with a one-third to 50% change of graduating or more than one remedial course to take would get a one-third grant, two-thirds loan package, and everyone else would qualify for aid only for sub-collegiate level continuing education rather than college degree programs.
On the financial need side, the answer is probably to err on the side of being generous, recognizing that the harm of underfunding a student who is therefore forced to drop out or not attend college in the first place is greater than the harm associated with rewarding academically able low to moderate income kids by providing them scholarships that are a bit more generous than they strictly need to get by. Creating incentives can be just as worthwhile as meeting true economic needs.