04 March 2016

Law School Grades, Law School Attrition, LSAT Scores And Bar Prep Classes

Law School Attrition and Median LSAT Scores

The median LSAT scores of entering 1L's at a law school are strong predictors of the likelihood that first year law school students at the school will drop out of that law school.

Law schools with median LSAT scores of 160 or more have drop rates of about 0.2%.  Law schools with median LSAT scores of under 150 drop out at rate of about 13%.  There are material differences in drop out rates in between for law schools with median LSAT scores of 150-154 (5%) and 155-159 (2%), respectively.

Keep in mind that these are only the percentage of students who drop out.  It doesn't include the large number who don't pass the bar exam, who pass (perhaps on a second or later try), but never find legal employment.

Note that since even the lowest performing law schools have drop out rates of less than 25%, the LSAT score at which there is a high risk of attrition for individual law students will typically be significantly below the 25th percentile LSAT score at these schools, not the median LSAT score.

I have previously estimated that an LSAT score of about 145 is close to the floor at which attending law school can make sense for an individual.

Law School Grades and Bar Preparation Course Participation Predict Bar Passage Rates

Bar preparation courses as part of the law school curriculum are very helpful to students who do poorly in their first year of law school, but provide no benefit to those who do well in their first year of law school.

A study from the University of Denver Law School looked at bar passage rates based upon first year law school grades (in four bins, under 2.6, 2.6 or more but under 2.9, 2.9 but under 3.5, and 3.5 or more), and participation in three school sponsored bar preparation oriented courses.

100% of 1L's who had first year law school grades of 3.5 or more passed the bar exam.

96.5% of 1L's who had first year law school grades of 2.9 or more (including those with 1L grades) passed the bar exam (about two-thirds of DU law students are in this category).

83.2% of 1L's who had first year law school grades of at least 2.6 but less than 2.9 passed the bar exam.

64.5% of 1L's who had first year law school grades of less than 2.6 passed the bar exam (about one-eighth of DU law students are in this category).

Participation in one or more of the law school sponsored bar exam preparation courses provided no material benefit to students with first year law school grades of 2.9 or more.  But, participation in these courses materially improved the bar passage rate for students with 1L grades between 2.6 and 2.9, and profoundly improved the bar passage rate for students with 1L grades under 2.6.

How important are undergraduate, 1L and overall law school grades and LSAT scores to bar passage rates?

Certainly, they matter a lot, but they don't provide a full explanation either.  As the cited study explains (emphasis added):
Researchers often using correlation tests to explore whether two variables are associated with each other. Using the Pearson ProductMoment Correlation test, we determined that LSAT, 1LGPA, and GLGPA [ed. i.e. law school GPA for all years] were all positively correlated to bar exam scores with statistical significance (LSAT: n=637, r=.444, p=.000; 1LGPA: n=591, r=.626, p=.000; GLGPA: n=642, r=.721, p=.000). 
LSAT shows a medium positive relationship, 1LGPA shows a large positive relationship, and GLGPA shows an even stronger positive relationship. 
In order to determine the amount of variation shared by the independent variables (LSAT, 1LGPA, and GLGPA) and the dependent variable (bar exam scores), we squared the value of r to obtain the coefficient of determination. We found that LSAT and bar exam scores share about 20% of variance (r2=.197), 1LGPA and bar exam scores share about 40% of variance (r2=.391), and GLGPA and bar exam scores share about 50% of variance (r2 =.519). In sum, traditional law school variables share a moderate to strong relationship with bar exam scores but still leave nearly 50% or more of bar exam scores explained by other variables. 
For reference with respect to admissions, the correlations between UGPA and LSAT are nil (n=591, r=.049, p=.218) because the relationship as visualized on a scatterplot is virtually horizontal and the low correlation is not statistically significant. 
In contrast, the correlation between LSAT and 1LGPA is statistically significant but weak (n=591, r=.341, p=.000). That means that the relationship is only explained by about 15% in shared variation (r2=.149). The correlation between UGPA and bar exam scores is statistically significant (n=631, r=.300, p=.000) but even weaker with a shared variation of only about 9% (r2=.09). Based on our data, it is very difficult to predict bar exam scores or even law school grades based on just LSAT and UGPA.
Also, while the study doesn't really spell it out, virtually all of the variance in actual bar passage rates, which is all that really matters, happens at the low end of the LSAT and grades scale.  Students with good grades and LSAT scores almost universally pass the bar exam, because it is not too hard of an exam for these students if they are prepared. And, nobody ever asks what score you got on the bar exam if you passed it, so their bar exam scores are irrelevant.

In contrast, while most students at the low end who are admitted to DU Law also pass the bar exam, while a significant number do not. And, this is where all of the variance in bar exam scores that matters occurs.

Indeed from an analytical perspective, if you are trying to get at the factors that matter for the students who aren't almost sure to pass the bar exam, a regression model that includes the two-thirds of students who are virtually shoe ins to pass the bar exam isn't very helpful.  Any relationship between other variables and the bar exam scores of these higher achieving law students is noise that could distort the signal regarding the factors that matter for low achieving law students.  For example, including high achieving law students in a regression model will understate the importance of bar preparation classes for bar passage in the case of less academically able law students.

No comments: