The ACT predicts college grades much better when only the English and math subsections are used, and the remaining sections are omitted, than it does when all of the sections are used.
The reading and science subsections are so bad at predicting college grades that they simply add noise to the ACT composite figure, making it a less accurate measure of future college grades.
In relative terms, here is how predictive each subsection of the ACT is:
Math (significant at p=0.01 level) 26
English (significant at p=0.01 level) 16
Reading (significant at p=0.05 level) 3
Science (not statistically significant) -1
Yes, that's right, a higher science section score, controlling for a variety of other variables (college campus attended, high school GPA, race, gender, and college major) actually predicts worse, rather than better, college grades.
This means that simply by dropping two sections, the multiple choice sections of the ACT could be administered in half the time, while making the test as a whole a significantly more accurate predictor of future college grades.
The May 2013 study cited summarizes its methods and conclusions as follows:
We test for differences in predictive power among the four subscores using Ohio Board of Regents data on all students that matriculated to a four-year public college in Ohio in 1999. Along with demographic information about each student, the data contain important college performance measures such as grade point average (GPA) and indicators for dropping out.
Not surprisingly and confirming a long line of literature, we find a strong correlation between higher ACT composite scores and positive college outcomes. However, this overall correlation masks an important pattern: Mathematics and English scores are much more tightly correlated with college success than are Reading and Science scores. In fact, after controlling for Mathematics and English scores, Reading and Science provide essentially no predictive power regarding college outcomes.
The difference in the predictive ability for Mathematics and English versus Reading and Science scores is consistent across different specifications and data subsamples. The finding is robust when controlling for indicators of the college that students attend, high school performance, student demographics, and college major. The results are also consistent across a wide variety of college outcomes including GPA for the first and second years and dropout rates for the first and third years. The results are very similar across different universities of varied quality.
We also find that Mathematics and English scores are far better predictors than Reading and Science scores of high school GPA. This provides further evidence that the Reading and Science tests have very little predictive merit. Finally, we replicate our results using a smaller independent data source from a private university in the western United States.
Fascinating. Also, the Science score seems pretty straightforward. Good science score kids are much more likely to major in science where there is much less grade inflation.
ReplyDelete@Steve Greene. Given that the study controls for both college campus attended and college major, that explanation for the science score doesn't work. It could be that all actually useful aptitude in science is captured by the math school, and that the residual impact of the science score after controlling for math is negligible, but that math is more predictive of first years grades because it captures skills useful not only in science (where it is duplicative of the science score) but also in non-science fields where it provides a benefit that science knowledge does not.
ReplyDeleteWell, that's what I get for not reading the excerpt carefully. That does sound like a plausible explanation.
ReplyDelete