Excellent Discussion Of The SAT

The article I'm citing if by Charles Murray, a co-author of the notorious and controversial Bell Curve. This doesn't mean that it isn't a worthwhile discussion and it is replete with interesting facts.

The Little Debate: SATs vs. Subject Matter Tests

The first part of a careful analysis of the SAT college admission tests sticks to a relatively boring conclusion: Given high school grades and SAT subject matter tests, you can predict college success virtually as well as you can with high school grades, the SAT, and the SAT subject matter tests. The SAT and the SAT subject matter tests are highly corollated, and the subject matter tests are slightly better.

Specifically, a University of California study found that:

Achievement tests [ed. the subject matter tests] did slightly better than the SAT in predicting freshman grades. High school grade point average, SAT scores, and achievement test scores were entered into a statistical equation to predict the grade point that applicants achieved during their freshman year in college. The researchers found that achievement tests and high school grade point each had about the same independent role—that is, each factor was, by itself, an equally accurate predictor of how a student will do as a college freshman.

But the SAT’s independent role in predicting freshman grade point turned out to be so small that knowing the SAT score added next to nothing to an admissions officer’s ability to forecast how an applicant will do in college—the reason to give the test in the first place. In technical terms, adding the SAT to the other two elements added just one-tenth of a percentage point to the percentage of variance in freshman grades explained by high school grade point and the achievement tests. . . .

For schools in the bottom quintile of the ratings . . . the achievement tests did slightly better than the SAT in predicting how the test-takers would perform as college freshmen. . . . after controlling for parental income and education, the independent role of the SAT in predicting freshman grade point disappeared altogether. The effectiveness of high school grade point and of achievement tests to predict freshman grade point was undiminished. . . .

[The study] broke down its results by college campus (an A at Berkeley might not mean the same thing as an A at Santa Cruz) and by freshman major (an A in a humanities course might not mean the same thing as an A in a physical science course). The results were unaffected. Again, the SAT was unnecessary; it added nothing to the forecasts provided by high school grades and achievement tests. . . .

[T]he correlation between the SAT Verbal and the Literature Achievement test was a very high 0.83 (a correlation of 1.0 represents a perfect direct relationship). The correlation between the SAT Math and the Math IC achievement test was 0.86.

Big deal, one set of tests by a company is slightly better than another. Of course, an ordinary SAT and high school GPA are almost as good as SAT subject matter tests and high school GPA. Pick your poison. In his view, he favors the subject matter tests primarily because "[T]he image of the SAT has done a 180-degree turn. No longer seen as a compensating resource for the unprivileged, it has become a corrosive symbol of privilege."

Bigger points

1. Test coaching is only modestly effective:

From 1981 to 1990, three separate analyses of all the prior studies were published in peer-reviewed journals. They found a coaching effect of 9 to 25 points on the SAT Verbal and of 15 to 25 points on the SAT Math. In 2004, Derek Briggs, using the National Education Longitudinal Study of 1988, found effects of 3 to 20 points for the SAT Verbal and 10 to 28 points for the SAT Math. Donald Powers and Donald Rock, using a nationally representative sample of students who took the SAT after its revisions in the mid-1990s, found an average coaching effect of 6 to 12 points on the SAT Verbal and 13 to 18 points on the SAT Math.

2. The SAT and related tests are not ethnically biased:

To determine whether a test is biased, just compare its predictive validity for different groups. This has been done for the SAT in multiple studies over the decades, and the results have shown that the SAT predicts college performance as well for poor test-takers as for rich test-takers, as well for ethnic minorities as for whites, and as well for women as for men. The caveat to this conclusion is a tendency for the SAT to overpredict, not underpredict, the college performance of African Americans. On average, it indicates they will do better than they actually do.

3. Smart people have smart kids (emphasis added):

Extrapolating from the 2006 data on means and standard deviations reported by the College Board, about half of the 700+ scores went to students from families making more than $100,000 per year. But the truly consequential statistics are these: Approximately 90 percent of the students with 700+ scores had at least one parent with a college degree. Over half had a parent with a graduate degree. . . . [T]he test isn’t the problem. The children of the well educated and affluent get most of the top scores because they constitute most of the smartest kids. They are smart because their parents are smart. The parents have passed their smartness along through parenting practices that are largely independent of education and affluence, and through genes that are completely independent of them.

4. Meritocracy has culled dumb money and the underprivileged from elite colleges:

Harvard offers an easy way to summarize the revolution that accelerated after World War II. As late as 1952, the mean SAT Verbal score of the incoming freshman class was just 583. By 1960, the mean had jumped to 678. In eight years, Harvard transformed itself from a college with a moderately talented student body to a place where the average freshman was intellectually in the top fraction of 1 percent of the national population. But this change did not mean that Harvard became more socioeconomically diverse. On the contrary, it became more homogeneous. In the old days, Harvard had admitted a substantial number of Boston students from modest backgrounds who commuted to classes, and also a substantial number of rich students with average intelligence. In the new era, when Harvard’s students were much more rigorously screened for intellectual ability, the numbers of students from the very top and bottom of the socioeconomic ladder were reduced, and the proportion coming from upper-middle-class backgrounds increased.

The other high-ranking schools have similar stories to tell. In a sample of 11 of the most prestigious colleges studied by William Bowen and his colleagues between the mid-1970s and the mid-1990s, the proportion of students in the top SES quartile rose from about a third to a half of all students, while the share in the bottom quartile remained constant at one-tenth. And these were schools such as Princeton and Yale that get first chance to admit the scarce and sought-after candidates of high ability from poor backgrounds.

When, in 2003, Anthony Carnevale and Stephen Rose expanded the definition of top-tier colleges to include 146 schools, fully 74 percent of the students came from families in the top SES quartile, while only 3 percent came from the bottom quartile. Ethnic diversity has increased during the last half century, but not socioeconomic diversity.

Analysis

He concludes that college admissions are not driving the growing divide between the haves and have nots in our society by being unmeritocratic:

From 1907 to 2007, the correlation between intellectual ability and socioeconomic status (SES) increased dramatically. The socioeconomic elite and the cognitive elite are increasingly one. . . . Because upper-middle-class families produce most of the smartest kids, there is no way to reform the system (short of disregarding intellectual ability altogether) to prevent their children from coming out on top. We can only make sure that high-ability students from disadvantaged backgrounds realize that the nation’s best colleges yearn for their applications and that their chance of breaking out of their disadvantaged situations has never been better—in short, that the system is not rigged.

He doesn't take the next logical step to ask if it is possible to solve the gaps by improving schools. But, if he had, tests like the CSAPs would show that profound and persistent ethnic and income based achievement gaps in standardized tests are firmly in place by 3rd grade (the link starts the analysis at 4th grade test results, but the gaps are present in 3rd grade tests to a similar degree) and persist at about the same magnitude through the tenth grade, across all subjects, despite significant institutional efforts to close those gaps.

The public schools in Colorado aren't materially aggravating the achievement gaps that are there in 3rd grade, but the public schools also have only the slightest impact in closing those achievement gaps.

The data show persausively that no modest effort to reform K-12 education, or make college admissions more meritocratic, will make much difference in social stratification. There are essentially three possible ways to reduce social stratification which aren't precluded by what we know:

1. Very early intervention (well before age 9) designed to improve academic achievement.

2. Radical and intense overhaul of how K-12 education is run that puts intense efforts into helping kids who have fallen behind catch up and learn the academic skills that upper middle class kids pick up by osmosis.

3. A departure from an academically oriented meritocratic ideal.