26 January 2006

Religious Schools Teach Math Poorly.

In one of the most convincing challenges to conventional wisdom about school performance yet made a study has determined that:

Contrary to common wisdom, public schools score higher in math than private ones, when differences in student backgrounds are taken into account. . . .

The researchers looked at achievement and survey data from NAEP's [National Assessment of Educational Progress] 2003 national sample of 190,000 fourth-graders in 7,485 schools and 153,000 eighth-graders in 6,092 schools. The schools in the sample were categorized by NAEP as public (non-charter), charter and private, with the private schools broken down further by Catholic, Lutheran, conservative Christian and "other private."

NAEP is considered the only nationally representative ongoing assessment of U.S. academic achievement, and is often referred to as the "gold standard" of school performance data. . . .

Using a statistical analysis known as hierarchical linear modeling, the Lubienskis found that regular public schools scored "statistically significantly higher" than private and charter schools at the fourth-grade level. With 10 points roughly considered a grade-level difference in achievement, the regular public schools were trailed by 11.9 points by conservative Christian schools, 7.2 points by Catholic schools, 4.2 points by Lutheran schools, 5.6 points by all other private schools, and 4.4 points by charter schools.

At the eighth-grade level, the regular public schools were trailed by 10.6 points by conservative Christian schools and by 3.8 points by Catholic schools.

Lutheran and charter schools led regular public schools by 1.0 and 2.5 points, respectively, and all other private schools were 2.3 points below regular public schools, but all of these three gaps were determined to be statistically insignificant by the researchers.

To determine differences in students' backgrounds, the researchers used NAEP survey data related to the students' socioeconomic status, which included their eligibility for free or reduced lunch and their access to learning resources in the home, such as books and a computer. The researchers also incorporated survey data on students' race and ethnicity; gender; disability and limited English proficiency.

The more religious content a private school has, the more it impairs math learning. Secular private or charter schools (which are also, by necessity secular), and relatively mainline religious schools, which tend to have less religous content than conservative Christian or Catholic schools, teach math about as well as public schools.

The average student at a private school does better on standardized tests because they exclude poor students. This is the only reason for the difference. The full paper can be found here.


Elaine said...

Has anyone done a study of whether extremely religious folks are simply less intelligent than the mainstream? That would be another explanation of why students in schools with highly religious curriculums did worse.

Andrew Oh-Willeke said...

There have been a number of IQ studies (most involving college students) done that have concurred with your suggestion spread over a long period of time, many of which at points in time when IQ was viewed far less skeptically in the social science community and hence done with less attention to study design details that would have been carefully considered if they were done today.

The authors of the study did not suggest the theory you raise, but did rule out variables of geographic impacts either regional or rural v. urban, which would would expect to show fairly strong corollations with the theory you suggest.

However, similar assumptions (not necessarily specifically based in intelligence, so much as the overall impact of ethnicity and socioeconomic status on a person's school performance, which is known to exist markedly in our society for whatever reason by the time children reach 4th or 8th grades) are implicit in the model which incorporates race and ethnicity as factors to be controlled for in comparing academic performances of children at different schools.

Considering what one might generously call a poor rural White background as an ethnicity distinct from a middle class urban White background (even in the case of children of first generation urban immigrants still tied to rural culture), and finding that this ethnic distinction is statistically significant in predicting educational performances could very likely explain both the strong showing of Lutheran Schools (which are almost entirely outside of the area where "Southern culture" prevails) and the weak showing of Conservative Christian schools (which are disproportionately within that region), although it still wouldn't be very helpful in explaining the poor showing of Catholic schools, unless you further disaggregate the construct of a monolithic "white" ethnicity (with Northern European, Southern European and Anglo-Irish-Scottish components). But, if you go that route, then you also introduce several more degrees of freedom into the consideration without a lot of background in the literature showing a real difference in the real world that would justify the introduction of additional variables. Are the differences in the schools (for which there is a fair amount of literature support such as a strong connection between school type and the proportion of teachers who are certified) or due to intrawhite ethnic or religious affiliation differences (for which there is little support in the literature).

It would take particularly strong evidence, on the theory of extraordinary claims require extraordinary proof, for me to be comfortable with even seriously considering that conclusion.

One alternative theory suggested by the authors, along similar lines, was that it could be that many students sought out private schools based upon a lack of success in a public school program. To expand on that idea a little, if that lack of success was accompanied by discipline problems, many parents would see a school with a strong religious content as a particularly desirable alternative.