November 22, 2004Red/Blue ACTThe guys at Powerline are engaged in a takedown of Jim Holt's denigration of the Red States. It includes the following statement: Next, Holt says that the states vary in the "value they place on education." That's certainly true. The effect can be measured in a number of ways. One of the most basic is to look at high school students' test scores. Look here, for example, where average ACT scores are compiled by state. As of 2004, the average ACT score in America was 20.9. Presumably if the blue states are far ahead of the reds in their concern for education, the red states should be clustered well below the average. Right, Mr. Holt? Sure. Well, let's check the actual data. Among the red states showing better than average achievement for their high school students are Alaska, Arizona, Idaho, Indiana, Iowa, Kansas, Missouri, Montana, Nebraska, North Dakota, Ohio, South Dakota, Utah, Virginia, and Wyoming. So one half of the states that voted for President Bush have school systems turning out above-average scholastic achievement tests. Unfortunately, the data does seem to indicate a difference in ACT scores among those tested. Looking at the data more closely does seem to indicate the dangers of eyeballing patterns, and seeing if the data "looks" random. Warning! Serious Statistical Geekery Ahead! If you want the conclusions, skip to the last paragraph. I ran three different statistical tests. The first was a simple correlation between the ACT scores and Red/Blue status, signified by a 1 (red) or a 0 (blue):
For a sample size of 51, a correlation of 0.14 (or -0.14) would be considered significant, as a rule of thumb. All of these are significant. Next, I used something call the Mann-Whitney U-Test. Bsaically, you rank all the scores, assigning 1 to the lowest, all the way to 51 for the highest. Ties are averaged. The test statistic is a z. In this case, the higher score, the stronger the chance that the two sets of rankings are different.
These are huge z-scores. The Messiah comes at z = 4.0. We can state with near-certainty that the ACT scores of red states rank lower than those of blue states. Finally, I directly compared the average ACTs. This is a two-part test. The first part showed that the variances of the two samples could be treated as the same. That made the second part a simple t-test. Since there are 31 red states, and 20 blue ones (including DC), there were 19 degrees of freedom. The 90% confidence threshhold for 19 degrees of freedom is 1.33:
All of these are significant, too, with the base exception of the math scores. Here's my Excel file. So what? Well, we can say that the ACT scores in red states are almost certainly lower than those in blue states. Since some states test only a few percent of students using the ACT, it's hard to tell how that affects scores. It's possible that some states' scores are dragged down by broader participation. It's also possible that some states have such low participation that their ACT scores aren't representative of their school systems. A cursory correlation suggests that about 30% of the scores' declines are explained by higher participation, and that red states are more likely to have higher participation rates, with a correlation of 0.52. They may have stronger teachers' unions, stronger bureaucracies, fewer private schools, higher SAT participation. In any case, there's nothing in the data presented to indicate that Holt's sneering thesis about red-staters caring less about education is true. Posted by joshuasharf at November 22, 2004 11:14 PM | TrackBack |
|