The Detroit Free Press recently ran an interesting series of stories about charter schools in Michigan which made me wonder if I could use regression analysis to partition the variation in test results on the basis of race, poverty, location, and type of school. This analysis uses MEAP scores for the 5th grade.

tl;dr — For profit charter schools generally produce worse results than public or not for profit charters while not for profit charters produce better results.

If you’re thinking “eeeeeeeeeehhhhhhhwwwwwwwww math”, I hope to soon incorporate the results into an on line map to make it easy to look at the results for particular schools .

If you are unfamiliar with interpreting the results of regression analysis, or would like a refresher, I suggest reading this description of interpreting the results from R.

## Reading

Coefficients: | |||||

Coefficient | StdError | t value | Pr(>|t|) | ||

(Intercept) | 557.78 | 6.83184 | 81.64 | 2E−16 | *** |

# of 5th Graders | −0.0058 | 0.00421 | −1.37 | 0.17032 | |

Ratio of Free Lunch | −29.376 | 1.09306 | −26.88 | 2E−16 | *** |

Ratio of Reduced Price Lunch | −7.8321 | 3.78783 | −2.07 | 0.03884 | * |

School Size | −0.0009 | 0.00105 | −0.87 | 0.38502 | |

Native Am. Ratio | −20.985 | 6.64949 | −3.16 | 0.00163 | ** |

Asian Ratio | −5.4472 | 5.75865 | −0.95 | 0.34435 | |

Black Ratio | −28.327 | 5.00546 | −5.66 | 1.8E−08 | *** |

Native Hawaiian Ratio | −55.509 | 46.7412 | −1.19 | 0.23519 | |

White Ratio | −20.324 | 5.00343 | −4.06 | 5.1E−05 | *** |

Hispanic Ratio | −24.364 | 5.29625 | −4.6 | 4.6E−06 | *** |

Female Ratio | 10.4082 | 2.36368 | 4.4 | 1.1E−05 | *** |

For Profit Charter | −0.9124 | 0.57287 | −1.59 | 0.11146 | |

Not For Profit Charter | 2.85784 | 1.30613 | 2.19 | 0.02883 | * |

Ratio of BA/BS in County | 7.55157 | 7.91992 | 0.95 | 0.3405 | |

Ratio of Grad Degree in County | 6.84643 | 8.00726 | 0.86 | 0.39268 | |

localeCity:Midsize | 1.9757 | 1.01248 | 1.95 | 0.05121 | . |

localeCity: Small | 2.61064 | 0.91296 | 2.86 | 0.0043 | ** |

localeRural:Distant | 2.8529 | 1.08825 | 2.62 | 0.00884 | ** |

localeRural:Fringe | 2.19042 | 1.03396 | 2.12 | 0.03431 | * |

localeRural:Remote | 4.49699 | 1.31645 | 3.42 | 0.00065 | *** |

localeSuburb:Large | 1.77036 | 0.89231 | 1.98 | 0.04744 | * |

localeSuburb:Midsize | 3.12932 | 1.33136 | 2.35 | 0.01888 | * |

localeSuburb:Small | 1.87596 | 1.20596 | 1.56 | 0.12003 | |

localeTown:Distant | 3.87374 | 1.23399 | 3.14 | 0.00173 | ** |

localeTown:Fringe | 2.43674 | 1.56102 | 1.56 | 0.11874 | |

localeTown:Remote | 4.33804 | 1.35007 | 3.21 | 0.00134 | ** |

Ratio of Math Takers | 5.06051 | 8.65669 | 0.58 | 0.55892 | |

Ratio of Reading Takers | −0.1406 | 7.46828 | −0.02 | 0.98498 | |

Ratio of Science Takers | 2.51456 | 6.4303 | 0.39 | 0.69582 |

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 6.46 on 1445 degrees of freedom

Multiple R-squared: 0.741, Adjusted R-squared: 0.736

F-statistic: 142 on 29 and 1445 DF, p-value: <2e-16

One thing to keep in mind about this data is that when it says a ratio, it means the total number divided by the other number and not a whole number. So the black ratio is the number of black 5th grade students divided by the total number of 5th graders. You can think of the coefficients as representing the change to the test score if all the participants are of that category. For example, a school that was 100% female would have a MEAP reading score 10 points higher.

The regression results suggest that charter schools deliver results that are different from the public schools. Not for profit charter schools add almost 3 points to the reading score. For profit charter schools reduce the score by almost a point (at lower threshold of statistical confidence).

The location variables show that almost anywhere that isn’t Detroit perform much better than schools in Detroit. Like the school type variable, the location variables are dummy variables. That means one variable is left out of the output and serves as a kind of baseline, which is why City: Large doesn’t appear in the output.

## Math

Coefficient | Std Error | t value | Pr(>|t|) | ||

(Intercept) | 527.372 | 10.3832 | 50.79 | 2E−16 | *** |

# of 5th Graders | −0.009 | 0.00639 | −1.41 | 0.1585 | |

Ratio of Free Lunch | −32.252 | 1.66126 | −19.41 | 2E−16 | *** |

Ratio of Reduced Price Lunch | −17.669 | 5.75684 | −3.07 | 0.0022 | ** |

School Size | 0.00067 | 0.0016 | 0.42 | 0.6736 | |

Native Am. Ratio | −1.9356 | 10.1061 | −0.19 | 0.8481 | |

Asian Ratio | 40.4056 | 8.75216 | 4.62 | 4.2E−06 | *** |

Black Ratio | −18.678 | 7.60744 | −2.46 | 0.0142 | * |

Native Hawaiian Ratio | 18.8099 | 71.0386 | 0.26 | 0.7912 | |

White Ratio | −7.6855 | 7.60434 | −1.01 | 0.3123 | |

Hispanic Ratio | −9.8284 | 8.04938 | −1.22 | 0.2223 | |

Female Ratio | 5.21048 | 3.59239 | 1.45 | 0.1472 | |

For Profit Charter | −1.2754 | 0.87066 | −1.46 | 0.1432 | |

Not For Profit Charter | 2.89958 | 1.98509 | 1.46 | 0.1443 | |

Ratio of BA/BS in County | −10.772 | 12.0369 | −0.89 | 0.371 | |

Ratio of Grad Degree in County | 28.963 | 12.1697 | 2.38 | 0.0174 | * |

localeCity:Midsize | 1.37839 | 1.5388 | 0.9 | 0.3705 | |

localeCity: Small | 2.77564 | 1.38754 | 2 | 0.0456 | * |

localeRural:Distant | −1.0587 | 1.65395 | −0.64 | 0.5222 | |

localeRural:Fringe | 0.42601 | 1.57145 | 0.27 | 0.7864 | |

localeRural:Remote | −0.1376 | 2.00077 | −0.07 | 0.9452 | |

localeSuburb:Large | 1.4715 | 1.35615 | 1.09 | 0.2781 | |

localeSuburb:Midsize | 0.96939 | 2.02343 | 0.48 | 0.632 | |

localeSuburb:Small | −1.4365 | 1.83284 | −0.78 | 0.4333 | |

localeTown:Distant | −0.2753 | 1.87544 | −0.15 | 0.8833 | |

localeTown:Fringe | −1.0562 | 2.37248 | −0.45 | 0.6562 | |

localeTown:Remote | −0.5238 | 2.05188 | −0.26 | 0.7986 | |

Ratio of Math Takers | −12.54 | 13.1567 | −0.95 | 0.3407 | |

Ratio of Reading Takers | 16.7817 | 11.3505 | 1.48 | 0.1395 | |

Ratio of Science Takers | 18.0389 | 9.77294 | 1.85 | 0.0651 | . |

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 9.82 on 1445 degrees of freedom

Multiple R-squared: 0.633, Adjusted R-squared: 0.626

F-statistic: 86.1 on 29 and 1445 DF, p-value: <2e-16

If we lower the standard to an 85% level of confidence, the pattern of for profit charters reducing a point and not for profit charters adding a point to the scores is similar to the one for reading.

I included the ratio of graduate degree holders to county population in order to control for the impact of college towns. The higher the ratio the more likely it is to be a college town. Unlike reading, it is a statistically significant variable.

On location, it seems that places other than Detroit do not enjoy the same advantage they had in reading. That small city is statistically significant may be do to the impact of college towns on the results.

The ratio of science takes is the number of 5th graders at the school who took the science MEAP divided by the total number of 5th graders. While it is not clear why every student doesn’t take the exam, one explanation is schools who encourage attendance for “count day“.

## Science

Coefficient | Std Error | t value | Pr(>|t|) | ||

(Intercept) | 532.374 | 7.66488 | 69.46 | 2E−16 | *** |

# of 5th Graders | −0.0109 | 0.00472 | −2.31 | 0.02098 | * |

Ratio of Free Lunch | −28.308 | 1.22634 | −23.08 | 2E−16 | *** |

Ratio of Reduced Price Lunch | −7.4145 | 4.2497 | −1.74 | 0.08125 | . |

School Size | 2.7E−05 | 0.00118 | 0.02 | 0.98194 | |

Native Am. Ratio | −11.896 | 7.46031 | −1.59 | 0.11101 | |

Asian Ratio | 7.25412 | 6.46084 | 1.12 | 0.26172 | |

Black Ratio | −24.767 | 5.61581 | −4.41 | 1.1E−05 | *** |

Native Hawaiian Ratio | −27.779 | 52.4407 | −0.53 | 0.59639 | |

White Ratio | −11.083 | 5.61352 | −1.97 | 0.04853 | * |

Hispanic Ratio | −16.516 | 5.94205 | −2.78 | 0.00552 | ** |

Female Ratio | 6.35563 | 2.6519 | 2.4 | 0.01667 | * |

For Profit Charter | −1.2096 | 0.64272 | −1.88 | 0.06004 | . |

Not For Profit Charter | 6.19395 | 1.4654 | 4.23 | 2.5E−05 | *** |

Ratio of BA/BS in County | 4.38208 | 8.88565 | 0.49 | 0.62197 | |

Ratio of Grad Degree in County | 4.00445 | 8.98363 | 0.45 | 0.65585 | |

localeCity:Midsize | 0.09766 | 1.13594 | 0.09 | 0.9315 | . |

localeCity: Small | 1.9431 | 1.02428 | 1.9 | 0.05802 | |

localeRural:Distant | 1.63264 | 1.22094 | 1.34 | 0.18137 | |

localeRural:Fringe | 1.4853 | 1.16004 | 1.28 | 0.20061 | |

localeRural:Remote | 3.0684 | 1.47697 | 2.08 | 0.03793 | * |

localeSuburb:Large | 0.38863 | 1.00111 | 0.39 | 0.69793 | |

localeSuburb:Midsize | 1.82408 | 1.49369 | 1.22 | 0.22221 | |

localeSuburb:Small | −0.4758 | 1.353 | −0.35 | 0.72515 | |

localeTown:Distant | 2.81157 | 1.38445 | 2.03 | 0.04246 | * |

localeTown:Fringe | 0.68612 | 1.75136 | 0.39 | 0.69529 | |

localeTown:Remote | 3.14625 | 1.51469 | 2.08 | 0.03796 | * |

Ratio of Math Takers | 7.27185 | 9.71225 | 0.75 | 0.45414 | |

Ratio of Reading Takers | 28.6602 | 8.37893 | 3.42 | 0.00064 | *** |

Ratio of Science Takers | −20.082 | 7.21438 | −2.78 | 0.00545 | ** |

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 7.25 on 1445 degrees of freedom

Multiple R-squared: 0.741, Adjusted R-squared: 0.735

F-statistic: 142 on 29 and 1445 DF, p-value: <2e-16

The science results show an even greater impact of for profit and not for profit charter schools on the science results. Not for profit charter schools adding more than 6 points to the MEAP science score. For profit charter schools reduce it by more than 1 point.

One counter intuitive result is that the higher the ratio of students taking the science portion the lower the score (by 20 points). This may reflect a general deficiency in science across Michigan.

## 2 thoughts on “Using Regression Analysis to Identify Factors in MI Primary School Test Performance”