Using Regression Analysis to Identify Factors in MI Primary School Test Performance

The Detroit Free Press recently ran an interesting series of stories about charter schools in Michigan which made me wonder if I could use regression analysis to partition the variation in test results on the basis of race, poverty, location, and type of school.  This analysis uses MEAP scores for the 5th grade.

tl;dr — For profit charter schools generally produce worse results than public or not for profit charters while not for profit charters produce better results.

If you’re thinking “eeeeeeeeeehhhhhhhwwwwwwwww math”, I hope to soon incorporate the results into an on line map to make it easy to look at the results for particular schools .

If you are unfamiliar with interpreting the results of regression analysis, or would like a refresher, I suggest reading this description of interpreting the results from R.

 Reading

Coefficients:
Coefficient StdError t value Pr(>|t|)
(Intercept) 557.78 6.83184 81.64 2E−16 ***
# of 5th Graders −0.0058 0.00421 −1.37 0.17032
Ratio of Free Lunch −29.376 1.09306 −26.88 2E−16 ***
Ratio of Reduced Price Lunch −7.8321 3.78783 −2.07 0.03884 *
School Size −0.0009 0.00105 −0.87 0.38502
Native Am. Ratio −20.985 6.64949 −3.16 0.00163 **
Asian Ratio −5.4472 5.75865 −0.95 0.34435
Black Ratio −28.327 5.00546 −5.66 1.8E−08 ***
Native Hawaiian Ratio −55.509 46.7412 −1.19 0.23519
White Ratio −20.324 5.00343 −4.06 5.1E−05 ***
Hispanic Ratio −24.364 5.29625 −4.6 4.6E−06 ***
Female Ratio 10.4082 2.36368 4.4 1.1E−05 ***
For Profit Charter −0.9124 0.57287 −1.59 0.11146
Not For Profit Charter 2.85784 1.30613 2.19 0.02883 *
Ratio of BA/BS in County 7.55157 7.91992 0.95 0.3405
Ratio of Grad Degree in County 6.84643 8.00726 0.86 0.39268
localeCity:Midsize 1.9757 1.01248 1.95 0.05121 .
localeCity: Small 2.61064 0.91296 2.86 0.0043 **
localeRural:Distant 2.8529 1.08825 2.62 0.00884 **
localeRural:Fringe 2.19042 1.03396 2.12 0.03431 *
localeRural:Remote 4.49699 1.31645 3.42 0.00065 ***
localeSuburb:Large 1.77036 0.89231 1.98 0.04744 *
localeSuburb:Midsize 3.12932 1.33136 2.35 0.01888 *
localeSuburb:Small 1.87596 1.20596 1.56 0.12003
localeTown:Distant 3.87374 1.23399 3.14 0.00173 **
localeTown:Fringe 2.43674 1.56102 1.56 0.11874
localeTown:Remote 4.33804 1.35007 3.21 0.00134 **
Ratio of Math Takers 5.06051 8.65669 0.58 0.55892
Ratio of Reading Takers −0.1406 7.46828 −0.02 0.98498
Ratio of Science Takers 2.51456 6.4303 0.39 0.69582

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 6.46 on 1445 degrees of freedom
Multiple R-squared: 0.741, Adjusted R-squared: 0.736
F-statistic: 142 on 29 and 1445 DF, p-value: <2e-16

One thing to keep in mind about this data is that when it says a ratio, it means the total number divided by the other number and not a whole number.  So the black ratio is the number of black 5th grade students divided by the total number of 5th graders.  You can think of the coefficients as representing the change to the test score if all the participants are of that category.  For example, a school that was 100% female would have a MEAP reading score 10 points higher.

The regression results suggest that charter schools deliver results that are different from the public schools.  Not for profit charter schools add almost 3 points to the reading score.  For profit charter schools reduce the score by almost a point (at lower threshold of statistical confidence).

The location variables show that almost anywhere that isn’t Detroit perform much better than schools in Detroit.  Like the school type variable, the location variables are dummy variables.     That means one variable is left out of the output and serves as a kind of baseline, which is why City: Large doesn’t appear in the output.

Math

Coefficient Std Error t value Pr(>|t|)
(Intercept) 527.372 10.3832 50.79 2E−16 ***
# of 5th Graders −0.009 0.00639 −1.41 0.1585
Ratio of Free Lunch −32.252 1.66126 −19.41 2E−16 ***
Ratio of Reduced Price Lunch −17.669 5.75684 −3.07 0.0022 **
School Size 0.00067 0.0016 0.42 0.6736
Native Am. Ratio −1.9356 10.1061 −0.19 0.8481
Asian Ratio 40.4056 8.75216 4.62 4.2E−06 ***
Black Ratio −18.678 7.60744 −2.46 0.0142 *
Native Hawaiian Ratio 18.8099 71.0386 0.26 0.7912
White Ratio −7.6855 7.60434 −1.01 0.3123
Hispanic Ratio −9.8284 8.04938 −1.22 0.2223
Female Ratio 5.21048 3.59239 1.45 0.1472
For Profit Charter −1.2754 0.87066 −1.46 0.1432
Not For Profit Charter 2.89958 1.98509 1.46 0.1443
Ratio of BA/BS in County −10.772 12.0369 −0.89 0.371
Ratio of Grad Degree in County 28.963 12.1697 2.38 0.0174 *
localeCity:Midsize 1.37839 1.5388 0.9 0.3705
localeCity: Small 2.77564 1.38754 2 0.0456 *
localeRural:Distant −1.0587 1.65395 −0.64 0.5222
localeRural:Fringe 0.42601 1.57145 0.27 0.7864
localeRural:Remote −0.1376 2.00077 −0.07 0.9452
localeSuburb:Large 1.4715 1.35615 1.09 0.2781
localeSuburb:Midsize 0.96939 2.02343 0.48 0.632
localeSuburb:Small −1.4365 1.83284 −0.78 0.4333
localeTown:Distant −0.2753 1.87544 −0.15 0.8833
localeTown:Fringe −1.0562 2.37248 −0.45 0.6562
localeTown:Remote −0.5238 2.05188 −0.26 0.7986
Ratio of Math Takers −12.54 13.1567 −0.95 0.3407
Ratio of Reading Takers 16.7817 11.3505 1.48 0.1395
Ratio of Science Takers 18.0389 9.77294 1.85 0.0651 .

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 9.82 on 1445 degrees of freedom
Multiple R-squared: 0.633, Adjusted R-squared: 0.626
F-statistic: 86.1 on 29 and 1445 DF, p-value: <2e-16

If we lower the standard to an 85% level of confidence, the pattern of for profit charters reducing a point and not for profit charters adding a point to the scores is similar to the one for reading.

I included the ratio of graduate degree holders to county population in order to control for the impact of college towns.  The higher the ratio the more likely it is to be a college town.  Unlike reading, it is a statistically significant variable.

On location, it seems that places other than Detroit do not enjoy the same advantage they had in reading.  That small city is statistically significant may be do to the impact of college towns on the results.

The ratio of science takes is the number of 5th graders at the school who took the science MEAP divided by the total number of 5th graders.  While it is not clear why every student doesn’t take the exam, one explanation is schools who encourage attendance for “count day“.

 

Science

 

Coefficient Std Error t value Pr(>|t|)
(Intercept) 532.374 7.66488 69.46 2E−16 ***
# of 5th Graders −0.0109 0.00472 −2.31 0.02098 *
Ratio of Free Lunch −28.308 1.22634 −23.08 2E−16 ***
Ratio of Reduced Price Lunch −7.4145 4.2497 −1.74 0.08125 .
School Size 2.7E−05 0.00118 0.02 0.98194
Native Am. Ratio −11.896 7.46031 −1.59 0.11101
Asian Ratio 7.25412 6.46084 1.12 0.26172
Black Ratio −24.767 5.61581 −4.41 1.1E−05 ***
Native Hawaiian Ratio −27.779 52.4407 −0.53 0.59639
White Ratio −11.083 5.61352 −1.97 0.04853 *
Hispanic Ratio −16.516 5.94205 −2.78 0.00552 **
Female Ratio 6.35563 2.6519 2.4 0.01667 *
For Profit Charter −1.2096 0.64272 −1.88 0.06004 .
Not For Profit Charter 6.19395 1.4654 4.23 2.5E−05 ***
Ratio of BA/BS in County 4.38208 8.88565 0.49 0.62197
Ratio of Grad Degree in County 4.00445 8.98363 0.45 0.65585
localeCity:Midsize 0.09766 1.13594 0.09 0.9315 .
localeCity: Small 1.9431 1.02428 1.9 0.05802
localeRural:Distant 1.63264 1.22094 1.34 0.18137
localeRural:Fringe 1.4853 1.16004 1.28 0.20061
localeRural:Remote 3.0684 1.47697 2.08 0.03793 *
localeSuburb:Large 0.38863 1.00111 0.39 0.69793
localeSuburb:Midsize 1.82408 1.49369 1.22 0.22221
localeSuburb:Small −0.4758 1.353 −0.35 0.72515
localeTown:Distant 2.81157 1.38445 2.03 0.04246 *
localeTown:Fringe 0.68612 1.75136 0.39 0.69529
localeTown:Remote 3.14625 1.51469 2.08 0.03796 *
Ratio of Math Takers 7.27185 9.71225 0.75 0.45414
Ratio of Reading Takers 28.6602 8.37893 3.42 0.00064 ***
Ratio of Science Takers −20.082 7.21438 −2.78 0.00545 **

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 7.25 on 1445 degrees of freedom
Multiple R-squared: 0.741, Adjusted R-squared: 0.735
F-statistic: 142 on 29 and 1445 DF, p-value: <2e-16

The science results show an even greater impact of for profit and not for profit charter schools on the science results.  Not for profit charter schools adding more than 6 points to the MEAP science score.  For profit charter schools reduce it by more than 1 point.

One counter intuitive result is that the higher the ratio of students taking the science portion the lower the score (by 20 points).  This may reflect a general deficiency in science across Michigan.

 

Advertisements

2 thoughts on “Using Regression Analysis to Identify Factors in MI Primary School Test Performance

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s