Comparing NHA Charter School Performance with Nearest Neighbor

Responding to a recent Detroit Free Press series, the CEO of National Heritage Academy (NHA) had this to say:

Apples-to-apples comparisons to neighboring district schools help explain the appeal. More than 90% of NHA’s Michigan schools had a higher overall proficiency rate than neighboring district schools on the Michigan Educational Assessment Program.

Which raises interesting questions that no one else has documented because life is short and math is hard.  In this post I’m going to compare NHA schools performance with the nearest neighbor for 5th grade MEAP results in Math, Reading, and Science.

I’ve created a custom Google map to document the locations (be sure to click on the icons to see the schools results, demography, and socio-economic indicators).

Below is the analysis, but watch out, there will be MATH!

Statistical Analysis

The best statistic for this analysis is student t-test for paired samples with unequal variance where:

$\mu_{1}=$ The average score for NHA schools

$\mu_{2}=$ The average score for the next nearest schools

$\sigma^2_{1}=$ The standard deviation of NHA schools scores

$\sigma^2_{2}=$ The standard deviation for the next nearest schools

$\bar{d} =$ The sum of the differences between NHA school scores and the next nearest neighbor.

The t statistic is calculated as:
$t=(\bar{d}-(\mu_{1} -\mu_{2}))\div\sqrt{(s_{1}^2\div n_{1}) - (s_{2}^2\div n_{2)})}$

The above calculated t will be compared to the following:

$t_{\alpha,n-1}=$   Using a 99% confidence level this value is -2.41

This test compares the idea that there is something different with the idea that there is nothing different.  The nothing different is called the NULL Hypothesis.  In math language that is:

$H_{0}:\mu_{1}-\mu_{2}=0$

The something different is called the Alternative Hypothesis is expressed as:

$H_{1}:\mu_{1}-\mu_{2}>0$

This means that if we calculate the t value for the data and its larger than -2.41, we reject the NULL Hypothesis and conclude  that NHA test scores are higer.

Math

t = 1.9224

mean of the differences is: 4.544

Reading

t = 1.84

mean of the differences is: 3.57

Science

t= 2.27

mean of the differences is: 4.3

What does this mean in English?  We can conclude that NHA schools score higher on all MEAP categories than their next nearest neighbor at a 99% level of confidence.

Possible Explanations

There are a number of possible reasons that NHA schools have such strong out performance over their nearest neighbor.  In a previous post I did a regression analysis that showed that composition by gender, race, and socio-economic status strongly impacted test scores.  There is an enormous amount of academic research supporting a causal link between more free lunch students and lower academic performance.  My own regression shows that while both higher free lunch and reduced price lunch ratios negatively impact test performance, the free lunch category is much bigger in its impact.

Using the same t-test above:

Ratio of Free Lunch Students

t = -1.78

mean difference between is: -0.068

Ratio of Reduced Price Lunches

t = 5.1469

mean difference between is: 0.034

Using that test, we can conclude at 99% level of confidence that NHA schools have fewer free lunch students and more reduced price lunch students.   Because the mean difference is expressed as a ratio, you can multiply it by 100 to get turn it into a percentage.  That means that if the nearest school has 50% free lunch and 10% reduced price lunch students, the NHA school will have 43.2% free lunch students and 13.4% reduced price lunch students.  Reports about school performance often obscure the difference between the two by combining them and a reporting them as a single percentage and thus obscuring the fact.

Fewer free lunch students are helping NHA school’s performance

Advertisements