Massachusetts urban charter schools boost achievement sharply, but results for nonurban charters are mixed

Ω

Researchers from the Harvard Graduate School of Education, MIT and the University of Michigan have released the results of a new study that suggests that urban charter schools in Massachusetts have large positive effects on student achievement at both the middle and high school levels. Results for nonurban charter schools were less clear; some analyses indicated positive effects on student achievement at the high school level, while results for middle school students were much less encouraging.

The study is published in a report by the Center for Education Policy Research at Harvard titled, Student Achievement in Massachusetts’ Charter Schools.

The study uses an innovative research design based on school lotteries. The lottery-based research design relies on apples-to-apples comparisons: among those who apply to a given set of charter schools, applicants who were randomly offered a charter seat are compared with otherwise similar students who were not offered a seat. The school lotteries, which are required under the state’s charter law when a school has more applicants than seats, provide a way to answer the common complaint that the charter school applicants are ‘different’ from their peers in the traditional public schools.

Large positive effects of charter schools on student achievement were found at both the middle and high school levels but varied by subject and community type. As a group, Massachusetts’ charter middle schools boost average math scores but have little effect on average English Language Arts (ELA) scores. Results for charter high schools show strong effects in both math and ELA. But a more detailed analysis shows that the impact of charter schools on student achievement varies markedly across communities: Urban charter schools show large, positive and statistically significant effects across subjects and grades. The subset of oversubscribed urban charter schools generates especially large MCAS score gains in middle and high school math. Other urban charter schools generate smaller though still positive effects.

By contrast, nonurban middle schools do not appear to boost scores and the students attending these schools may even be falling behind their counterparts in traditional public schools. The picture for nonurban charter high schools is more mixed, with lottery estimates showing no achievement gains except in a subgroup of applicants of low socio-economic status.

“This study replicates our earlier findings showing substantial gains for charter school lottery winners. Although our urban sample size has increased somewhat, the findings for urban schools are unchanged: students offered a seat at an urban charter school end up well ahead of those denied admission in a lottery. On the other hand, the small and even negative effects for non-urban charter schools is a new result that shows just how much heterogeneity there is in the charter world,” said Angrist.

The overall findings of this study are consistent with previous research using the same methods conducted in 2009 and 2010. In one study, the team compared student achievement in Boston’s charter, pilot and traditional schools. That research is published in a report by The Boston Foundation titled, Informing the Debate: Comparing Boston’s Charter, Pilot and Traditional Schools. Findings from that groundbreaking study found positive effects of charter schools on student achievement at both the middle and high school levels and across subjects. In fact, the impact on middle school math was particularly dramatic; the effect amounting to half of a standard deviation, an effect large enough to move a student from the 50th to 69th percentile in student performance in one year. Likewise, in a study published in May 2010, the team report large achievement gains for students who won seats at the KIPP School in Lynn, Massachusetts (also an urban area with a large minority population).

The new study is not designed to answer the question of why particular schools boost achievement only whether they succeed in doing that. However, a survey of administrators at participating charter schools revealed clear differences in approach between schools set in urban and non-urban communities. For example, urban charters spend considerably more time on traditional math and reading instruction and are more likely to subscribe to a “no excuses” philosophy (emphasizing traditional skills and comportment). Urban charters also typically run a longer school day than their nonurban counterparts. Finally, it’s worth noting that urban students start well behind most non-urban students. “Urban schools boost scores by bringing these students up to suburban achievement levels. Most non-urban students start out doing relatively well; charter attendance leaves their achievement essentially unchanged but at a relatively high baseline level,” said Angrist.

"The findings of this study are provocative. They suggest, as in previous studies, that students in Massachusetts' charter middle and high schools often perform better academically than their peers in traditional public schools. The task now is to learn more about what is working in charter schools -- such as more effective instruction, engaging academic curricula, longer school days, longer school year, or other practices -- so that similar elements can be implemented in traditional public schools," said Mitchell Chester, Commissioner of Elementary and Secondary Education for Massachusetts.