Review Claims Charter Report Overstates Results

*


'Everyone Wins' ignores factors besides competition to explain marginally improved public school achievement in NYC, according to reviewer

A recently released Manhattan Institute report looks at the competition effects of New York City's charter schools and concludes that "students benefit academically when their public school is exposed to competition from a charter." A new review of that report finds that the report's findings are minimal and may be explained by factors other than charter-school competition.

The report, Everyone Wins: How Charter Schools Benefit All New York City Public School Students, was reviewed for the Think Twice project by Patrick McEwan, professor of economics at Wellesley College.

McEwan's complete review is here.

The report's focus reflects an important premise behind the market-competition approach to education reform. While critics of market theory contend competition from charter schools or other alternatives will end up penalizing the public schools that must educate most children, market advocates contend competition will help the entire school population. Competition, this argument runs, doesn't just expand choices for parents; it also prods existing public schools to improve in order to avoid losing students.

Everyone Wins draws on three years of test score data in mathematics and English Language Arts (ELA) from New York City public schools as well as data on the percentage of students leaving public schools for charter schools. As McEwan explains, the report uses the rate of departures for charter schools as a proxy to measure increased "pressure on public school administrators to 'compete,' improve test scores, and staunch the flow of students to charter schools." Using appropriate statistical controls, the report finds that increasing competition does not appear to be associated with improved math test scores, while it has "small positive effects on ELA scores" that are "slightly larger among public school students with lower levels of achievement."

In his review, McEwan observes that the report itself is modest in its conclusions and that it "correctly notes that the statistical findings do not necessarily imply that increases in the measure of competition cause test scores to rise..." However, the report's title suggests a much more positive definitive outcome, and the nuances are lost as well in the executive summary.

McEwan praises the report on several counts: its use of "a high-quality, longitudinal dataset of student achievement," its use of appropriate statistical methods, and its contribution to "an established literature that finds roughly consistent effects of 'competition' that are often zero or slightly positive, depending on the state and method."

But he also points out important limitations. The statistical methods used, "while appropriate...cannot control for several potential biases," McEwan writes. "As a result, the measured effects of competition could also reflect the influence of shifting peer quality, declining class size, or other unobserved variables." Other sources of market pressure, such as private schools or schools of choice, are also not considered in this report.

The report notes in passing some of those limitations, but "does not make a serious attempt to assess their validity," the reviewer concludes. In the end, then, it does not deliver what it claims, and we do not know whether, in fact, "everyone wins."