Comparison of Charter School versus Public School Productivity: No Validity

Bigger Bang, Fewer Bucks?: The Productivity of Public Charter Schools in Eight U.S. Cities, published by the University of Arkansas Department of Education Reform, contends that charter schools produce more achievement per dollar invested, as compared to public schools. 

Professor Julian Vasquez Heilig of California State University Sacramento reviewed the report and identified a variety of methodological choices made by the authors that threaten the validity of the results. 

The report is focused on city-level analyses in eight U.S. cities (Atlanta, Boston, Denver, Houston, Indianapolis, New York City, San Antonio, and Washington D.C.) and uses cost effectiveness and Return on Investment ratios. It concludes that charter schools deliver a weighted average of an additional 4.34 NAEP reading points and 4.73 NAEP math points per $1000 invested.

However, Professor Vasquez Heilig points out that the report fails to account for the non-comparability of the student populations in charter and comparison public schools. Four other problems also undercut the report’s claims. 

•  First, the report uses revenues rather than actual expenditures, despite well-established critiques of this approach. 
• Second, the report’s lack of specificity (e.g., using state-level data in city-level analyses and completely excluding race and gender) plagues the accuracy and validity of its calculations.
• Third, the authors fail to reconcile their report with the extensive literature of contrary findings. 
• Finally, even though the think tank’s earlier productivity report included a caveat saying that causal claims would not be appropriate, and even though the new report’s analyses also are insufficient to make causal claims, the new report omits that caution.

The evidence in this report is so flawed that it provides no valid guidance to educators or policymakers.