Charter School Report Provides Little Basis for Advocacy

The overall research base is now clear that the charter school sector largely mirrors the conventional public school sector in terms of students’ test scores. This is again confirmed by a recent analysis of charter schools in Michigan. A new review of that study points to some limitations but concludes that it employs solid analytic methods and relies on a large, impressive dataset.

Charter School Performance in Michigan was reviewed for the Think Twice think tank review project by professor Andrew Maul of the University of Colorado Boulder. Maul’s scholarly work focuses on measurement theory, validity, and generalized latent variable modeling. The review is published by the National Education Policy Center, housed at the CU Boulder School of Education.

The Michigan report is the work of the Center for Research on Education Outcomes (CREDO) at Stanford University. The CREDO researchers analyzed differences in student performance at charter schools and traditional public schools in Michigan.

Up to this point, the majority of high-quality research studies on charter effects in the U.S. have tended to show no meaningful impact—positive or negative—on student achievement. Against this background, the new CREDO Michigan study has been trumpeted by charter advocates as showing a “smashing success” for charter schools.

In truth, the new study estimates that students in charter schools in Michigan experience 0.06 standard deviations more academic growth than comparison students in traditional public schools. As Maul points out, “This is equivalent to saying that about a tenth of one percent of the variation in academic growth is associated with school type.” Such a finding of almost no difference between charters and non-charters is very much in line with the overall body of past research. Some studies suggest slight benefits, some suggest slight harm, and many show no difference.

The study itself has both strong and weak elements. “As with CREDO’s previous reports on charter schools, the study employs a large and comprehensive dataset and fairly solid analytic methods,” Maul notes. But he goes on to point out “significant reasons for caution in interpreting the study’s results.”

Maul’s discussion of research methods is somewhat technical but is clearly explained in his review. One concern is the study’s failure to use methods that could have directly modeled both individual student growth and school-level effects, such as hierarchical linear modeling, which would have been better matched to the goals of making generalizable statements about both students and schools. He also questions whether the seven variables used in the study’s “virtual twin” matching approach are truly sufficient to capture all meaningful differences between charter students and those at traditional public schools.

Despite such caveats, Maul concludes that the study contains enough information that it represents an interesting contribution to the research literature on charter school effectiveness.

Find Andrew Maul’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-charter-performance-michigan

Find Charter School Performance in Michigan, published by CREDO at Stanford University, on the web at:
http://credo.stanford.edu/pdfs/MI_report_2012_FINAL_1_11_2013_no_watermark.pdf.

The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policy makers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.