Impact of For-Profit and Nonprofit Management on Student Achievement: The Philadelphia Intervention, 2002–2008
What Types of Schools Were Contrasted?
In the summer of 2002, the Philadelphia school district contracted with private organizations to take over the management of its 46 lowest performing elementary and middle schools.
Thirty schools were contracted out to the for-profit management organizations Edison Schools, Victory Schools, and Chancellor Beacon Academies.
Sixteen schools were contracted out to four nonprofits: Foundations, the University of Pennsylvania, Temple University, and Universal Companies.
These organizations were subject to the district’s existing collective bargaining agreements.
What is this study about?
The study examined whether shifting from traditional district management to management by a for-profit or nonprofit organization improves student achievement.
The study analyzed data on six cohorts of elementary and middle school students (2002–03 through 2007–08) in the Philadelphia school district.
The study used three approaches to estimate effects on student test scores. For two of the three approaches, the study did not present evidence on test score equivalence at baseline, which is one of the standards used by the WWC to assess validity. This quick review focuses on the approach that presents evidence on baseline test-score equivalence.
The more rigorous approach contains two main analyses. The first analysis (referred to below as the “District Comparison Analysis”) examined data from six school years and included about 116,000 students. The math and reading scores of students in schools that were managed by a for-profit company or a nonprofit organization were compared to the test scores of students in the 71 lowest-performing schools that remained under regular district management.
The second analysis (referred to below as the “Matched Comparison Analysis”) matched forprofit- managed schools and non-profit- managed schools to the 30 schools under regular district management that were most similar to them based on prior test scores and student demographic composition. The test scores of students in schools managed by a for-profit or nonprofit organization were compared to those of students in matched comparison schools.
WWC Rating of District Comparison Analysis
The District Comparison Analysis described in this report is not consistent with WWC evidence standards
Cautions: The analysis compared groups of students that were not similar at baseline. The students enrolled in schools managed by for-profit companies had lower baseline test scores than students in the comparison schools run by the district. In addition, the student population served by the nonprofit-managed schools had a different demographic composition from the population served by the schools managed by the district.
What did the study authors report about the District Comparison Analysis?
The study reported that, on average over the six years following the shift in school management, students in for-profit managed schools had math and reading test scores that were not significantly different from those of students in comparison schools run by the district. During this same period, students in nonprofit managed schools had math and reading test scores that were about one-seventh of a standard deviation lower than students in the comparison schools.
However, the WWC does not consider these results to be conclusive because there is evidence that the schools were not initially equivalent. The differences may reflect differences in the student populations served by the different groups of schools and not the effect of the school management strategies.
WWC Rating of Matched Comparison Analysis
The Matched Comparison Analysis described in this report is consistent with WWC evidence standards with reservations
Strengths: The intervention and comparison schools were well-matched on baseline test scores and demographic composition of the student body.
Cautions: Although the study matched treatment and comparison schools on a number of observable characteristics, unobserved differences between the groups of schools and the students that attend them may remain. In addition, the study examined six years of data. Although the students enrolled in these groups of schools may have initially been similar, the composition of students enrolled in these schools may have changed over time; therefore, the student populations in the schools compared in the analysis may not have remained similar throughout the follow-up period. Differences in achievement between the groups of students compared in this analysis may not be solely attributable to the effect of different school management strategies.
What did the study authors report about the Matched Comparison Analysis?
The authors found no statistically significant differences between student test scores at schools managed by for-profit or nonprofit organizations and students in their respective matched comparison schools.
The WWC has reservations about these results because students enrolled in privately managed schools may differ from students enrolled in traditionally managed schools in ways not controlled for in the analysis.
No comments:
Post a Comment