Monday, September 30, 2013

Three studies of the NYC Bonus Program

In 2007, as part of its accountability system, the NYC Department of Education set school-level goals for student academic performance and growth for each school. Each year, it awarded Progress Report scores to schools based on three components: increased student achievement on state reading and math exams (25% of score), yearly student progress (60% of score), and measures of the learning environment (15% of score). The program operated in high-need schools from school years 2007–08 through 2010–11, with schools randomly assigned to either an intervention or a comparison group in 2007–08.

Under the SPBP, school staff could receive bonuses based on their schools’ Progress Reports. If a school was randomly selected for the program, it then had to secure votes in favor of program participation from 55% or more of its full-time union teachers in order for the school to be eligible for bonuses. Participating schools that reached 100% of their school-level goals could receive lump-sum payments of $3,000 per union teacher; those that reached at least 75% of their goals received $1,500 per union teacher. A four-member, school-level compensation committee decided in advance how to distribute payments among teachers and other staff.

The three studies of the NYC Bonus Program that were reviewed by the WWC are listed below. One study examined the effect of the program on individual student test scores; the other two looked at school average scores. Each of them included an “intent to treat” analysis, in which all study students and schools are included in the analysis, based on whether the school was initially assigned to the program. Two of the studies also included a “treatment on treated” analysis, which examines the effects based on which schools actually implemented the program. the Marsh study also examined academic achievement outcomes measured at the student level. However, the report did not contain enough information to determine a study rating for that portion of the study.

Fryer, R. G. (2011). Teacher incentives and student achievement: Evidence from New York City Public Schools (NBER Working Paper No. 16850). Cambridge, MA: National Bureau of Economic Research.

Rating: Meets WWC evidence standards without reservations

Goodman, S. F., & Turner, L. J. (2010). Teacher incentive pay and educational outcomes: Evidence from the New York City Bonus Program. New York: Columbia University.

Rating: Meets WWC evidence standards without reservations

Marsh, J. A., Springer, M. G., McCaffrey, D. F., Yuan, K., Epstein, S., Koppich, J., Kalra, N., DiMartino, C., Peng, A. (2011). A big apple for educators: New York City's experiment with schoolwide performance bonuses. Final evaluation report. Santa Monica, CA: RAND Corporation.

Rating: Meets WWC evidence standards without reservations

No comments: