Making a Difference in Education Reform, ProComp External Evaluation Report

Ω

ProComp is an ambitious alternative teacher compensation system developed by Denver Public Schools (DPS) in cooperation with the Denver Classroom Teachers Association (DCTA). This report summarizes an evaluation of ProComp conducted collaboratively by The Evaluation Center and the Buechner Institute for Governance at the University of Colorado Denver and the Center for Education Data and Research at the University of Washington. This report describes the implementation of various elements of ProComp and results of the analyses of the association of ProComp with student achievement and changes in the composition of the DPS workforce.

The evaluation examined ProComp from implementation beginning in January 2006 through the 2009--_10 school year. The evaluation used a mixed--_methods design that incorporated data from interviews, focus groups, surveys, document reviews, and analyses of existing district data (e.g., human resources data and assessment results). The available data, and hence evaluation design, did not allow for a definitive distinction between ProComp effects and the impacts of other factors (e.g., concurrent reform efforts, economic changes, demographic shifts) on workforce recruitment and retention and student achievement. However, use of available statistical controls strengthened findings of effects associated with ProComp. The evaluation also incorporated a theory of change, which was used as a guide to examining a logical causal chain between ProComp elements and potential impacts.

Key Findings:

- Professional Development Units
As part of ProComp, DPS sought to redesign its professional development based on research-based practices. The result was the creation of the Professional Development Unit (PDU) element. This element was introduced as individual and small-group self-directed PDU studies. Over time, PDU learning experiences were expanded to include distric-sponsored and school-based PDU offerings. However, the student achievement analysis found PDUs were not significant predictors of CSAP student achievement in either math or reading.

- Advanced Degrees and Licenses
Another element in the Knowledge and Skills component is the Advanced Degrees/Licenses element. This incentive is most similar to the traditional salary schedule and provides salary increases for earning a master’s or doctorate degree and/or for receiving an advanced license such as National Board for Professional Teaching Standards and School Nurse Practitioner Certification. The Tuition/Student Loan Reimbursement element reimburses teachers for expenditures for past education or professional training. Teachers liked receiving these incentives, but there is limited evidence that they led to changes in instructional practice or improved student CSAP achievement.

- Student Achievement
A primary goal of the ProComp system is to increase student achievement in DPS. DPS has experienced significant student learning gains across grades and subjects, but it is not clear that this was the result of ProComp. There was not a consistent pattern across grade levels and subjects in the relationship between ProComp and observed achievement gains. In some cases, the gains appeared primarily among students with ProComp teachers, while in other cases it is Non-ProComp teachers who appeared to be more effective. Though puzzling, these findings are consistent with research on other well-known interventions that include elements similar to ProComp.