Mixed reliability results from New Jersey’s principal evaluation pilot

 

The purpose of this study was to describe the measures used to evaluate principals in New Jersey in the first (pilot) year of the new principal evaluation system and examine three of the statistical properties of the measures: their variation among principals, their year-to-year stability, and the associations between these measures and the characteristics of students in the schools.

 The study reviewed information that developers of principal practice instruments provided about their instruments and examined principals' performance ratings using data from 14 districts in New Jersey that piloted the principal evaluation system in the 2012/13 school year.

The study had four key findings:

First, the developers of principal practice instruments provided partial information about their instruments' reliability (consistency across raters and observations) and validity (accurate measurement of true principal performance).

Second, principal practice ratings and schoolwide student growth percentiles have the potential to differentiate among principals.

Third, school median student growth percentiles, which measure student achievement growth during the school year, exhibit year-to-year stability even when the school changes principals. This may reflect persistent school characteristics, suggesting a need to investigate whether other evaluation measures could more closely gauge principals' contributions to student achievement growth.

Finally, school median student growth percentiles correlate with student disadvantage, a relationship that warrants further investigation using statewide evaluation data.

Results show a mix of strengths and weaknesses in the statistical properties of the measures used to evaluate principals in New Jersey.