Does Pennsylvania’s Tool for Evaluating School Leaders Measure Their Effectiveness Accurately?

This study examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania’s tool for evaluating the leadership practices of principals and assistant principals. The study analyzed three key properties of the FFL: internal consistency, score variation, and concurrent validity. 

To measure the internal consistency of the FFL, Cronbach’s alpha was calculated for the full FFL and for each of its four categories of leadership practices. 

Score variation was characterized by the percentages of school leaders earning scores in different portions of the rating scale. 

Concurrent validity was assessed through a regression model for the relationship between school leaders’ estimated contributions to student achievement growth and their FFL scores. 

Based on a pilot in which 336 principals and 69 assistant principals were rated by their supervisors in 2012/13, this interim report finds that the full FFL had good internal consistency for both principals and assistant principals. However, most scores for specific leadership practices were in the top two of four possible performance levels, and FFL scores were not associated with school leaders’ contributions to student achievement growth. 

These findings suggest that more evidence is needed on the validity of using FFL scores to identify effective and ineffective school leaders.