The student growth percentile model lacks reliability for high-stakes decisions

The student growth percentile model is a method that some states use to evaluate teachers based partly on student learning. Using this method, a score is computed based on student growth that is assumed to reflect a teacher’s current and future effectiveness.

Regional Educational Laboratory West recently conducted a study in partnership with the Nevada Department of Education, which was considering using teacher-level scores derived from a student growth percentile model for its teacher evaluation system. Three years of math and reading score data were analyzed for elementary and middle school students and teachers from Washoe County School District, the second-largest district in Nevada and one of the 60 largest districts in the nation.

The study examined the stability over years of teacher-level growth scores from the Student Growth Percentile (SGP) model, which many states and districts have selected as a measure of effectiveness in their teacher evaluation systems. The authors conducted a generalizability study using three years of data in mathematics and reading for nearly 370 elementary and middle school teachers from Washoe County School District in Reno, Nevada’s second-largest district. 

 

The study found that in mathematics, half of the variation among teachers’ annual growth score (median SGPs) was attributable to differences among teachers, while half was random or unstable. In reading, .41 of the variance in annual scores was attributable to differences among teachers, while .59 was due to random or unstable sources.

In other words, the study finds that at least half of the variance in teacher scores from the statistical model that used Nevada data was due to random or otherwise unstable sources rather than reliable information that could predict future student performance. Even when derived by averaging several years of teacher scores, the authors determined that effectiveness estimates are unlikely to provide the level of reliability that is desired when using scores for high-stakes decisions, such as tenure or dismissal.

Given these findings, the study authors conclude that Nevada and other states may want to be cautious about using scores from the student growth percentile model as measures of teacher effectiveness for high-stakes decisions.