Stabilizing Accountability Metrics Can Improve the Identification of Low-Performing Schools

 

A new REL Mid-Atlantic report describes a way to increase the accuracy of school accountability metrics. This approach, called Bayesian hierarchical modeling or Bayesian stabilization, reduces measurement error, which can obscure school performance, especially in small schools or student subgroups. In turn, reducing measurement error in school accountability systems can help states target the schools and subgroups that most need additional support. Under the Every Student Succeeds Act, states across the country must identify schools with low-performing student subgroups for Targeted Support and Improvement (TSI) or Additional Targeted Support and Improvement (ATSI). But measurement error—random differences between students’ test scores and their true abilities—introduces a risk that a subgroup might have low scores due to bad luck rather than truly low performance. The study used Bayesian stabilization and data provided by the Pennsylvania Department of Education to improve the reliability of subgroup proficiency measures used to designate schools for TSI or ATSI. The report demonstrates how Bayesian stabilization increases the reliability of subgroup proficiency measures, especially for small subgroups where random error tends to be greatest. Pennsylvania incorporated Bayesian stabilization as a “safe harbor” alternative that can move schools out of, but not into, ATSI status in its 2022 calculations, helping to reduce the risk that a school is identified for ATSI based on bad luck rather than true performance. Read the report.