Identifying early warning indicators

The purpose of this study was to identify a set of data elements for students in grades 8 and 9 in three Ohio school districts that could serve as accurate early warning indicators of their failure to graduate high school on time and to comparatively examine the accuracy of those indicators. 

 

In order to identify the set of indicators with the optimal accuracy for each district, the research team collected student-level data on two cohorts of grade 8 and 9 students in each school district. Datasets used in the analyses included students’ four-year graduation status (the outcome) and 8th and 9th grade data on attendance, coursework, suspensions, and test score records (the candidate early warning indicators).

 

Results show that the optimal cut point for classifying students as at risk varied significantly across districts for five of the eight candidate indicators included in the study. Across the three districts and two grades, different indicators were identified as the most accurate predictors of students’ failure to graduate on time. End-of-year attendance rate was the only indicator that was a consistent predictor for both grades in all three districts. 

 

The most accurate indicators in both grade 8 and grade 9 were based on coursework (GPAs and course credits). Consistent with prior literature, failing more than one class and earning one or more suspensions also were strong predictors of failure to graduate on time. 

 

On average, indicators were more accurate in grade 9 than in grade 8. Findings illustrate why it is important for districts to conduct local validation using their own data to verify that indicators selected for their early warning systems accurately predict students’ failure to graduate on time. 

 

The methods laid out in this study can be used to help districts identify the best off-track indicators, and indicator cut points, for their particular early warning systems.