Implementation of Response to Intervention Studied

Regional Educational Laboratory Midwest worked with Milwaukee Public Schools to develop a system for measuring implementation of Response to Intervention (RTI), a method that uses tiered levels of instruction adapted to student needs. School improvement coaches were trained to rate Milwaukee public elementary schools’ implementation of RTI using a rubric of 33 indicators.

This study, conducted through the Midwest Urban Research Alliance, examined the ratings produced by that system to determine the system’s reliability, the progress of schools in implementing RTI, and any relationship between the ratings to school characteristics. The report also discusses the utility of this implementation monitoring system for the district’s continuous improvement efforts.



School improvement coaches were trained and certified to conduct school data reviews. These reviewers visited 70 elementary schools serving grades K-5 in a single urban school district. During each visit, two reviewers made ratings on the 34-indicator rubric and entered their ratings into a dashboard system. Reviewers reconciled discrepant ratings and the reconciled ratings were analyzed. To determine the reliability of the rubric, the study team estimated inter-rater reliability using percent agreement and Cohen’s Kappa to account for chance ratings. Coefficient alphas were calculated to estimate inter-item reliability. To determine how well schools were implementing RTI, average ratings were calculated for each school on the total rubric and six components and converted into categories: “little fidelity”, “inadequate fidelity”, “adequate fidelity”, and “full fidelity”.

The study team also calculated Pearson product-moment correlations to study relationships between implementation ratings and characteristics of teachers and students in the schools. Results indicated that the ratings made by the trained data reviewers were reliable even when accounting for chance. Among the 68 visited schools that had complete data, 53 percent of the schools were implementing RTI with adequate fidelity after two years. However, 68 percent of the priority schools did not reach adequate levels of implementation fidelity.

Findings also revealed that most schools have yet to implement instruction for diverse students and Tier III instruction with fidelity.

Of the contextual factors studied, correlations were found between implementation scores and teacher and student characteristics. The system can be used to produce reliable evidence about the level of RTI implementation in schools and which components of RTI need to be the focus of professional development and coaching.

Also, if RTI is indeed an effective school improvement strategy, then by monitoring implementation fidelity of RTI, school districts can improve the chances that RTI produces the expected impacts in their school settings. Establishing an implementation monitoring system requires district staff time to complete training, conduct the data reviews, resolve rating discrepancies, and enter the data into a dashboard system.

Among the findings:

•    Fifty-three percent of schools showed adequate fidelity to the RTI framework, but implementation varied by school type (priority, focus, or other). Schools whose students were struggling academically also struggled with implementing RTI;

•    The biggest challenges for schools was implementing the multi-tiered instruction component, especially the tier 3 subcomponent and the evaluation component; and

•    Schools’ implementation of RTI was related to student and teacher characteristics.

Milwaukee Public Schools plans to use the results to identify which RTI processes need the most improvement. The report also describes a process that other districts can follow to develop their own implementation fidelity monitoring system for the RTI framework or for other interventions.