Saga Math has been evaluated in three large-scale randomized controlled trials involving over 6,000 largely Black and Latinx 9th and 10th graders from economically disadvantaged schools. The first study took place in the 2013–14 school year in 12 high poverty high schools in Chicago. Over 2,000 male 9th and 10th grade students were identified as at-risk for drop-out and were randomized to receive Saga or routine Chicago Public Schools (CPS) services. After one year, the intent-to-treat (ITT) analysis, which includes students assigned to Saga tutoring regardless of any uptake, resulted in Saga students scoring significantly higher on standardized math achievement test scores (effect size = +0.09). Impacts were even more robust in the treatment-on-the-treated (TOT) analysis – students who received Saga tutoring significantly outperformed those in the control condition (effect size = +0.18).
The second study, expanded to 15 CPS high schools, randomized 2,710 9th and 10th graders to receive either Saga tutoring or CPS routine services. The estimated ITT effect of being assigned to treatment on end of year math achievement tests was significantly positive (effect size = +0.14) and the estimated TOT effect (impact for those actually receiving tutoring) was even greater (effect size = +.40). Across the two studies, large impacts outside of standardized test gains were also noted: significant increases in grades in regular math courses, declines in math course failures, and persistent gains a year or two after tutoring on math test scores and math grades were found for Saga students in comparison to those receiving business-as-usual interventions.
The third RCT, examining the effects of the Saga Technology tutoring model that has 9th grade students spending every other day with a tutor for an hour, at a 2-students-to-1-tutor ratio, with the off days spent on CAL, took place in 7 schools across Chicago and NYC during the 2018-2019 and 2019-2020 school years. Results are focused on the 2018-2019 cohort, as end-of-the-year standardized tests were not administered in 2020 because of school closures due to Covid-19. In the first year, approximately 2000 students were randomized to receive Saga’s daily blended-learning high dosage tutoring or to a business-as-usual control condition. The sample included more than 90% qualifying for free and reduced price lunch, 57% Latinx, 24% Black, and 17% identified as diverse learners. After controlling for pretests and other baseline demographics, students assigned to receive Saga performed significantly higher on standardized math measures at the end of the school year (effect size= +0.19) than control students. Additionally, participating in Saga tutoring resulted in significant increases in math GPA and decreases in math course failures.
No comments:
Post a Comment