Personalized Learning Study's "Promising" Claims Diminished by Limited Evidence and Weak Generalizability

A recent report from the RAND Corporation explores three school-wide initiatives funded by the Gates Foundation to promote personalized learning. The report includes many strengths, but a review explains that the study provides little support for the evidence about personalized learning to be described as “promising” for all students.

William R. Penuel, Professor of Learning Sciences and Human Development, and doctoral candidate Raymond Johnson, both at the University of Colorado Boulder, reviewed Continued Progress: Promising Evidence on Personalized Learning for the Think Twice Think Tank Review Project at the National Education Policy Center, housed at the University of Colorado Boulder’s School of Education.

Personalized learning encompasses a range of strategies, from developing learner profiles with individualized goals using data to providing personalized learning paths, in which students have choice, get individualized support, and engage in learning outside school. Accordingly, the term personalized learning can mean many different things. In this report, RAND researchers organized personalized learning according to five different strategies:

• Learner profiles with individualized goals using data from multiple sources that students and teachers both access,
• Personalized learning paths, in which students have choice, get individualized support, and engage in learning outside school,
• Competency-based progression,
• Flexible use of time, space, and technology, and
• Developing academic and non-academic career and college readiness skills.

The researchers found evidence for the promise of personalized learning, and they based this conclusion on analyses comparing achievement data from students in 62 schools implementing personalized learning with students in a matched “virtual comparison group.” Specifically, they found that implementing personalized learning approaches was associated with higher scores on a common assessment.

The reviewers pointed out, however, that two of the factors associated with positive learning gains—student grouping and making flexible use of learning spaces—do little to distinguish these schools from many other schools that may not claim to be implementing personalized learning. In fact, only the practice of engaging students in analyzing their own data showed a consistent relationship to positive outcomes.

More broadly, the study lacked a threshold for what qualified as implementing personalized learning in the treatment schools. This is particularly important because some of the strategies that require the most disruptive strategies and the largest departures from current practice, such as competency-based progression, were rarely implemented in the studied schools.

The study also lacks broad utility for judging the value of personalized learning because of the special characteristics of schools in the sample, Penuel and Johnson explained. Charter schools represented 90% of the sample used to analyze student achievement results, and nearly half came from one of the three initiatives studied.

The reviewers conclude that readers should be skeptical of what promise the report’s evidence actually provides for any given model of personalized learning being promoted or considered. The study does suggest that some practices are associated with some test score gains, but those practices may be quite different from those promoted under the flag of personalized learning.