Test students early and often (online at least)

The number of online educational offerings has exploded in recent years, but their rapid rise has spawned a critical question: Can such “virtual” classes cut through the maze of distractions — such as email, the Internet, and television — that face students sitting at their computers?

The solution, Harvard researchers say, is to test students early and often.

By interspersing online lectures with short tests, student mind-wandering decreased by half, note-taking tripled, and overall retention of the material improved, according to Daniel Schacter, the William R. Kenan Jr. Professor of Psychology, and Karl Szpunar, a postdoctoral fellow in psychology. Their findings are described in a paper published this week in the Proceedings of the National Academy of Sciences.

“What we hope this research does is show that we can use very strong, experimentally sound techniques to describe what works in online education and what doesn’t,” said Szpunar. “The question, basically, is how do we optimize students’ time when they’re at home, trying to learn from online lectures? How do we help them most efficiently extract the information they need?

“Some students I’ve talked to say that it takes them as long as four hours to get through an hourlong, online lecture because they’re trying to combat all the distractions around them,” he continued. “If we give students an incentive to pay attention to what they’re doing, it’s going to save them time. This is one way to do that.”

Ironically, Schacter said, while online classes have exploded in popularity in the past few years, there remains “shockingly little” hard scientific data about how students learn in the virtual classroom.

“A lot of people have ideas about what techniques are effective,” he said. “There’s a general folk wisdom that says lessons should be short and engaging, but there’s an absence of rigorous testing to back that up.”

To get at that question, he, Szpunar, and research assistant Novall Khan devised two experiments.

In the first, a group of students was asked to watch a lecture that had been broken up into four segments of about five minutes each. After each segment, students were asked to do several math problems. Some students were then tested on the material from the lecture, while a control group did more math problems.

In the second experiment, participants were separated into three groups. Similar to the first experiment, all began watching a lecture that had been broken up into four segments. The difference was that students were interrupted, and asked whether their minds were wandering.

“It was surprising how high the baseline tendency to mind-wander is,” Schacter said. “In our experiments, when we asked students if they were mind-wandering, they said yes roughly 40 percent of the time. It’s a significant problem.”

Following each segment, all three groups again did a set of math problems. Some students were then tested on the lecture, some did more math problems, and some were given the chance to study material from the lecture a second time.

Surprisingly, Schacter said, in both experiments, students who were tested between each segment — but not the others, even those who were allowed to study the material again — showed a marked drop in mind-wandering and improved overall retention of material.
“It’s not sufficient for a lecture to be short or to break up a lecture as we did in these experiments,” Schacter said. “You need to have the testing. Just breaking it up and allowing them to do something else, even allowing them to re-study the material, does nothing to cut down on mind-wandering, and does nothing to improve final test performance. The testing is the critical component.”

Those tests, Schacter and Szpunar believe, act as an incentive for students to pay closer attention to the lecture because they know they’ll have to answer questions at the end of each segment.

“Whether it’s in the classroom or online, students typically don’t expect to have to summarize a lecture in a way that makes sense until much later on,” Szpunar explained. “But if we give them an incentive to do that every now and then, students are actually much more likely to set everything else aside, and decide they can get to that text after class, or they can worry about their other class later, and they’re able to absorb the material much better.”

Another surprising effect of the testing, Szpunar said, was to reduce testing anxiety among students, and to ease their fears that the lecture material would be very challenging.
Going forward, Schacter said, he hopes to research whether the testing effect can also reduce mind-wandering in the classroom.

“We know that there is mind-wandering in classroom lectures,” he said. “Testing intervention hasn’t been tried yet, but I think both Karl and I expect it would have similar, and possibly even stronger, results, because these experiments were conducted in a very controlled setting.”

As online courses are increasingly touted as a large part of the future of higher education, Szpunar said he hopes the findings help to lay out a blueprint that can ensure students get the most out of such studies.

“At the very least, what this says is that it’s not enough to break up lectures into smaller segments, or to fill that break with some activity,” he said. “What we really need to do is instill in students the expectation that they will need to express what they’ve learned at some later point. I think it’s going to be a very sobering thought for a lot of people to think that students aren’t paying attention almost half the time, but this is one way we can help them get more out of these online lectures.”