MOOCs Not Yet Meeting High Expectations, New Study Finds

The rollout of MOOCs, or massive open online courses, three years ago by some of the country’s leading universities triggered predictions that bricks-and-mortar campuses would soon be obsolete and that learning would be forever changed.

Well, maybe—but not any time soon, according to one of the first comprehensive studies of MOOCs from the perspective of institutions, released on May 15 by researchers at Teachers College, Columbia University. The study is based on 83 interviews with faculty members, administrators, researchers, and other actors in the MOOC space from 62 institutions, mostly in the U.S. It includes 13 case studies to illustrate how MOOCs are successfully being used to address institutional goals.

The study finds that a primary goal for institutions offering MOOCs is to extend institutional reach and access to education. “MOOCs are providing educational opportunities to millions of individuals across the world,” write Fiona M. Hollands (Ph.D. ’03) and Devayani Tirthali (Ed.D. ’13, Ed.M. ’12), respectively of the College’s Center for Benefit-Cost Studies of Education and the Institute for Learning Technologies, in their report, “MOOCs: Expectations and Reality.” However, to date, “most MOOC participants are already well-educated and employed.” Consequently, “the evidence suggests that MOOCs currently are falling far short of ‘democratizing’ education and may, for now, be doing more to increase gaps in access to education than to diminish them.”

Using MOOCs to “build and maintain brand” is another frequently mentioned institutional goal, but while MOOCs often generate media attention, “isolating and measuring impact of any new initiative on brand is a difficult exercise,” the report suggests. Indeed, increasing access to online offerings and enhancing brand may be contradictory goals, because the former can be seen as diminishing the selectiveness of the offering institution.

Hollands and Tirthali report that it is still too early to know whether MOOCs can live up to the hype of providing a cost-effective means for producing better educational outcomes on a mass scale. Cost analyses of MOOC production and delivery at four different institutions found that costs ranged from $39,000 to $325,000 per MOOC. “MOOCs have, so far, proved to be a significant drain on time and money for institutions,” Hollands and Tirthali write. That picture could change as institutions reuse MOOC materials, share them with each other, develop common courses, replace on-campus courses with MOOCs, and save on faculty teaching time and facilities costs. Revenue streams from MOOCs have been slow in materializing. Unless costs of MOOC production can be recovered through fees, Hollands and Tirthali speculate that “free, non-credit bearing MOOCs are likely to remain available only from the wealthiest institutions that can subsidize the costs from other sources of funds.”

As for improving learning outcomes, MOOCs, on the whole, cannot yet make that claim. “While interviewees provided many examples of how MOOCs have been used to change instruction, for the most part, actual impact on educational outcomes has not been documented in any rigorous fashion,” the report asserts.

However, two cases highlighted in the report provide examples of positive effects on student performance as a result of adopting MOOC-inspired strategies such as frequent assessment and automatic feedback, or of integrating MOOCs into flipped on-campus courses.

Hollands and Tirthali conclude that “while the potential for MOOCs to contribute significantly to the development of personalized and adaptive learning is high, the reality is far from being achieved.” To get there, “a great deal of coordination and collaboration among content experts, instructors, researchers, instructional designers, and programmers will be necessary.”

Hollands and Tirthali make several recommendations to institutions for increasing the value of MOOCs to improve access and educational outcomes, and to reduce the costs of higher education. These include:

• Identifying multiple channels of communication to advertise MOOCs to less connected audiences, and providing more instructional scaffolding to serve less educated participants

• Assessing the impact of MOOC pedagogy on educational outcomes by conducting pre- and post-assessment of participant skills and knowledge

• Developing metrics to assess gains in cognitive and non-cognitive skills that can be applied outside the MOOC environment

• Finding ways to confer economic value on MOOC completion, such as providing employer-recognized credentials

• Using MOOCs to substitute standardized courses across multiple campuses, or for continuing professional development and certification

• Working towards standardization of data formats across online learning platforms in order to facilitate research

• Establishing an accreditation system to evaluate MOOCs and other non-degree-based learning experiences to allow learners to accumulate a portfolio of credentials.