"The Efficiency Index" Makes No Sense

A recent report purporting to score and rank national education systems on efficiency has drawn extensive media attention in both Europe and North America. But a new review published today explains that the report has serious problems and generates extreme conclusions and unrealistic policy proposals.
 
Professor Clive Belfield reviewed The Efficiency Index, written by Peter Dolton, Olivier Marcenaro-Gutiérrez, and Adam Still and published by GEMS Education Solutions, based in London, England. Belfield’s review is published today by the Think Twice think tank review project of the National Education Policy Center, housed at the University of Colorado Boulder School of Education.
 
Belfield is an economist at Queens College in New York whose research focuses on resource allocation and cost-effectiveness.
 
The Efficiency Index ranks 30 countries on their educational system “efficiency” through a model that compares national test scores, national teacher wage rates, and pupil-teacher ratios. The test scores used are from the Programme for International Student Assessment (PISA).
 
“Looking across the 30 countries, the model predicts that, in order to get a 5% increase in PISA scores, teacher wages would have to go up by 14% or class sizes would have to go down by 13 students per class,” Belfield writes. “But the optimal wages and class sizes for any given country may sometimes demand an increase or decrease in one or the other factor.”
 
Based on its model, the report identifies those wage levels and class sizes that are optimally efficient for each country. Those optimal levels, Belfield notes, are sometimes surprisingly extreme. Switzerland, for example, would have to cut wages nearly in half to achieve its “optimal” teacher salary, while Indonesia would have to triple teacher wages. “For four countries, the optimal class size is estimated at fewer than two students per teacher,” Belfield writes.
 
Such anomalies expose the weaknesses in each of the study’s three key elements, Belfield says: “the output measure is questionable, the input measures are unclear, and the econometric method by which they are correlated does not have a straightforward economic interpretation.”
 
Consequently, the report does nothing more than “satisfy an apparent keenness for reports that rank countries – and especially for reports that castigate low-rank countries,” Belfield writes – but it fails, he concludes, to advance an understanding on how to make education more efficient.