Taking PISA Seriously: How Accurate are Low Stakes Exams?

PISA is seen as the gold standard for evaluating educational outcomes worldwide. Yet, as it is a low-stakes exam, students may not take it seriously resulting in downward biased scores and inaccurate rankings. This paper provides a method to identify and account for non-serious behavior by leveraging information in computer-based assessments in PISA 2015. 

The authors show that this bias is large: a country can rise up to 15 places in rankings if its students took the exam seriously. They ask where the bias is coming from and show that around half of it comes from the proportion of non-serious students, while 36% comes from their ability, with the remaining coming from the extent of non-seriousnes.