Is there a connection between teacher value-added scores and student lifetime earnings?

Although policymakers may grab onto easy answers, questions about teacher effectiveness—how we measure it and what we can conclude about a teacher’s long-term impact—are being heatedly debated among scholars.

Today, the National Education Policy Center published a clear and detailed response to some of the most influential research claims about teacher effectiveness.

Those claims were made by researchers Raj Chetty, John Friedman, and Jonah Rockoff, who assert a connection between teachers’ “value-added” scores and what their students will earn over their lifetimes. Those assertions made their way into President Obama’s State of the Union message two years ago; they also surfaced in this month’s ruling by a California judge, in the Vergara case, who found that certain due process protections for teachers violated that state’s constitution.

That research, however, cannot bear the weight of critical scrutiny, according to an expert review by Moshe Adler published by the NEPC in April.

Adler’s review—published by NEPC as part of its Think Twice think tank review project—examined two working papers presenting research by Chetty and his colleagues and published by the National Bureau of Economic Research. Adler is an economist affiliated with both Columbia’s Urban Planning Department as well as the Harry Van Arsdale Jr. Center for Labor Studies at Empire State College, SUNY, and the author the 2010 book, Economics for the Rest of Us: Debunking the Science That Makes Life Dismal.




Chetty, Friedman, and Rockoff wrote a response—which is posted on the NEPC website—taking issue with the criticisms. Accordingly, we asked Adler to continue this important debate, replying to those responses. Adler has now presented an item-by-item explanation of why those responses are inadequate to address the study’s weaknesses.

The influence of the Chetty team’s research includes this statement in President Obama’ 2012 State of the Union address: “We know a good teacher can increase the lifetime income of a classroom by over $250,000.” Similarly, the judge in California’s Vergara litigation cited Chetty’s testimony and the team’s research as evidence that “a single year in a classroom with a grossly ineffective teacher costs students $1.4 million in lifetime earnings per classroom.”

The research claim rests on complex statistical analyses that attempt to attribute a student’s test score growth over a year to one or more of that student’s teachers—and then attempts to link those scores to subsequent student earnings.

According to Adler, however, the judge’s and the president’s conclusions step far beyond what the research can validly demonstrate. Adler explains why the evidence that Chetty and his colleagues cite cannot adequately support either claim: that a teacher’s value-added scores reflect that teacher’s quality, or that the scores predict future student earnings.

“Despite widespread references to this study in policy circles, the shortcomings and shaky extrapolations make this report misleading and unreliable for determining educational policy,” Professor Adler concluded in his April 10 review. That review raised at least nine different concerns, including the improper use of prior research and the failure to report important results when those results contradicted  the authors’ conclusions.

The points Adler makes should give pause to policymakers who have assumed that the evidence is sufficiently solid to be relied upon to frame education reform strategies.

Although the concerns Adler discusses are sometimes technical, they are very clearly explained. Policy makers and researchers considering using the study from Chetty and his colleagues are strongly encouraged to read this robust exchange, which brings forth and clarifies serious and important issues.



Find Moshe Adler’s review ; the Chetty, Friedman, and Rockoff reply; and Adler’s response to their reply, on the NEPC website at:
http://nepc.colorado.edu/ thinktank/review- measuring-impact-of-teachers.

Find Measuring the Impacts of Teachers, by Raj Chetty, John N. Friedman, and Jonah E. Rockoff, on the web at:
http://www.nber.org/papers/w19423 and
http://www.nber.org/papers/w19424.