Review Explains Weaknesses in Teach for America Evaluation

The controversial organization Teach for America, which places recent college graduates as teachers in low-income communities, recently funded a study by Edvance Research to assess the effects of TFA teachers and alumni on student test scores in Texas.

While the report found some significant improvements in the scores of middle school students taught by TFA alumni (those who continued teaching after their obligatory 2-year TFA contract ended) and even novice TFA teachers, a new review urges caution in reading those findings.

The report, Evaluation of Teach for America in Texas Schools, was reviewed for the Think Twice think tank review project by Edward J. Fuller, an associate professor and Director of the Center for Evaluation and Education Policy Analysis in the Department of Education Policy Studies at Penn State University, and Nathan Dadey, a doctoral candidate at the University of Colorado Boulder. The review is published today by the National Education Policy Center, housed at the CU Boulder School of Education.

Fuller previously served as the Director for Research at the Texas State Board for Educator Certification and Program Director for Evaluation at the Charles A. Dana Center at The University of Texas at Austin.

The Edvance Research study matched TFA and non-TFA schools and students on demographic and achievement characteristics. It then used the matched student data in a multi-level regression analysis to estimate the effect of being taught by a TFA teacher on mathematics and reading test scores for two groups of students: those in grades 4 and 5 and those in grades 6 through 8, performing separate analyses for TFA teachers and TFA alumni – resulting in a total of eight separate analyses (e.g., TFA alumni in math for grades 4-5 vs. non-TFA teachers in math for grades 4-5).

Of the eight comparisons performed, three were statistically significant: mathematics and reading scores of middle-level students taught by TFA alumni, and mathematics scores for middle-level students taught by novice TFA teachers. The elementary-level (grades 4-5) analyses all showed no significant difference between TFA and non-TFA teachers.

Fuller and Dadey find that while the findings appear large enough to be relevant to public policy, several issues related to the report’s statistical model make it likely that the actual size of the TFA teacher effects differ than what is found in the report. These issues relate to measurement error, the control variables and the unit of analysis.

The expert reviewers also have found cause for concern in sample construction, matching procedures and interpretations, all of which caution against attributing the apparently better outcomes to the presence of TFA teachers.

Because of those limitations, Fuller and Dadey caution that the report does not provide solid evidence that either TFA teachers or TFA alumni have a measured effect on student test scores. Instead, they suggest that the report indicates the value of future investigations more carefully examining the performance of the relatively small percentage of middle-level TFA teachers who remain after their initial two-year commitment.