Study Finds Rigorous Classroom Observations Can Identify Effective Teachers

Ω


Cincinnati’s teacher evaluation system pinpoints link between teaching practices and student achievement

A new study of Cincinnati’s Teacher Evaluation System (TES), a rigorous evaluation program based on classroom observations, finds that teachers receiving high ratings (as scored by trained peer and administrative evaluators) are more effective in promoting student achievement growth. For example, a student who begins the year at the 50th percentile on the state reading and math test and is assigned to a teacher in the top quartile in terms of overall TES scores will perform on average, by the end of the school year, three percentile points higher in reading and two points higher in math than a peer who began the year at the same achievement level but was assigned to a bottom-quartile teacher.

By way of comparison, the authors note that the impact of being assigned to a teacher in the top-quartile rather than one in the bottom quartile in terms of their total effect on student achievement as measured by student-test-based measures of teacher effectiveness is seven percentile points in reading and six points in math. In other words, the observed teacher practices included in the TES evaluation system appear to capture a little less than half of the overall differences in teacher effectiveness.

These results, based on a study by a team of scholars at Harvard, Brown and Stanford universities, are reported in the Summer 2011 issue of Education Next. Their findings from Cincinnati offer new evidence that “evaluations based on well-executed classroom observations do identify effective teachers and teaching practices.” During the yearlong TES process, teachers are typically observed and scored four times: three times by a peer evaluator external to the school and once by a local school administrator. Both peer evaluators (experienced classroom teachers who serve as full-time evaluators for three years) and administrators must complete an intensive training course and accurately score videotaped teaching examples according to a specific rubric.

The authors point out that the Cincinnati system of evaluation is different from the standard practice in place in most American school districts, where perfunctory evaluations assign the vast majority of teachers “satisfactory” ratings, leading many to “characterize classroom observation as a hopelessly flawed approach to assessing teacher effectiveness.”

The study’s results are based on a sample of 365 teachers in reading and 200 teachers in math. The researchers analyzed records of each TES classroom observation conducted by the Cincinnati district between the 2000-01 and 2008-09 school years. In addition to TES observation results, the researchers analyzed students’ demographic, program participation, and test score data from the 2003-04 through 2008-09 school years. For all teachers in the sample, the average score on the Overall Classroom Practices index (a teacher’s average score across eight standards of teaching practice) was 3.21 (between “Proficient” and “Distinguished” categories), yet one-quarter of teachers received an overall score higher than 3.53 and one-quarter received a score lower than 2.94, indicating, the authors note, that “there is a fair amount of variation from teacher to teacher.”

The researchers also used teachers’ scores on particular elements considered by the TES observation system to discern relationships between more specific teaching practices and student outcomes across academic subjects. For example, among students assigned to different teachers with similar overall TES scores, math achievement will grow more for those students whose teacher scores relatively better on the classroom management portions of the TES observations. They note that the data gleaned from the TES observations “allow researchers to connect specific teaching practices with student achievement outcomes, providing evidence of effective teaching practices that can be widely shared.”