Variability in Pretest-Posttest Correlation Coefficients by Student Achievement Level

Ω

State assessments are increasingly used as outcome measures for education evaluations and pretest scores are generally used as control variables in these evaluations. The correlation between the pretest and outcome (posttest) measures is a factor in determining, among other things, the statistical power of a study.

This report examines the variability in pretest-posttest correlation coefficients for state assessment data on samples of low-performing, average-performing, and proficient students to determine how sample characteristics (e.g., achievement level) affect pretest-posttest correlation coefficients.

As an application, this report illustrates how statistical power is affected by variations in pretest-posttest correlation coefficients across groups with different sample characteristics. Achievement data from four states and two large districts are examined.

The results confirm that pretest-posttest correlation coefficients are smaller for samples of low performers than for samples representing the full range of performers, thus, resulting in lower statistical power for impact studies than would be the case if the study sample included a more representative group of students.