Using State Assessments to Measure Student Achievement in Evaluations of Educational Interventions

Ω

State assessments provide a relatively inexpensive and increasingly accessible source of data on student achievement. In the past, rigorous evaluations of educational interventions typically administered standardized tests selected by the researchers ("study-administered tests") to measure student achievement outcomes. Increasingly, researchers are turning to the lower cost option of using state assessments for measures of student achievement.

This NCEE Reference Report, Estimating the Impacts of Educational Interventions Using State Tests or Study-Administered Tests, identifies and describes factors that could affect the precision of impact estimates when evaluations use state assessments instead of study-administered tests. The study is based on data from three randomized controlled trials.

The authors found that the impact estimates based on state assessments were not significantly different from the impact estimates based on study-administered standardized tests. However, they found significant differences in the precision of the impact estimates from models that used different combinations of the two types of tests to provide pre-test scores (scores from a test administered before the implementation of the intervention) and post-test scores (scores from a test administered after the implementation of the intervention). In particular, the authors found that:

* Models that used the same test for both the pre- and post-test yielded more precise impact estimates than do models using different tests (e.g., obtaining pre-test scores from state assessments and post-test scores from a standardized, study-administered test).
* Models with two pre-test covariates, one from each type of test (state assessment and study-administered standardized test), yielded more precise impact estimates than models with a single pre-test covariate from one of the two types of tests.
* Models that specified the dependent variable as the simple average of the post-test scores from the two types of tests yielded more precise impact estimates and smaller sample size requirements than did models that based the dependent variable on post-test scores from only one of the two types of tests.

Ω