Tuesday, February 21, 2012

Evaluation of the Effectiveness of the Alabama STEM Initiative


Full report

Partly motivated by the 1996 National Assessment of Educational Progress scores, which were below the national average for Alabama’s grade 4-8 students in mathematics and grade 8 students in science, the Alabama State Department of Education (ALSDE) developed a statewide initiative to improve mathematics and science teaching and student achievement in kindergarten through grade 12 (K-12). The Alabama Math, Science, and Technology Initiative (AMSTI) is a two-year intervention intended to better align classroom practices with national and statewide teaching standards—and ultimately to improve student achievement—by providing professional development, access to materials and technology, and in-school support for teachers.

The effect of AMSTI on student achievement in mathematics after one year, as measured by end-of-the-year scores on the Stanford Achievement Test Tenth Edition (SAT 10) mathematics problem solving assessment of students in grades 4–8, was 2.06 scale score units. The difference of 0.05 standard deviation in favor of AMSTI schools is equivalent to a gain of 2 percentile points on the SAT 10 mathematics problem solving assessment for the average control group student had the student received AMSTI. The 0.05 standard deviation is statistically significant but smaller than the effect the research team believed would be detectable by the experiment as designed. Whether this size effect is educationally important is an open question. It may be useful to convert this effect into a more policy-relevant metric—additional student progress measured in days of instruction. In these terms, the average estimated effect of AMSTI was equivalent to 28 days of additional student progress over students receiving conventional mathematics instruction.

The effect of AMSTI on student achievement in science,as measured by end-of-the-year scores on the SAT 10 science assessment, required only in grades 5 and 7, was not statistically significant after one year.


No comments: