By Mark Schneider, Director of IES | November 2, 2022
On Monday, October 24, 2022, the National Center for Education Statistics released the latest NAEP results, covering reading and math in grades 4 and 8. As is the norm with "Main NAEP" results, lots of data were included in the release and even more data are available via the NAEP Data Explorer. I want to highlight just a few of the findings that strike me as particularly important for understanding where the nation stands academically as the COVID-19 pandemic begins to recede in our rear-view mirror.
The largest declines recorded in NAEP history. As foretold by other assessments, including NAEP's Long Term Trends, the results starkly illustrate the COVID-19 pandemic's negative effect on students' academic performance. Fourth grade math scores are down 5 points, putting the nation about where it was in 2003. Eighth grade math sank even further, by 8 points, taking us back to around 2000. Reading scores were also down, but not by as much: 3 points in both grades 4 and 8.
Students struggling before are really struggling now. Continuing one of the most troubling trends in recent NAEP assessments, scores for grade 4 students at the bottom of the score distribution often fell far more than those of high-performing students. For example, in grade 4 math, the scores of students in the bottom decile fell by 7 points, while those of students in the top decile fell by only 2 points. In reading, grade 4 students in the lowest decile saw their scores fall by 6 points, while students in the top decile saw no decline.
Many of you know that I am particularly concerned about students performing below NAEP's "basic" level. There were significant increases in the proportion of students scoring below NAEP Basic for every combination of grade and subject. Consider that between one-quarter and over one-third of America's students are below basic in core reading and math skills.
Percentage of students scoring below NAEP Basic
| 2019 | 2022 | |
|---|---|---|
| Grade 4 Math | 19% | 25% |
| Grade 8 Math | 31% | 38% |
| Grade 4 Reading | 34% | 37% |
| Grade 8 Reading | 27% | 30% |
NAEP captures a moment in time—and this is no normal moment. Here's an important caveat: NAEP ended its data collection in March 2022. There is some evidence that students were recovering from the pandemic-induced learning loss as schools reopened and more normal teaching conditions resumed in the spring of 2022. (See for example here, here, or here.) While NAEP is an accurate snapshot of student performance early in 2022, state tests administered later than March may reflect recovery occurring later in the school year.
Our nation's best tool for state-to-state comparisons reveals losses across the board. One of Main NAEP's most important contributions is its state-by-state reporting—and, while these data can provide insights into the effects of policy on student performance (see most recently the growth in NAEP scores in Mississippi), the "horse race" between the states always garners lots of attention.
The big show, for me, is always the matchup involving some of the nation's largest states, including California, Texas, New York, and Florida. But it turns out that the state-by-state results are so uniform that it's hard to find winners and losers in almost any matchup across states.
The breadth of the declines across states is breathtaking.
Given such widespread declines across the states, it is hard to see how state policy regarding the timing and duration of school closings affected NAEP scores. That doesn't mean that these policies didn't matter; rather, it is likely that NAEP state scores are simply too blunt an instrument to pick up any impact.
Emily Oster, one of the nation's most careful researchers tracking school opening during COVID, has generously shared her school-by-school data with IES. Our plan is to merge these data with NAEP school level assessment data plus the detailed school, student, and teacher data NAEP collects to create a large and comprehensive dataset to help researchers better evaluate the relationship between a wide variety of school contexts and policies and NAEP scores.
Prepare for over-interpretation. There has been and will continue to be lots of analysis and commentary driven by these latest results, since they are the first Main NAEP results post pandemic (NCES released NAEP's Long Term Trends results for 9-year-olds in late August). That this release comes just a few weeks before elections may attract much attention to the state-by-state results in search of potential political messaging, especially in states where there are gubernatorial elections. NAGB, NAEP's governing board, debated whether to postpone releasing this year's NAEP data until after the November elections, but decided that the risks of NAEP being politicized were outweighed by NAEP's responsibility to make these data available as quickly as possible (indeed, one of the most compelling arguments for not delaying the release was that such a delay could by itself be viewed as a political decision).
But as you read (or perhaps offer) your hot takes, remember that the NAEP results do not reveal causality for any one pet theory about how one or another policy affected learning. We are likely looking at years of research on how the pandemic has affected student outcomes; the test results released this week will not provide any clear answers on which policies helped or harmed students the most.
This is why IES was built. We do not stop at simply measuring learning—we work to improve it. This week marks IES' 20th anniversary, and we will begin our celebration of two decades of learning about what works for whom under what conditions. More importantly, we will be inviting educators, policymakers, researchers, and others to join us in looking ahead to the future of IES. The Institute and our nation are facing challenges unlike any we've ever seen. It is crucial that we approach this moment with the right tools, the right people, and the urgency our circumstances demand. We need both your best efforts and your best wishes as we confront this challenge.
As always, feel free to reach out to me at mark.schneider@ed.gov with comments or suggestions for ways in which IES can help speed the academic recovery of our nation's students.
No comments:
Post a Comment