WHAT SHOULD WE REALLY LEARN FROM NEW ORLEANS AFTER THE STORM?

In July of 2018, the Education Research Alliance for New Orleans released a comprehensive, summative longitudinal report on the effects on student outcomes of the package of reforms  implemented in New Orleans following hurricane Katrina in the fall of 2005. 

This policy brief reviews the findings of this report,  offers critique of their methods and interpretation of findings and attempts to provide broader  policy context for those findings.   

In summary, Harris and Larsen find significant positive effects of Post-Katrina New Orleans  school reforms on short-term student achievement measures, and longer term college attendance,  persistence and completion:

We find that the package of reforms improved the quantity, quality, and equity of  schooling in the city on almost every available measure, increasing average test scores by  0.28-0.40 standard deviations, high school graduation by 3-9 percentage points, college  attendance by 8-15 percentage points, college persistence by 4-7 percentage points. 

They attribute these results to the “market-based” reforms adopted  following Katrina, and go to great lengths to dismiss or downplay threats to the validity of this  conclusion.

But for many reasons, that attribution may be misguided.

 a) First, the authors downplay the potential influence of significant changes in the  concentration of poverty across neighborhoods and schools—specifically the reductions  in extreme poverty which may contribute significantly to the improved student outcomes  in the years following Katrina;  

b) Second, the authors understate the importance of the substantial increases to funding  which occurred concurrently with organizational and governance changes in the district,  specifically disclaiming the importance of increased funding by suggesting that the  funding increases would not have existed but for the reforms;  

c) Third, the authors argue, without evidence, that similar funding increases provided to the  old, New Orleans school system would not likely have had similar impact, claiming they  would have been inefficient or wasteful. At the same time the authors sidestep the fact  that much of the funding increase in the new system was allocated toward increased and  duplicative overhead expenses, as well as increased transportation costs resulting from  citywide choice;  

d) Fourth, the authors define the treatment as the package of market-based reforms, which  are largely changes to the governance and organization of New Orleans schools, rather  than focusing on the types of schools, programs and services, and qualifications of  incoming staff who entered this marketplace.