What We Can Learn from New Orleans' Post-Katrina School Reforms

• By Dian Schaffhauser
• 08/20/18

Reforms put in place in New Orleans after Hurricane Katrina devastated the Gulf Coast in 2005 turned out to have many positive impacts, according to a new report published by the Education Research Alliance. Two researchers, Douglas Harris from Tulane University and Matthew Larsen from Lafayette College, found that the move to state-controlled charters boosted student achievement, high school graduation rates, college entry numbers, the college persistence rate and the college graduation rate.

However, the researchers cautioned, the results aren't necessarily replicable in other districts "because New Orleans had several advantages ... in making the reforms work." Nevertheless, they wrote, "the fact that New Orleans improved so much, in such a short period, on so many measures means that the city's experiences are worthy of attention."

Here's what took place.

Up until the hurricane, Orleans Parish Public Schools was a district in turmoil. It ranked 67th out of 68 Louisiana districts in math and reading test scores for the 2004-2005 school year. The high school graduation rate was 56 percent, about 10 points below the state average. An investigation by the Federal Bureau of Investigation had found financial malfeasance in the district. And eight superintendents had served between 1998 and 2005, each lasting an average of just 11 months on the job. A state-run recovery school district had already been put in place to take over the schools.

In the wake of the hurricane, the state turned the schools over to nonprofit organizations to run. According to the report, these charter schools had control of personnel decisions and nearly every other aspect and were held accountable to the state through performance-based contracts. Every teacher was fired and the teacher union contract was allowed to expire and never replaced. Also, the new system allowed families to choose schools from anywhere in the city instead of having them assigned by neighborhoods; the state funding was based "almost entirely on the number of students they attracted." The government role quickly switched from operating schools to overseeing them.

To derive their findings, the researchers used two approaches: doing an analysis of "returnees only," those students who returned to city schools after the hurricane, in order to compare the same students over time; and running a "cross-cohort" analysis, in which outcomes for whole schools in hurricane-affected districts throughout Louisiana were compared to those in New Orleans. The data was generated from the years 2001 to 2014.

The impact of the reforms was impressive in multiple areas:

• Student achievement increased dramatically, moving test scores from the 50th percentile to the 66th percentile (depending on the subject and analysis method);
• High school graduation grew by 3-9 percentage points;
• The college entry rate rose by 8-15 percentage points;
• The college persistence rate increased by 4-7 percentage points; and
• The college graduation rate expanded by 3-5 percentage points.

The project also examined how the reforms affected achievement gaps between advantaged and disadvantaged students — touching on a point that is often brought up by critics of market-based reforms, that "markets are designed for efficiency not equity."

To understand those aspects, Harris and Larsen carried out their analyses separately for Black and White students and for low-income (those eligible for free or reduced-price lunches) and non-low-income students.

Their findings: On all of the measures — test scores, graduation rates and college outcomes — the reforms "clearly benefited Black and low-income students." Of course, they pointed out, because more than 80 percent of students are from black or low-income families, the average effects were almost certain to reflect the results for disadvantaged groups.

Yet, the conclusions were murkier regarding the question of reducing achievement gaps. As they reported, "If we look only at test scores, the results would suggest that the reforms may have increased achievement gaps." However, the report added, the results "are consistently positive" when the focus is on other, longer-term outcomes. Because Black and low-income students experienced "larger positive effects" than White students with regard to high school graduation and college outcomes, the achievement gaps declined.

As they summarized, "The reforms increased outcomes on average and seem to have improved equity as well. Black and low-income students saw higher test scores, high school graduation rates, and college results, and probably reduced gaps on the latter two measures. Only the achievement gaps in test scores may have worsened."

Could other school districts somehow replicate the New Orleans outcomes for themselves? The researchers said they would consider that "unwise" for many reasons, among them:

• New Orleans became "school reform central." People came from all over the country "because they wanted to help the city rebuild." Its school reform work became a beacon that "attracted ambitious and talented people." Other cities wouldn't necessarily have that "first-mover advantage" in drawing the best and the brightest.
• The bar was low. Overhaul was a foregone conclusion, and it's much easier "to improve on clear failure." Other cities would probably "see smaller effects."
• New Orleans is a city. Reform works better in cities than suburban or rural areas, the researchers wrote, because "families will have more choices and there will be more competition between schools"; and the results stand out because student outcomes tend to be lower to begin with.

What is worth considering, the report added, is the role played by government, even in this "market-based reform" model. "The state could decide which schools entered the market, and because schools operated under performance contracts, the state could also force poor performers to close. In these respects, the power of government was different than before and still strong."

The state recovery school district, for its part, became more involved over time "in student enrollment and transfer, special education, school discipline and transportation." The schools maintained control over "curriculum, budgets, schedules and, critically, personnel." But it "was not a free-for-all either." Such divvying of responsibility makes the New Orleans successes more relevant to charter schools, which adhere to government rules, than to school voucher approaches, "which rely almost entirely on market forces."

"Ultimately, school reform is about serving students, their families, and the communities in which they live," the report concluded. This study was designed to focus on the first element: how students were affected by the reforms. Given that, the researchers wrote, "We hope this evidence is useful both locally and nationally, as education leaders work to provide all children with the education they deserve."

The full report, along with a technical report on the same research project, is available on the Education Research Alliance website.