Policy & Funding

Report: School Improvement Grant Program Results Based on 'Underpowered' Research

• By Dian Schaffhauser
• 10/08/18

FutureEd has called the federal government to task for dropping support for the School Improvement Grant program. The SIG program, begun during the Obama presidency, with a total $7 billion investment in about 1,250 of the lowest-achieving public schools, was later called a failure by the Education Department's Institute of Education Sciences. A 2017 research project found no significant gains when comparing students in SIG schools to students in similar schools without the block grant support.

According to FutureEd, current Education Secretary Betsy DeVos "is quick to mention this study when she says there is no point putting more money into struggling public schools."

However, a subsequent review of data by FutureEd Senior Fellow Marshall Smith and researcher Alan Ginsburg concluded that the federally funded SIG study "was wrong to conclude that SIG didn't make a difference." Their assertion in a new report: The 2017 study used a sampling of schools that "was too small to draw dependable conclusions" and that, in comparison to the overall SIG program, it was skewed to urban schools and students from disadvantaged backgrounds. "In research parlance, the study was severely 'underpowered,'" the report stated.

 

FutureEd's own review of the IES-funded study and analysis of 17 separate state and local reports, which examined a collective 449 SIG schools, found that in 12 of the 17 studies, schools made "significant improvements in student achievement." Gains surfaced in math, reading or high school graduation compared to non-SIG schools.

The primary concern for FutureEd is that the school improvement funding going out to states and localities under the Every Student Succeeds Act will ignore lessons already learned during the SIG era in turning around states' lowest performing schools. The report's suggestion: that the Education Department follow a different approach in evaluating the performance of the ESSA efforts and other federal projects, by putting more emphasis on studies done at the state and local level.

As Marshall Smith, one of the researchers on the FutureEd report, noted, "We learned from the state and local studies that many of the successful SIG schools used well researched strategies as their interventions. These are the lessons we should be sharing."

"Revisiting SIG: Why Critics Were Wrong to Write Off the Federal School Improvement Grant Program" is openly available on the FutureEd website.