Policy & Accountability

How State Accountability Systems May Overlook Low Performers

• By Dian Schaffhauser
• 12/01/20

Even as education leaders are encouraged to look at the data to understand which pockets of students need special kinds of support for their learning, a new article and infographic from a research organization have suggested that some "subgroups" of students are too small to register on the radar, which means they get passed over.

A project described by the Regional Educational Laboratory Program set out to understand why some states had a disproportionate number of middle schools with low-performing students with disabilities. In one state, according to the project, middle schools accounted for two-thirds of all schools targeted for improvement under the rules of the Every Student Succeeds Act. As a result, those schools received additional support from the state to help those subgroups improve. But what about the same subgroups in elementary school or high school? How come they weren't targeted for extra help too?

The problem is tucked into the process states may use to identify the "Targeted Support and Improvement" (TSI) schools. Each state comes up with a plan for identifying those schools that underperform through their accountability systems. Those systems typically look at academic achievement, progress and graduation rates within their schools, among other aspects. Each state sets a minimum number of students that each school and subgroup must meet for each performance element before that element is included in the overall accountability score. Schools are tagged for TSI when their subgroup accountability scores are low compared to the overall student population in the state.

The study found that those middle schoolers with disabilities didn't perform "substantially and consistently worse" than the ones in lower or upper grades. However, the schools they attended were "much more likely" to have a sufficient number of students with disabilities taking the state exams to meet state-set minimum thresholds. That meant the subgroups' proficient rates counted more often toward those schools' accountability scores.

The researchers concluded that the sample sizes in elementary and high schools were just too small, thereby masking poor performance.

The article and infographic offered two ways states can overcome this blind spot:

To update their accountability systems so schools "are only compared with other schools that meet minimum sample size requirements for the same performance dimensions"; and

To incorporate statistical techniques to make the accountability scores or small sample sizes more precise.

The coverage of the project is openly available as a blog article and infographic on the REL Mid-Atlantic website.