Research
Study: Tech-Enabled Early Warning Systems Can Have Positive Impact on Chronic Absenteeism and Course Failure Rates
A controlled study involving 73 schools and more than 37,000 students found that early warning systems can have a statistically significant positive impact on student outcomes in K–12 schools, even when those systems are not used to their full potential.
The report, "Getting students on track for graduation: Impacts of the Early Warning Intervention and Monitoring System after one year," was prepared by the Regional Educational Laboratory (REL) Midwest for the United States Department of Education's Institute of Education Sciences (IES), National Center for Education Evaluation and Regional Assistance. The study was administered by the American Institutes for Research.
For the study, researchers used the Early Warning Intervention and Monitoring System (EWIMS). Thirty-seven schools (18,634 students) were assigned to used EWIMS in the 2014–2015 school year; the remainder delayed implementation and were used as a control group.
As the report's authors described it: "EWIMS is a systematic approach to using data to identify students who are at risk of not graduating on time, assign students flagged as at risk to interventions, and monitor at-risk students' response to intervention. The EWIMS model provides schools with guidance to implement a seven-step process, supported by the use of an early warning data tool. The tool uses validated indicators, based on prior research, to flag students who are at risk of not graduating on time ... and allows schools to assign students to interventions and monitor their progress. The indicators used to flag at-risk students in the tool are chronic absence (missed 10 percent of instructional time or more), course performance (failed any course, grade point average below 2.0), behavioral problems (suspended once or more), and an off-track indicator (failed two or more semester-long or three or more trimester-long core courses or accumulated fewer credits than required for promotion to the next grade). The EWIMS model is intended to help schools efficiently use data to identify at-risk students and provide targeted supports."
The study found three statistically significant impacts from the implementation of EWIMS:
- Chronic absenteeism was four percentage points lower in EWIMS schools than in control schools (10 percent versus 14 percent);
- The percentage of students failing one or more courses was five percentage points lower in EWIMS schools (21 percent versus 26 percent);
- Average GPAs were higher in EWIMS schools (2.98 versus 2.87).
The two percentage point difference between students with GPAs below 2.0 (17 percent in EWIMS schools and 19 percent in control schools) was not statistically relevant.
One of the noteworthy aspects of the study was that the implementation of EWIMS was "low" at the school level. In other words, even a poor implementation of the strategy proved valuable in a statistically significant way.
"In nearly all participating schools, overall implementation of the EWIMS seven-step process was low, and implementation was challenging. Nevertheless, EWIMS schools were more likely than control schools to report using an early warning system and having a dedicated team to identify and support at-risk students, but EWIMS schools did not differ from control schools in the frequency of data review or the number and type of interventions offered."
The researchers said implementation of EWIMS did not have a detectable impact on a school's "data culture," or the "ways in which schools use data to make decisions and identify students in need of additional support."
The complete report is available on the Institute of Education Sciences site at ed.gov.