Predictive Analytics in K-12: Advantages, Limitations & Implementation

• 06/12/14

Predictive analytics is growing rapidly in popularity among school district leaders. K-12 school districts are collaborating with universities and businesses (IBM, Dell) at an accelerating pace, using advanced analytics to create innovative new models and tools to advance students' performance. Many school districts across the nation are now using predictive analytics to predict and monitor student performance to improve learning.

School districts use predictive analytics in several ways:

1. To build early warning indicators based on students' attendance, course failure and behavior to predict dropouts^{1, 2};
2. To predict on-time high school graduation and being on track in Grade 9^3,4,5;
3. To examine indicators that predict college- and career-readiness and postsecondary success⁵;
4. Recently predictive analytics has also gained momentum in identifying and retaining great teachers⁶.

The purpose of this article is to clarify the definition of predictive analytics and describe advantages and limitations of predictive analytics and what school districts need to have to conduct such analysis. (Abundant news articles in recent years show school districts' use of predictive analytics in decision making. These articles typically describe specific uses of results of analytics at a specific system, but they do not discuss or describe the analysis process itself.) In addition, this article will provide some informational resources and examples of how school districts are utilizing predictive analytics results to improve instruction and resource allocation.

What Is Predictive Analytics?
Predictive models are mostly regression-based analyses conducted to examine which student-, classroom- and/or school-based indicators empirically predict student outcomes.

The statistical analysis uses a combination of potentially actionable indicators to predict an outcome that needs attention and improvement. Most prediction models tend to use lagging indicators, for example, using Grade 9 indicators to predict the likelihood of on-time graduation from high school; indicators in Grades 8-11 predicting the likelihood of college readiness and immediate college enrollment.

Indicators can include such measurable events as:

• Course grades;
• Attendance;
• Course level;
• Suspensions; and
• Extracurricular involvement.

But also less obvious data can be used, such as classroom observations and non-cognitive variables. Analyses provide a predicted score for each individual student so students can be grouped objectively into categories needing high-, medium- or no-intervention to succeed. Predictive analytics cannot say that the indicators caused the outcome but can show what combination of indicators is related to the outcome.

Advantages of Predictive Analytics
Indicators are generally data that school staff have significant control over and where conditions in the academic environment can be altered by the school staff. For example, enrollment in rigorous courses or student attendance are indicators that school staff can alter or influence.

Results of analytics assist district and school leaders and staff because those staffers can use data as the basis to inform their decisions. District leaders can use this information to monitor progress of individuals and groups, identify needs and align resources, develop and evaluate intervention programs to assist students and re-allocate resources to address problems and deficiencies more effectively. Change in policy can be based on sound, objective information developed out of analytics. In addition, school staff can use results in designing intervention strategies. Teachers can use these predictive analytics results to improve instructional efforts tailored to individual students. Teachers can have an informed conversation with parents on their child's progress. Teachers can talk to students about individual student's information and assist the student in setting goals.

With predictions one can actively approach potential problems. Predictive analytics can help teachers identify students at risk and monitor students' progress over time, thereby providing the necessary support and intervention to students that are in need.

Limitations of Predictive Analytics
Predictions are based on data available from a school district's database. Other factors such as family, peers, counselors and teachers can influence student progress and outcomes over time. Therefore, teachers and administrators should use predictive analytics as one resource but in combination with other factors/measures. For example, a student who might be identified as at-risk one year for family issues might perform better during the next school year when the issue is resolved.

Predictions are usually based on lagged indicators (historical data on an individual), for example, indicators at end of Grade 9 that predict students' on-time high school graduation three years later. Therefore, teachers should take into consideration student learning that has taken place during this time period, changes in student behavior, and other personal and social factors that might influence student academic outcome. Researchers should conduct periodic cross-validation studies and update the model as needed to ensure that the model accuracy is not compromised due to changes in student cohorts, course offerings, and instructional assessments. The advantages of predictive analytics outweigh the disadvantages and districts should be encouraged to pursue predictive analytics.

What Do School Districts Need To Conduct Predictive Analytics?
School districts must have a reasonably well developed data system with several years of data in various areas to form accurate predictions. Predictive analytics is ubiquitous throughout the business and actuarial industries but as a specialty practice has gotten traction in the education sector only relatively recently. Districts need to ensure that adequate and accurate data are loaded and retained in the system and that data are accessible and usable. Not every school district has extensive data warehouse capabilities to store student data.

School districts can outsource to firms that deal with "big data" and analytics. However, with budget cuts this might not be feasible at some districts. Instead, districts can earmark adequate technical human resources, that is, invest in researchers trained to conduct predictive analytics or train existing staff to use readily available predictive modeling software (for example, SPSS Modeler software by IBM). Training in relevant applied analytic methods is readily available. Non-profits with research branches and personnel who are developing models might play a role to collaborate with districts.

Not all school districts have the luxury of having researchers/data scientists and not all researchers/data scientists see school districts as an option to serve, since such educational predictive analytics has gained traction only recently. One strategy to recruit talent might be through the Strategic Data Project (SDP) Fellowship housed at the Center for Education Policy Research (CEPR) at Harvard University⁷. SDP recruits talented data strategists from multidisciplinary backgrounds (for example, education, engineering, sciences and policy) and places them in school districts and other educational agencies to conduct rigorous analyses using education data and share results and findings with key stakeholders to improve student outcomes.

Districts also need resources to build data dashboards (using staff effort, specialized expertise and software) and to automate the models and tools so teachers can have real-time data on students. Providing meaningful, actionable information to teachers at the right time is critical for student success. Districts need to put systems in place to train district and school leaders and staff on how to use the results. District leaders need to be trained on how to use their system's data for policy implications, principals on best resource allocations, and teachers and other staff on instructional support, interventions and effectively addressing individualized student needs.

Informational Resources and Tools for Predictive Analytics
I was closely involved in developing prediction models and application of those models to school settings. One such model used Grade 3 first semester indicators (course grade, attendance and suspension) that predicted Grade 6 first quarter marking period average³. These grade levels were chosen because they are the significant transitional periods developmentally for a child. Number of absences during first semester of Grade 3 had the strongest negative association with Grade 6 marking period average by end of quarter 1. Attendance can be altered by school staff. The model was piloted as an applied, predictive tool in five schools. One way the schools used the days absent indicator was to identify and monitor students who were chronically absent. This allowed quick follow-up with the attendance secretary as to the reason for students' absence thereby reducing parent follow-ups and thus staff time. Researchers could examine data after a year to see if this effort also helped reduce absences. This article provides a sample monitoring tool at the end that teachers can use. These articles are good reads for technical information^{4, 5}.

Following are some informational resources on predictive analytics tools and how the results are used by school staff:

• Driving A New Dialogue With Students: How Predictive Analytics Is Transforming Progression And Achievement For K-12 Educators
• Putting Student Data To The Test To Identify Struggling Kids
• Data Analytics Helps School Districts Serve Students Better
• A Crystal Ball for Student Achievement
• Dell Provides Schools and Universities With Predictive Analytics-Based Decision Support System To Help Improve Student Outcomes and School Efficiency
• Predictive Analytics Tackles a School District's Enrollment Problems
• Schools Find Uses for Predictive Data Techniques
• IBM Predictive Analytics Solution for Schools and Educational Systems

References

1. http://www.gse.harvard.edu/cepr-resources/files/news-events/sdp-fellowship-capstone-chen.pdf
2. http://www.gse.harvard.edu/cepr-resources/files/news-events/sdp-fellowship-capstone-keenan-marquez-pham.pdf
3. http://montgomeryschoolsmd.org/departments/sharedaccountability/reports/2012/Grade_3_Indicators_07_23_12.pdf
4. http://montgomeryschoolsmd.org/departments/sharedaccountability/reports/2011/Grade%208%20Prediction%20Model%20Prin%20Memo%203-2.pdf
5. http://montgomeryschoolsmd.org/departments/sharedaccountability/reports/2011/Grade%209%20Indicators_5-25-11%20FINAL.pdf
6. http://hepg.org/hel-home/issues/27_3/helarticle/using-research-to-predict-great-teachers_501
7. http://www.gse.harvard.edu/sdp/partners/index.php