February 2006 — Features
Print this articleClick here to receive your FREE subscription to T.H.E. Journal
ISKME Special Series Part I >> Data Driven
Savvy school administrators are using data to improve instruction districtwide.
Christmas came early for the School District of Philadelphia
(PA). In early December, the district announced that, for the fourth consecutive
year, students showed measurable improvement on standardized, nationally normed
TerraNova tests.
“While we admit that there is still a lot of room for improvement, we have seen indications of higher performance growth, but also a continued reduction in students scoring in the bottom quartile,” said Paul Vallas, district CEO, as trending results were summarized in December.
What’s fueling the school district’s ongoing momentum? Vallas credits a managed instructional program that includes data-driven instruction and decision-making characterized by six-week benchmarks and assessments, plus increased and standardized professional development during and after the regular school day and year.
Riding the Data Bandwagon
Philadelphia isn’t the only district realizing such performance growth. In this, the first of a four-part series, we will explore how school districts across the country are embracing data-driven decision making to assess performance, analyze trends, and deliver individualized instruction that ultimately improves district and student outcomes.
This multipart series leverages comprehensive research from the Institute for the Study of Knowledge Management in Education (ISKME; www.iskme.org), an independent, nonprofit educational think tank located in Half Moon Bay, CA.
Many of the findings presented here are based on ISKME’s groundbreaking research report, “Anatomy of School System Improvement: Performance-Driven Practices in Urban School Districts,” which studied 28 urban school districts nationwide.
The complete 100-page report is online at www.newschools.org.
Money Matters
Predictably, budget constraints can undermine even the most well-intentioned data project. A few districts reported that they use new student, financial, or human-resource information systems, but others said that antiquated technology infrastructure left them with limited access to timely data.
One of the greatest challenges facing today’s districts is matching resources to goals, says Lisa Petrides, president of ISKME, adding that, although they developed strategies to align their financial, human, and technological resources to meet their goals, many districts came up against significant barriers and limitations.
Several districts reported that at the time of the survey, they were under acute fiscal stress and had limited flexibility in the use of their funds to achieve their overall goals. Two-thirds of the districts described substantial budgetary challenges for the 2005 fiscal year. Many districts had, in fact, made significant cost cuts, such as curtailing summer school or reorganizing grade levels to consolidate school sites in order to bring expenses in line with revenues.
In some districts, even the haves are actually have-nots. Indeed, according to ISKME Vice President Thad Nodine, districts that have updated information systems reported that their systems were silo-based, resulting in the inability to perform data queries across departments. Even in cases where some cross-departmental standards were in place, districts often lacked the time and training necessary to conduct meaningful data analysis.
Nodine says that antiquated systems included processes such as using timecards that had tobe keyed in by hand; creating budgets one at a time on Excel spreadsheets; completing purchase orders in triplicate; and providing teachers with manually completed rosters.
Reinventing the Rules
Still, resourceful district administrators are finding ways to automate their routine processes. For example, some surveyed districts created data warehouses to store student information, and then made them available for querying by administrators.
At the same time, districts can increasingly leverage low-cost and even free open source software. With assistance from external consultants, districts can design IT systems that leverage such open source software offerings as the Apache Web server (www.apache.com), Linux (www.linux.org) operating system, and MySQL database (www.mysql.com). “Increasingly, open source software is as reliable, scalable, and customizable as traditional commercial software,” says Ed Golod, president of Revenue Accelerators Inc. (www.rac-inc.com), a New York consulting firm that serves technology executives.
However, many schools have yet to turn open source software into knowledge-based systems. Indeed, more than 75 percent of superintendents within surveyed districts lack access to data that they can break down in meaningful ways from their desktops. And only three of the 28 districts have developed graphical dashboards that provide a “one-click” option to get updated information on a specific area or issue.
These dashboards are widely available in business intelligence products from Cognos Inc. (www.cognos.com), Business Objects SA (www.businessobjects.com), and other major software providers. However, warns ISKME’s Petrides, many commercial software packages were designed for the business sector and aren’t easily customized for the education sector.
Alas, few surveyed districts had linked human resources and payroll systems to finance and budget systems, and even fewer had linked student information systems to these other systems, as well. Some districts had developed automated systems for teachers to enter or scan student results on district-wide assessments, the results of which could then be immediately aggregated or disaggregated in various ways. However, it is important to note that none of the districts had implemented all of these elements, according to the ISKME report, and most had only a few in place.
Five Keys to Performance-Driven Practices
- Effective data gathering. Examine how your district staff and teachers used to gather and share data, including student achievement data as well as other organizational data. Then develop best practices for gathering and sharing data.
- Outcome assessments. Districts need to provide context for the data that they gather. This effort helps to identify curricular gaps through assessment results, and it also examines how district assessments are aligned with state standards and whether professional development is structured to better address curricular gaps.
- Monitoring and feedback. Progressive districts monitor the effectiveness of specific programs or efforts, and highlight evidence that shows how departments, different sites, and certain programs are performing.
- Ownership of outcomes. Specific individuals throughout the district must be held accountable for the performance of students. Who analyzes current measures and assessments? Who is responsible for the results, and how and what interventions are put in place? Determining ownership also includes designing processes for evaluating the effectiveness of these interventions, incentives to achieve desired results, and evidence of an organizational culture that supports these efforts.
- Building a learning organization. Establish the processes and behaviors that deliver a performance-driven culture. Staff and faculty must match the district’s resources to its goals, and then evaluate the fit between them on an ongoing basis.
Real-World Success
Despite these challenges, some districts are finding real-world success with data-driven decision making. Just ask administrators at the School District of Philadelphia. As part of a comprehensive school-improvement planning process, the district rolled out SchoolNet Inc.’s (www.schoolnet.com) Instructional Management System—a software platform that facilitates data-driven decision-making.
“[District CEO] Paul Vallas has done a great job making sure the school district’s IT programs, data-driven decision-making, and core curriculum all come together,” says Jonathan Harber, president and CEO of SchoolNet.
AT A GLANCE: SDP
At the Philadelphia district, for instance, IMS leverages Microsoft Corp.’s SQL Server 2000 database (www.microsoft.com/education/sql.mspx), which houses and integrates the district’s student information, state tests, local benchmark assessment data, curriculum, educational resources, and standards information.
More than 1,500 people throughout the district trained on IMS prior to the recent 50-school rollout. That training included four-person “principal teams” (a school principal, a technology lead, a math coach, and a literacy coach) from the district’s 250-plus schools. Harber says that the system now enjoys widespread use by teachers in the district, who can immediately access assessment data tied to the curriculum and can personalize instruction for their students.
Moreover, principals can become effective coaches to guide planning and remediation. And district-level administrators can tailor professional development programs to real-time district needs, and thus better meet the targets of the demanding No Child Left Behind Act.
History Lesson
IMS has also addressed several data-intensive challenges that had previously undermined the School District of Philadelphia’s performance. For instance, Harber notes, roughly 35 percent of students move within the district from one school to another during a typical year. To ensure that no transient student falls through the cracks, IMS provides teachers and administrators with immediate and complete access to their students’ profiles.
Data-driven decision-making has addressed several other challenges within the district. For starters, the district can now measure not just individual performance on standardized tests, but also performance on subjects with subgroups aggregated by ethnicity, gender, special needs, and other criteria.
