Data-Driven Decision Making :: The 'Other' Data
##AUTHORSPLIT##<--->
Most districts know the benefits of tabulatingand tracking student performance. But nowmany are finding there's even more to gain fromnumber crunching by casting out a wider net.
DATA, DATA EVERYWHERE! It's more than achipper jingle for the information age—it's a daily realityfor school systems, which must determine how tobreak down so many numbers efficiently, effectively,and to their best advantage. Between standardized testsand tools from companies such as Pearson Education,Wireless Generation, and Riverside Publishing, educatorsand district superintendents alike are up to theireyeballs in facts and figures about student performancethat they can use as the basis for curricular decisions.Still, there's more to assessment than student performance.A growing number of K-12 school districtsacross the country have turned to traditional technologiesto collect new information about non-academicindicators, such as employee retention, transportationefficiency, or how many students purchaselunch when the cafeteria offers hot dogs.
Many districts also chart demographic data, taking basic responses on enrollment forms and funneling them into a student information system for the scoop on how many students are coming from single-parent homes, or how many students are only children. In the majority of cases, districts incorporate statistics on gender and race, as well as information on the percentage of new students who speak English as a second language.
In a recent though still rare development, schools have signed up to collect data about student perceptions. Melissa Getz, president of Online School Surveys in Albany, CA, says several districts have hired her organization to help them compile information on issues such as how safe students feel on campus, how fair they believe their teachers are, and whether they see racism on campus.
"All of this data is important to some degree," she says. "The bottom line is that the more data you collect, the better your decisions will be."
Four Kindsof Data
In a 2003 article in EducationalLeadership magazine, Victoria Bernhardt,executive director of the nonprofitEducation for the Future Initiative in Chico,CA, names four different kinds of datathat schools should collect as part of theirdata-driven decision-making efforts:
- Demographics: Describes the students,the school's staff, the school,and the surrounding community.
- Perceptions: Helps educatorsunderstand what students, parents,teachers, and the community thinkabout the learning environment.
- School processes: Highlights schoolprograms, instructional strategies,assessment strategies, and classroompractices.
- Student learning: Includes a varietyof measurements that show theimpact of the education system onstudents.
In the article, Burnhardt insists that thevery best DDDM strategies enable schoolsto cross two (or more) categories of dataand chart the results to see trends developingover time.
"Not until you intersect [at least two]data categories at the school level andover time will you be able to answer questionsthat allow you to predict whether theactions, processes, and programs you areoperating will meet the needs of all students,"she writes. "By crossing all fourdata categories, you are taking intoaccount who your students are, how theyprefer to learn, which subgroups areachieving, and with which processes studentsachieve."
Getting the Whole Picture
Since 2005, upward of 50 school districts in the state of Minnesota have turned their data warehousing efforts over to a piece of proprietary software put together by the Central Minnesota Educational Research and Development Council (CMERDC). While the software was designed to collect information about student performance, most districts are using it to collect non-academic data, too.
Consider what Anoka-Hennepin School District 11 in Coon Rapids is doing. There, technology officials are using the CMERDC software to catalog data about human resources, transportation services, and food services to obtain a more accurate picture of how the district functions, and what changes can be made to improve efficiency.
For example, the district charts longevity among its employees. If data indicates that teachers are leaving before they hit a certain salary level, changes might be made to improve retention.
BE PREPARED
DATA-DRIVEN DECISION MAKING HAS BECOME SO INTEGRAL TOPENNSYLVANIA SCHOOL DISTRICTS THAT A STATE AGENCY ISINSTRUCTING EDUCATORS IN HOW TO MASTER IT.
At the Allegheny Intermediate Unit in Homestead, PA, technology officials builtthe organization's Comprehensive Data Analysis (CDA) program in 2005. A year later, theyfounded the Data Leadership Academy, a professional development initiative to show usershow to make the most of the data their systems collect.
Nancy Olenik, senior program director in the Department of EvaluationGrants and Data, says the academy kicked off with a session for the curriculumcoordinator, the federal programs coordinator, building principals, technologycoaches, and guidance staff to teach them what kind of data theycould collect and how they could use that data to their advantage. "We triedto build a good triangulation of data, then walked them through the processof interpretation," she says.
While the organization continues to hold data workshops, it essentially has become a marketinggroup charged with spreading the word about CDA to those who aren't yet believers.Volunteers in each district hold meetings to take newcomers through the process that wouldallow them to collect and organize their data. They teach attendees how to plot graphs, buildcharts, and extrapolate certain numbers.
"We're trying to move people past the initial cardiac arrest that comes when you talk aboutusing data," Olenik says. "Data doesn't have to be this big scary thing that reminds you ofyour college statistics class. It can be fascinating and, believe it or not, very useful whenapplied the right way."
A similar effort is under way at the St. Peter Independent School District in Minnesota. After thedistrict launched a proprietary data warehouse developed by the Central Minnesota EducationalResearch and Development Council, officials kicked off an informaltraining program by recruiting data coaches to hold regular seminars on how to collect data andmake sense of it. These seminars follow a step-by-step format, guiding participants through everyaspect of the data interpretation process, from collection to report generation, and actually actingon the information at hand. District Data Director Nancy Cluck says the seminars are basic, buthave proven to be important for those educators who aren't exactly technological whizzes.
"We'll do anything to make our people feel more comfortable with the system that's goingto be here for a while," she says. "Just because these people are teachers doesn't mean theycan't learn too."
"We might be a school, but we still need to run ourselves like a business," says Georgia Kedrowski, assistant director of technology and information services. "We have performance criteria that every department must fulfill, and those are to be efficient and effective."
At least one large urban district not only trackshow students get to school, but also collectsinformation on which students take books out ofthe library, and overlays the two data sets to seeif walkers and drop-off kids spend more timestudying than students who take the bus.
The district's most exhaustive non-academic data-driven decision-making efforts come in the area of demographics. District technologists dump US Census data into the warehouse along with information from its own head counts, which are taken each October. They also incorporate data from Excensus, a demographic services provider, that factors in community data to reveal how many households might have school-aged children within the designated timeframe.
Next, with the help of proprietary datamodeling software from CMERDC, the school runs projection analyses to determine enrollment figures for five and 10 years hence. In the last two years, these analyses have predicted a steady decline in the number of students coming into the district—a drop of nearly 200 students per year for the next decade. Kedrowski says the district is currently in the process of analyzing these shortfalls. What will sagging numbers mean for budgets? Will schools have to close? Will teachers lose their jobs? These are the questions school officials have been asking.
"There are school districts that have ignored that type of data and find themselves in all kinds of trouble," she says. "Luckily we've got enough control of our information to eliminate surprises."
Buying Off-the-shelf
When Fortune 1000 companies want to scale mountains of data for a competitive advantage, they turn to data warehousing applications to help organize material and business intelligence solutions to make sense of everything. Offerings from companies such as IBM, Oracle, SAP, and SAS are all popular in this arena. The price tag? Let's just say that any six- figure deal would be considered a steal. Naturally, with slimming budgets and dwindling IT staffs, most school districts can't afford such high-priced solutions. As an alternative, a number of districts have turned to vendors that offer technologies with similar functionalities and targets, at a lower price.
There currently are at least 25 to 30 vendors marketing DW solutions to K-12 school clients, including market players such as Teradata, Edmin, and SchoolNet, to name a few. Thad Nodine, vice president of the Institute for the Study of Knowledge Management in Education in Half Moon Bay, CA, says using these offthe- shelf packages is a good way for districts that aren't currently organizing non-academic data to test the waters.
"So long as you're not spending too much, it can't hurt to try out one of these packages to see exactly what kind of data you can collect and how you can use it," he says. "Of course, the key is making sure you're not spending too much."
Obviously, budgets for these kinds of solutions differ by district. At Charlotte-Mecklenburg Schools in Charlotte, NC, a relatively large district, the right price was six digits—a few hundred thousand dollars in all.
The district recently turned to a vendor named Mariner to handle its data warehousing and business intelligence needs. At the behest of a new superintendent with a new strategic plan, district officials had Mariner develop a data dashboard that ultimately will enable school administrators, teachers, and the public to review performance of the school district against a number of goals and objectives.
Though it's still in development, today the tool charts performance indicators like most such tools. The software also monitors whether the building of facilities makes budget, and keeps track of whether buses are on time. By tabulating periodic surveys and local media coverage, the system even charts public perception.
Charlotte-Mecklenburg Chief Accountability Officer Jonathan Raymond says this comprehensive approach is what makes the technology so successful.
"A lot of times, when districts embrace a new technology to handle data-driven decision making, they're only collecting certain kinds of data," he says. "This tool goes beyond the nitty-gritty to really give us a deep and wide perspective on what's going on, and how we can use that information to make the experience better for everyone involved."
While Charlotte-Mecklenburg entrusts its non-academic DDDM to one vendor, some districts take a hybrid approach and opt for multiple off-the-shelf packages. Michigan's Waterford School District has gone this route. Officials at this 12,000-student district recently enlisted the help of a trio of hardware and software vendors to capture digital snapshots of forms that deliver data the old-fashioned way: via the printed page.
The need for this solution became evident when district officials evaluated their enrollment process and realized that the district was processing up to 10,000 paper forms a year. In addition to manually entering names, ages, and addresses, Waterford personnel were keying all other data from handwritten CA-60 student forms into the district's student information system.
Add to these metrics the fact that a portion of the manually entered data was incorrect, and Dave Vultaggio, director of management information systems, says he knew there had to be a better way.
"Once we saw that we needed to modernize the system, we could have gone in any number of directions," he says, noting that in Michigan, as in many states, districts receive funding based upon how many students they enroll. "Rather than change everything from the ground up, we took a more conservative approach."
While this approach includes a variety of technologies, it is based largely on document scanning. First, district office personnel scan all forms and documents into fi-4120C scanners from Fujitsu. Thanks to proprietary TeleForm form-processing technology from Cardiff, parents also have the option of filling out forms online. All of this material is then stored in the Image Executive enterprise content management system from CEO Image Systems, where it can be accessed anytime.
Today, Vultaggio says, with all of its enrollment information stored electronically, the district can more easily catalog the demographics of a particular class and make decisions on all-important class size. He adds that centralizing student enrollment from 25 buildings to one single point of data entry, and reducing the number of staff members who support the process from more than a dozen down to two, has taken a great burden off of the annual enrollment process.
"In our current economic climate, it is important that schools find ways to operate efficiently," Vultaggio says. "With this system in place, we can make information readily available to anyone who needs it, using the power of our data to make informed decisions across the board."
Issues With Interoperability
Of course, multifaceted solutions that handle various types of non-academic data aren't perfect. Because this data quantifies different metrics in different ways, interoperability has become a huge concern among technologists who deal with these issues every day.
"There are school districts that have ignored[non-academic] data and find themselvesin all kinds of trouble."
—Georgia Kedrowski, Anoka-Hennepin School District 11
Just ask Laurie Collins, project strategist at the Schools Interoperability Framework Association in Washington, DC. SIFA works with vendors and districts to come up with a standard for data collection across the board. This standard—the latest version of which is Version 2.0r1— is based in extensible markup language, or XML. It is the same language that forms the basis of web services. According to Collins, it's difficult for schools collecting, say, human resources data in one system or language to utilize information from another system or language without some sort of conversion process. On most occasions, she notes, if a data-driven decision-making strategy requires a conversion process, users simply will give up. On the flip side, she says early adopters of the SIFA standard have been able to make great strides in correlating disparate non-academic data. To wit: At least one large urban district now not only tracks how students get to school (by walking, bus, or parent dropoff), but also collects information on which students take books out of the library, and overlays the two data sets to see if walkers and drop-off kids spend more time studying than students who take the bus.
"It's amazing how much you can get out of your data when you use it in conjunction with other data," Collins says, citing a 2003 magazine article as the impetus for SIFA's effort (see "Four Kinds of Data"). "For years this information has existed in silos, and it's time to take a more holistic approach."
Collins admits that the interoperability is mostly "for technology people," and not something for educators or even district administrators to worry about. Still, she says, collecting data about non-academic indicators and doing so in a way that facilitates contrasting and comparing never hurts.
"If nothing else, this can help districts highlight what their business processes really are, and which processes are working well enough to actually generate results," she says. "That knowledge alone can be useful; anything beyond it is priceless."
-Matt Villano is a freelance writer based in Healdsburg, CA.
This article originally appeared in the 09/01/2007 issue of THE Journal.