What's in It for Us
By following a foursome of best practices from the business world, K-12 districts can guide their BI initiatives toward an effective implementation.
- By John K. Waters
- 04/01/10
Numbers crunching isn’t what it used to be. What might once have been the task of a minimalist keypunch is now an elaborate operation presided over by dedicated business intelligence (BI) systems. No one knows this better than the enterprise. Companies of all sizes must translate their mountains of raw data into meaningful insights and actionable information. The ability to do so has evolved from a competitive differentiator into a mission-critical function.
“Most companies need BI solutions even to understand what’s going on in their business,” says Boris Evelson, principal analyst at Forrester Research. “At one level, it’s about getting the raw data under control. We’re not just talking about gigabytes or even terabytes today, but petabytes, which can’t be usefully organized in a spreadsheet. At another level, it’s about regulatory compliance. Organizations of all kinds—healthcare, financial, life sciences, manufacturing, utilities, education—face an ever-increasing regulatory reporting complexity that makes producing a monthly or quarterly report a huge and difficult task.”
The rise of BI to the top of every C-level executive’s list of priorities also reflects what Stephen O’Grady, senior analyst at RedMonk, sees as a broad-spectrum shift away from intuition-based decision-making toward an evidence-based approach.
“The adoption of business intelligence in a variety of industries is part of a larger trend that’s saying, ‘Look, we have a lot of data. Why don’t we put that to work to help us to make better decisions?’” O’Grady says. “In the sports world, for example, decisions are being made more and more on the basis of metrics and performance. Google is the most famous name in the technology world; it makes decisions about its interface, for example, based on user response rates. Medicine is moving in this direction too.”
And so is education. K-12 has been exploring the potential of BI processes and practices for about a decade, according to Rita Sallam, research director at Gartner, with mixed results. “We’ve seen a real uptick in K-12 interest in business intelligence,” Sallam says, “particularly as a result of the accountability reporting requirements of No Child Left Behind. But deploying a BI strategy can be extremely challenging. You can’t just slap on a dashboard and go.”
Sallam, O’Grady, and Evelson say K-12 organizations must understand that a BI solution is far more comprehensive than that, requiring a data warehouse to collect and keep the information, and analytics tools to allow users to observe performance and trends over time, isolate and explore specific issues, and even generate predictions.
To ensure a successful BI implementation, the three analysts suggest districts apply several best practices developed in the business world—and avoid some common mistakes that could undo their initiatives.
1) Coordinate Your People
“Contrary to common belief, you don’t start a BI project with technology,” says Evelson. “Leaving it all to the techies is a surefire recipe for disaster. Why? Because they’re not the ones who understand the information. If I’m a CFO at a large school district, I’m the one who’s going to know what kind of metrics I need to measure the financial performance of my district, not my IT developers.”
Sallam agrees. “Never just hand off a BI project to IT,” she says, noting that getting users to employ the system requires more than merely installing it. “Never think, ‘If we build it, they will come.’ That approach never works out the way you hope it will.”
“In a lot of ways, the technology is the easy part,” O’Grady adds. “Getting everyone in an organization on the same page might be the greatest challenge of a BI implementation.”
The success of a BI project depends heavily on techies and non-techies working in tight collaboration, Sallam says, and also on groups within an organization with potentially “competing frames of reference” coming together and sharing data. “Some people just get locked into an Excel culture,” she says, “where they can load data into spreadsheets and perform their own transformations and analytics. They have to give up that control if the organization is going to get that one true, enterprisewide view of data that BI seeks to establish.
“It’s relatively easy to say, ‘We’re the finance department and we want to do some BI, and we’re going to agree among ourselves what we want to measure.’ That’s challenging enough. But you’re essentially already on the same page. You’ve got maybe one or two people doing the analysis, so things don’t necessarily get fractured. But when you add different lines of business within an organization, that’s when you run into some real challenges. The way the administration offices define a student, for example, may be different from the way a teacher or group leader defines a student. The administration may see a student as anyone who is enrolled in the school, whereas a teacher might define a student as someone who shows up for class.”
All this coordination among departments and between IT and non-IT people falls under the realm of data governance, Evelsonexplains. “Notice that these are conversations that have nothing to do with technology,” he says, “and they literally represent the most important part of the process.”
2) Find a Leader
So, how exactly do you get all of these people on the same page? Sallam suggests establishing an organizational structure designed specifically to bring them together to work through the tough issues of data definitions and governance. Gartner calls this structure a “business intelligence competency center.”
“It’s the best way we’ve seen to bring together people who never really talk with one another,” Sallam says. “There needs to be a mechanism, even if it’s a virtual one, to facilitate the process.”
A BI competency center, which Sallam believes should be run by a non-IT leader, can also drive user adoption throughout an organization. It can provide a platform for establishing a strong connection between a data warehouse and an organization’s goals, and for coordinating the business, technology, and communication skills required for a successful BI initiative.
But getting collaboration from disparate groups within a district also demands leadership. Sallam says successful BI strategies almost always have a push from the top. “Never underestimate the power of executive sponsorship. We’ve found that organizations with successful BI implementations benefited greatly from what you might call the political will to make it happen. They had the structure in place to facilitate governance. There was also someone involved who could say, ‘This will happen; this is a priority for our organization.’”
But that political will has to draw its strength from below, O’Grady insists. The sponsoring exec should be responding to a business need within the organization.
“A vision from the top can pull the people and resources together and create the momentum BI needs,” O’Grady says. “But the impetus has to come from the business users—the teacher who needs to understand where his students are weak so he can focus his instruction, or the administrator who needs to understand which programs are working best in her district.”
3) Start Slow
Executives love dashboards. They’re nice to look at, provide an easy-to-interpret glimpse of performance metrics, and are cheaper than full-blown BI tools. But Sallam warns against succumbing to the lure of the quick-and-dirty dashboard, noting that a dashboard is only as good as the data and data management infrastructures on which it sits.
“We see this all the time,” she says,“executives who say, ‘Just give me a dashboard,’ when they haven’t resolved their information infrastructure issues. Slap a dashboard on top of data that is either not of high quality or rife with unresolved definitions, and you’ll end up with different departments defining what they need differently, and then everybody shows up to a meeting with different numbers.”
“You can install a dashboard,” Evelson adds, “but what are you going to plug it into? You have data that lives in all kinds of applications, from the student information system to HR, finance, and transportation. The dashboard should be the last step in the process.”
The first step should be ensuring the dashboard is credible, which depends on the quality of the data poured into it. Flawed data renders analysis meaningless. That’s why a data warehouse is criticalto a BI project. But a district must also define what it wants the technology toaccomplish based on users’ understanding of the different types of decisions that business intelligence is designed to support, Evelson says.
“The ultimate goal ofBI is to make the right decisions,” he says. “But a strategic decision requires a different data set from an operational decision. To make a strategic decision—something long-term, such as whether the district should build a new school—you normally need very large and complexdata sets. You’ll want to look at historical patterns, understand what worked in the past and what didn’t—that sort of thing. These are complex reports that include a lot of history, cause-and-effect relationships, success factors, etc.
“Operational decisions have different requirements. A strategic report could take hours or even overnight to run. But if aparent calls and the school needs the answer to a quick question—a simple report that pulls up a few pieces of information—that phone query had better take seconds. The architecture, tooling, and technology around these two completely different requirements could be very different.”
4) Don’t Go It Alone
The market is teeming with BI dashboard vendors, but the list of vendors offering “end-to-end” business intelligence solutions that do it all, from providing scorecards and dashboards to managed reporting, ad hoc querying, and data integration, is relatively short. Evelson points to IBM, Oracle, SAS, SAP, Microsoft, and Information Builders as the top providers.
“If you already have a strong vendor presence in your organization,” Evelson says, “say you’re a Microsoft shop and you’re using SharePoint, Office, and SQL Server, from the technology side there’s very little reason to go anywhere else for business intelligence. If you’ve got Oracle or SAP in place for other parts of your operations, I’d say stick with that vendor for BI.”
But whether you’re expanding an existing vendor relationship or starting from scratch, Evelson strongly recommends seeking advice from the experts.
“I’d say start with a consulting company,” he says, “someone who will help you define the requirements and the solution without pushing hardware or software on you, and then help you find the right vendor. Never, ever do this stuff on your own. BI is all about accumulated experience, having a feel for what works and what doesn’t. And each organization is different. It very often takes people who have done this hundreds of times to get it right. You definitely need to work with people who have done this before so you can benefit from their mistakes. I can guarantee that if you decide to do this on your own, you’ll be learning from your own mistakes very quickly.”
Sallam, however, differs. She advises anyone interested in implementing a BI strategy to shop around, whatever vendor relationships may already be established. Business intelligence technologies have yet to be commoditized, so vendors often deliver significantly different features, she says.
“Never feel that you have to implement a BI tool from a vendor just because you have a strong data warehouse infrastructure from that vendor,” she says. “If another provider offers features that better suit your needs, there’s no reason not to consider that vendor’s products.” Keep in mind, Sallam adds, that end-to-end solutions can result in vendor lock-in, which can make it tougher to negotiate a better price.
Implementing a business intelligence strategy is clearly not for the faint of heart, Sallam says. She recalls an executive in the midst of developing an enterprise metrics framework complaining, “This stuff is just plain hard.”
“And it is hard,” Sallam says. “But if you do it right, it’s worth it. The fundamental activity of all organizations—whether it’s a pharmaceutical company, an automaker, a government agency, or a school district—is making decisions. That’s what BI is all about. It might be a teacher deciding what he or she needs to emphasize in the classroom, a principal trying to decide how to manage programs in the school to optimize overall outcomes, or a superintendent trying to manage a district for different strategic objectives. All of those decisions will be improved by actionable intelligence.
“Without a doubt, BI can help to improve all measures of success of what a schooldistrict does, including educational outcomes for kids.”
“In a lot of ways, the technology is the easy part. Getting everyone in an organization on the same page might be the greatest challenge of a BI implementation.”