Forget ROI, Here's the 5-Step Tech Investment Plan Districts Should Be Using

When it comes to technology projects, the value of an investment should be defined by its educational return, not its financial return.

• By Keith R. Krueger
• 07/03/13

Stop me if you've heard this one before. "Show me the ROI of technology in education, and then I will support your budget," says the chair of your school board. "Businesses show the return on investment for technology--why can't our district?"

This has become an increasingly common question in today's challenging fiscal environment, where the "new normal" makes for tight budgets. In fact, the Consortium for School Networking's 2013 IT Leadership Survey found that 80 percent of school district IT leaders predict flat or declining IT budgets for the upcoming year. Given this crushing fiscal reality, investing in new technologies is likely to mean making tough choices. So how does a savvy chief technology officer demonstrate that a technology project is worth the investment? Perhaps the traditional ROI formula is not the best metric to use.

In today's economy, according to a recent New York Times article, the value of what Americans get from digital technologies is not calculated as part of the gross domestic product. The GDP only measures the monetary value of the goods and services that Americans pay for, not the information that they gain by using technological tools. The same distinction can be made in education.

ROI is calculated by measuring benefits in dollars. But schools are not in business to make money, and should not measure success in terms of dollars. The business of schools is learning. Of course, if technology projects save money or improve efficiency, then a business-focused ROI is useful, but in general it is important to define the "value" of learning in education. School system leaders need to understand their educational goals and how technology will support those goals. In other words, they should focus not on demonstrating ROI, but what we at CoSN call the value of investment or VOI.

So how do you measure investments in technology that are focused on educational rather than financial benefits? A good place to start may be at your district's long-range strategic plan, which may contain goals similar to the following:

• Increasing student achievement
• Increasing student engagement
• Improving attendance and behavior
• Attracting and retaining staff
• Developing 21st century skills for students
• Decreasing dropout rates for at-risk students
• Engaging parents and communities

Certainly, dollar benefits can be assigned to some of these goals (for example, decreasing the dropout rate can have a measurable impact on district finances) but the mission of schools is not to increase revenues but rather to use public monies wisely in producing students who are thoughtful, productive citizens and members of the workforce. That is how schools "create value" for their communities. So, even though most aspects of schools' performance cannot be measured in financial terms, there are measurable ways to demonstrate to stakeholders the value of the investment in technology in terms of advancing the mission of the district.

The 5-Step CoSN VOI Methodology
CoSN's VOI website offers methodologies and tools designed to assess the costs and benefits of technology projects. The VOI approach can be used to evaluate the comparative costs and benefits of two or more projects competing for the same funding; to sell a project to stakeholders; to articulate the costs and benefits of a project to constituents; and, later, to determine whether a project should be sustained. The steps for performing a VOI assessment for each proposed project include the following:

1. Determine the Total Cost
To get started, you must understand the cost of a technology initiative over the life of the project--the project's total cost of ownership (TCO). Initial purchases, training, and implementation costs must be amortized or annualized, and ongoing costs must be added in. For your 1-to-1 initiative, for example, initial amortized costs could include devices over four years, yearly network upgrades for broadband capacity, and initial teacher training over four years. Unbudgeted but real costs include indirect labor such as the hidden cost of time that teachers spend in training and in handling technology problems. Typically, these unbudgeted expenses are as large as or larger than the direct costs. CoSN provides a free VOI Project Cost Estimator worksheet to help you to identify and summarize all of these costs.

2. Calculate Anticipated Savings and Revenues
Most projects, even those focused on qualitative benefits such as student achievement, have some cost savings. There may also be some anticipated increase of revenues based on higher attendance, grants, or state/federal aid. Some projects, such as administrative applications and teacher productivity tools, are implemented to save money. For these, CoSN provides a ROI calculator as part of the Project Cost Estimator. For projects focused on qualitative benefits, district leaders should bypass the ROI calculator and instead use a Project Benefits worksheet to articulate the benefits in measurable terms related to the district's mission.

3. Measure Qualitative Benefits
Since the purpose of schools is education and schools operate for the public good, many or most of the benefits of implementing technology cannot be measured in terms of dollars. We call these qualitative benefits. For these to be considered benefits, they must directly or indirectly affect the school or district's mission, goals, and mandates. Here's CoSN's suggested approach to calculating these anticipated benefits:

• Determine the school/district's goals and, if evaluating two or more competing projects, assign a relative importance to each.
• Align anticipated project benefits with the appropriate project goals.
• State the project's anticipated benefits in measurable terms, such as from today's measure, to the project's goal, by a certain date. For example, increased community support and interest could be measured by a projected increase of telephone contacts from x per month to y per month and website visits from x per month to y per month by June 2015. Likewise, student behavior can be measured by the number of disciplinary actions, and teacher retention can be measured by the rate of turnover or responses to satisfaction surveys.
• Agree on the effect of each benefit on the district's strategic goals and enter this information into CoSN's scoring tool. This step is useful for evaluating two or more competing projects and leads to a collaborative score from zero to 10 showing how much the benefit will affect the district goal. For example, a benefit of online learning may be that "all students will take and pass at least one online learning course before graduation." If the district has a goal that "our graduates will be lifelong learners," the project evaluation team may determine that this project supports the lifelong learner goal fairly strongly and give it a six. The total qualitative benefits score is the sum of each measured benefit effect on its district goal times the importance of its related goal(s).
• A final option is to enter the probability of success. When district leaders do this step, the total qualitative score described above is multiplied by the probability of success for the project, creating a risk-weighted benefits score.

4. Compare Projects
If you are looking at multiple projects, you should determine the cost and project benefits scores for each, then divide the score by the cost to calculate the VOI. The project with the highest VOI will offer the most benefits per dollar.

5. Evaluate the Results
Once your wildly successful project has been implemented, you have an opportunity to objectively review actual costs and benefits versus the projected costs and benefits. This will allow you to concisely respond to any project skeptics. Since the anticipated costs and benefits were stated in measurable terms, the following actual results can be measured:

• Actual costs versus anticipated costs
• Actual savings or revenues versus anticipated savings revenue
• Actual measurable benefits versus anticipated benefits

The Bottom Line
Assessing the value of proposed technology projects is vital if you as a technology leader want to have credibility with your school board, CFO, superintendent, and community. Clearly it takes work, even with CoSN's TCO and VOI methodologies and tools. But the reward for doing that work is informed decision-making based on a clear understanding of the projected benefits, which mean that you'll be able to answer your school board chair when she asks for the ROI of any technology investment.