The Co$t Factor When Is Interactive Distance Technology Justifiable?
##AUTHORSPLIT##<--->
Are you trying to answer the technology cost effectiveness question? Cost benefit, cost effectiveness, return-on-investment: however we say it, we often reduce “it” to the amount of bang for the buck. Those of us who advocate the use of technology try to use cost effectiveness data to disarm our detractors. More beneficially, we’d really like cost effectiveness data to help us identify efficient uses of educational technology. The reality is — like other innovations in education — technology has to compete for the elusive dollar. So, technology must and should be made to demonstrate its effectiveness. But how?
Deficiencies in Traditional Cost Models
A traditional approach to an analysis of educational technology in terms of effectiveness would organize the variables in the cost equation this way:
C
R = ---
BA
Total costs of the technology system (C) divided by its expected lifetime (A), divided again by the effectiveness (B) gives the cost to effectiveness return (R). This formula is difficult to apply, however, for several reasons. First, it’s difficult to identify the costs associated with technology development. The tangible costs are clear — they involve the physical infrastructure, the hardware, software and netware, as well as furniture, electrical and communication wiring, renovation costs to upgrade older buildings and, of course, ongoing line and maintenance costs.
Along with these are many intangible costs. Consider the time it takes to order, install, secure, and maintain the technology. Many of these tasks, whether performed by administrators, teachers or clerical staff, are embedded in general personnel costs and would be difficult to identify, especially when some staff are supported from the funds of several projects. Another relatively intangible cost is the significant amount of usually unreimbursed teacher time spent learning to use the technology and integrate it into curricula. We may even consider the “psychological stress” of technology adaptation. Teachers who do and do not use the technology are subject to time, peer, and career development stresses. Although the costs of these stresses would be extremely difficult to assess, they are real and can adversely affect job performance and personal health.
Another elusive element in cost effectiveness analysis is the length of time it takes to realize results. While some studies report that distance learning projects may take up to ten years to prove viable, few of us making investments in the technology are willing to wait ten years for return on our investment. Identifying technology benefits in the short term (2-5 years) can be challenging, let alone identifying the benefits on a more long-term (6-10 years) or intergenerational (15 or more years) basis.
Like educational technology costs, it may also be difficult to identify ways we consider technology effective. At the core of any discussion of technology effectiveness is the purpose and role of technology in the school. Previous research suggests that policymakers often interpret effectiveness as the ability of the technology to improve standardized test scores or increase numbers of college-ready students. Educators themselves have more interest in the effectiveness of the technology to increase student motivation to learn, and to prepare students for employment in a technologically pervasive workforce. Any of these outcomes are within distance learning’s capacity to address. Our experience, however, suggests that policymakers place narrow limits on the ways technology may be effective, while expecting broad and sometimes unattainable outcomes.
Interactive Television in Ohio
Our study of technology cost effectiveness takes place with participants of a statewide initiative introducing Interactive Distance Learning (IDL). Titled the Telecommunity Project, the initiative, funded by regional and local telephone companies through a Public Utilities Commission agreement, encourages schools to organize as local “Telecommunities.” These Telecommunities are sometimes structured around curricular themes integrating multiple content areas. For instance, a few consortia focus on area waterways and the historical and current implications those waterways have for commerce and regional growth. Other consortia have integrated curricula on the premise of simulated space travel and public television program production. Most, however, have more generalized objectives and focus on offering coursework or thematic units, electronic field trips, or special events such as interactions with experts.
All Telecommunities are provided hardware in the form of interactive television or interactive desktop video technology. Expenses for teacher development in applying these technologies is available as well. The Telecommunities are also fitted with fiber-optic connectivity and linked to a State of Ohio multi-user IDL ring to encourage exchange and interaction among the 400 schools in 23 funded consortia in the state.
Part of the ongoing evaluation of the initiative was to determine IDL cost effectiveness (Hawkes, Cambre, Lewis, 1999). To identify the possible criteria on which to gauge effectiveness, the evaluators organized focus groups with key participants in each Telecommunity. Teachers, administrators, museum curators, wildlife biologists, university personnel and others discussed the following question: “In what ways would you consider your system cost effective?” Focus groups were followed up with participant interviews and document reviews to flesh out the meaning of technology effectiveness.
Considerations for Determining ITV Cost Effectiveness
Our inquiry found that, in general, educators do not expect technology to take over the instructional role in the school setting. Therefore, IDL’s value cannot be measured in the traditional way, i.e. as the cause (instruction) responsible for the effect (test scores). Rather our research, like that of Leh, Sleezer & Anderson (1998), found that the unique school context, its focus and goals, heavily influence the perception of the value of educational technology. The indicators listed below represent 12 ways that educators in Ohio using IDL for student learning feel the cost effectiveness question can be answered. These are not meant to be criteria against which systems should be measured. They serve rather as indicators of progress.
1. A system is cost effective if it gives students, teachers and community members opportunities to take courses and engage in experiences that would be impossible without distance learning technology.
Numerous examples are evident throughout the Telecommunities courses and workshops being offered, both for students and for teachers. Virtual field experiences and other interactive learning events that were not available before the technology was in place are being provided. Some Telecommunities that have been in operation for more than a year insist that it is not just a matter of the number of opportunities provided, but the quality and value of these opportunities.
2. A system is cost effective if it allows participants to explore and to become pioneers.
Many participants suggest that the real benefit of telecommunications is the pioneering, exploratory opportunities it allows for both students and teachers. “We may never prove it saves money, but if it keeps us on the cutting edge of what can be done with technology, we will be happy,” says one school administrator. There is the feeling that educators need to be constantly challenging, stretching and exploring new things. This approach insists that projects be judged on their innovation and on the number and quality of new activities they generate.
3. A system is cost effective to the extent that students are actively engaged in learning and are enjoying what they are doing.
Increased student proficiency, “measured in both quality and quantity,” defines this measure of cost effectiveness. Many participants report that excitement, self-initiation, and joy of learning can be observed, if not measured in students as a result of their participation in Telecommunity projects. For instance, project directors at the International Space Station Telecommunity report that, “Teachers have already noticed extraordinary excitement, interest and engaged problem-solving on the part of students participating in the space missions.” Observable characteristics might include improvement in the style of student presentations and in their communication skills, and the extent to which students begin to take responsibility for their own learning.
4. A system is cost effective if there is a measurable increase in the number and kinds of activities offered, as well as in the number of students and teachers participating.
This characteristic of cost effectiveness demands steady growth over time. It would include counting total hours of use of the system, total number of students participating, total number of teachers and other educational personnel involved, along with the number and variety of applications for which the system is used. If a project is cost effective, it is reasonable to assume that the numbers in each of these categories will increase from one year to the next until system saturation is reached. Many participating educators would caution, however, that numbers for the sake of numbers are not what is desirable here. As mentioned before, participation and quality should increase in tandem.
5. A system is cost effective if it promotes understanding, collaboration, and tolerance among students of diverse backgrounds.
This non-traditional measure of cost effectiveness springs from observed “side effects” of telecommunications applications. The cultural mix of students brought together through the technology promotes interactions that would be unlikely without telecommunications. This outcome has been observed time and time again in Telecommunity projects with an attached value far beyond project developers’ highest expectations. Teachers sensitive to diversity issues can maximize this effect.
6. A system is cost effective if student performance improves.
While this characteristic is viewed with caution by most participants, some suggest that measurable performance based on the content of the lesson, increased attendance, and improved writing skills would be marks of a cost effective system. Improved performance based on pre and post course measures is also acceptable as evidence of system impact. Participants in our evaluation of the technology consistently agreed, however, that there are too many variables at work to make causal statements about the cost effectiveness of IDL based on standardized test scores.
7. A system is cost effective if good teachers can be shared, and if some teachers are freed up to do other things needed in the district.
In some Telecommunities there is a shortage of qualified teachers for certain subjects that distance learning helps to fill. In other instances, teachers whose classes are now taught from remote sites are free to work with students who need more individual attention. There was the suggestion that the cost effectiveness in this regard might not be immediately evident. “If an innovation yields more effective teaching, it can impact a whole generation of students,” one teacher observes. To determine whether an innovation has such long-range potential, it is necessary to take some risks that, in the short run, may not be cost effective.
8. A system is cost effective insofar as the content delivered is recyclable.
This characteristic d'es not necessarily imply that all Telecommunity efforts should be “canned” so they can be used over and over again. It d'es suggest that at least those activities and/or courses that have required large amounts of time and energy to develop, have been repeatedly successful, or involve the service of state or national organizations whose staff are required to repeat presentations, can and should be repeated as is appropriate. Experience will suggest how to recycle good content without losing the spontaneity that is so important to engaging interactivity.
9. A system is cost effective if it promotes better knowledge of and facility with technology.
Giving students high tech experiences and encouraging them to learn the technical aspects of these systems, as well as their potential for educational applications, should be one of the most important goals of using telecommunications in school. The argument is that this exposure prepares students in numerous ways for college and/or the world of work.
10. A system is cost effective if it helps “sell the school” to parents of prospective students.
For private and parochial schools, an emphasis on and integration of technology is a compelling recruitment tool. For all schools, a technology-infused system has improved parents’ attitudes toward the progressiveness of the curriculum and the effort the school makes to offer their students the very best education possible.
11. A system is cost effective if it is used to conserve resources in the daily operation of agencies.
This criterion suggests that teleconferencing could easily absorb the time and expense of driving teachers and administrators from remote areas to meetings across the state.
12. A system is cost effective if it promotes collaboration between students and teachers from different schools.
Just as athletic events bring students together in competition, there is value in bringing together students from different schools to collaborate in intellectual endeavors. Collaborations with peers from other schools motivate students to work together to achieve common ends and improve the quality of student work. Interactive technologies also prove effective in bringing teachers together in such a way that perspectives on educational policy, practice, and knowledge can be shared and supported.
Summary
We can theorize about and even try to legislate cost effectiveness in interactive telecommunication applications, but when all is said and done, the issue is highly idiosyncratic and pragmatic. The measures presented here represent a lever for educators in valuing educational technology in ways that bring new resources to the learning process and transform current resources for more productive use. Of course, not every telecommunications-supported distance learning endeavor will immediately produce evidence on all 12 measures. But even a handful of these indicators, if methodically addressed, can respond to the cost effectiveness interests of a local educational community.
Dr. Mark Hawkes is Director of Graduate Studies in Instructional Technology and Assistant Professor in the College of Education at Dakota State University. He has led and participated in a number of district and statewide evaluations of educational technology over the last decade. He continues to investigate the outcomes of educational technology application on student learning and teacher professional development. He holds a Ph.D. from Syracuse University.
E-mail: [email protected]
Dr. Marge Cambre, Emeritus Faculty member of the Ohio State University College of Education, is currently a full time consultant specializing in the evaluation of technology projects. She has taught and published in the areas of distance education, formative evaluation, and instructional television and consulted in these areas for several organizations including the North Central Regional Educational Laboratory.
E-mail: [email protected]
References
Hawkes, M., Cambre, M., and Lewis, M., 1999. The Ohio SchoolNet Telecommunity Evaluation — Year Three Evaluation Results: Examining Interactive Video Implementation, Adoption, and Resource Needs. Oak Brook, IL: North Central Regional Educational Laboratory.
Leh, A. S. C., Sleezer, C. M., & Anderson, V. A., 1998. Measuring the value of educational technology in different contexts. Educational Technology 38(4), 28-33.
This article originally appeared in the 08/01/2000 issue of THE Journal.