Testing Time: The Need for a New Focus On Technology and Results
##AUTHORSPLIT##<--->
Whatever your feelings are about No Child Left Behind, you cannot mistake its serious intent - relentlessly focused on the need to raise student achievement for grades 3-8. Students will be assessed in English and math, and in subsequent years the results will be used to make some high-stakes decisions. Test data must be reported by economic background, race and ethnicity, English proficiency and disability. The idea of measuring progress by subgroups is not only to demonstrate that overall student performance is improving, but that achievement gaps are closing between disadvantaged students and other students.
The pressure is on, and it's not surprising to see many administrators wanting to grasp all levers to help raise the scores as if their schools were underperforming factories that only needed to get their workers to produce more. And "all levers" surely includes technology.
John Bailey, the U.S. Education Department's director of educational technology, gave a speech at the National School Boards Association's Technology + Learning Conference last November in which he talked about the "productivity paradox" analogous to the one that affected the business sector in the 1980s when people wondered whether the heavy investments the private sector had made were making any difference. Bailey's reasoning was that education was still waiting for its 1990s boom with a growth in the universal marker of student achievement: test score increases.
The jury is clearly out on whether education will soon experience an equivalent bump in student achievement to the one realized by industry as a result of educational technology investments. The most important reason is that at a deep level, the analogy between education and business is flawed. The type of business transformation by technology that occurred during the 1980s was made possible because the business sector was clear about what specific processes it needed to automate.
Small and large firms found high-quality software purposely designed for their needs, reducing expenses through-out the production and distribution chains so that such revolutionary ideas as just-in-time ordering of products were possible. This allowed warehousing, transportation and labor costs to be radically slashed. For anything close to that to happen in education, technology would have to help students better manage the time they spend on learning in school, as opposed to shuffling through monotonous tasks and the hundreds of other things that take up time in school.
Filling the Knowledge Gap
The problem, however, is that education technology was never designed with the purpose of replacing any learning activity. It has also never posed a serious threat to the textbook industry. Despite all the rhetoric, the main ways computers have been used in the classroom have been to develop computing skills, for word processing and for the use of the Internet; with Microsoft PowerPoint and graphing thrown in for good measure. There has been excellent software developed - most notably in elementary school reading and math, as well as in engineer drafting at the high school level - but these have been on the margins. No "killer app" for math, social studies or science has emerged despite all of the titles produced.
The bottom line is that educational applications do not so much replace existing teaching and learning as they do supplement them; whether they are better than traditional forms of instruction remains unclear. Given that there are only a finite number of hours in the school day, is time spent playing a "Carmen Sandiego"-type geography game a better way of learning than reading a textbook or a number of other teacher-initiated approaches? The answer is: we just don't know.
If we are interested in finding answers to such questions, we need to do a better job of tracking the difference that such technology plays in learning. In 1997, the President's Committee of Advisors on Science and Technology (PCAST) called for an ongoing federally supported research program with complementary studies conducted by dozens of organizations to provide "rigorous, well-controlled, peer-reviewed, large-scale empirical studies to determine which [technology-supported] educational approaches are in fact most effective in practice" (PCAST 1997). Apart from a U.S. Education Department grant received in 1999 by SRI International, a nonprofit research institute, to support planning for a major program of rigorous, systematic educational technology research, there has been little follow up on the PCAST recommendation.
Instead of waiting for what could prove to be a very long time for a "30,000-foot level" analysis examining technology approaches, why not look at which technology interventions are working in the present, for whom and why. This means considering multiple approaches, engaging in a series of third-party evaluations with their pre- and post-test models as well as classroom observations. We need to supplement these approaches by requesting that vendors supply their own embedded assessment instruments that can be linked to the applicable standards.
It also means developing assessment technologies that provide students with the most suitable, individually planned sequence of knowledge units and sequence of learning tasks (examples, questions, problems, etc.) to find an optimal path through the learning material. Such paths enable a student who is not able to solve a problem or answer a question correctly with a just-in-time amount of content to fill the gap in the student's knowledge.
Embedding Assessment
Since we are now all sensitive to the fact that students learn differently, an active version of this model could be driven by students' knowledge and learning goals, allowing them to experiment with preferences on the type of available learning materials to drive sequencing of tasks within a topic. This technology has been available for at least 20 years, and is now much easier to implement on the Web.
All knowledge could be located on the server and programs could be developed to track the common mistakes. Identifying such errors on a systematic basis would enable the teacher to understand precisely where that critical Vygotskian term "Zone of Proximal Development" was located. Such technologies could provide us with a tool to understand the intricate connection between an old "unschooled mind," and the new emergent understanding by providing students with sensitive feedback that d'es not humiliate and shame, but, instead, instructs. However, we have rarely used its power in schools.
One major advantage of embedding assessment within learning activities is the heightened focus on learning outcomes. Through the act of developing or choosing formative assessment measures, teachers must set out the kinds of skills and knowledge they are trying to impart through learning activities. This reflection, in turn, supports better activity design and articulation of learning goals to students.
Research shows that the use of formative assessment as part of instruction increases learning (Black and Wiliam 1998). Assessments of the skills needed for the 21st century knowledge economy can be made more feasible through technology by providing assessment tasks that reflect the features of real-world problems and offer flexible ways of storing classroom-assessment data. In addition, a new learning technology consultant needs to be hired by the school district to blend curriculum and assessment.
Some advantages of these approaches include all of us becoming more educated in a practical way about what works and why. We might see an end to practices that lead to software-access rationing, which sometimes occurs when individual licenses are just too expensive. In addition, schools and districts might be able to aggregate demand for highly effective software. By aggregating demand in this way, groups of school districts, if not states, could challenge vendors with RFPs that request software vendors to pay attention to their collective needs based on the type of data examination referred to above. They need to determine that need based on an analysis of their own test results, which indicate where students are typically failing to understand. Thus, districts need to challenge vendors to provide evidence that they can meet this need.
Conclusion
The initial thrill of education technology seems over for now. Since most schools are connected to the Internet, the pressing need is not so much for expansion, but the maintenance of what currently exists. The only exception is the relentless appetite for faster connections. With budget w'es all too real as many states face multibillion-dollar deficits, the temptation might be to cut back on technology spending.
Clearly, technology leaders cannot be immune to the new cry for accountability and the need to show the results of technology investments. These leaders need to respond with deliberation and a new sense of clarity, rather than be as reactive as they were in the old days with an ad hoc examination of vendors' projects and claims. They must force vendors to work together on a new generation of products that can provide increased help at the individual student, teacher, parent and district levels.
A good starting point is for them to seek allies with parents who say they mainly purchase computers so their children can learn better, but continue to notice their children doing invaluable things and carrying home their crushing backpacks full of costly textbooks. The testing time for everyone has come.
References
Black, P., and D. Wiliam. 1998. "Inside the Black Box: Raising Standards Through Classroom Assess-ment." Phi Delta Kappan. October.
President's Committee of Advisors on Science and Technology (PCAST), Panel on Educational Technology. 1997. "Report to the President on the Use of Technology to Strengthen K-12 Education in the United States."
This article originally appeared in the 05/01/2003 issue of THE Journal.