Classrooms as Test-Beds for Educational Software Design
##AUTHORSPLIT##<--->
Innovative teachers know how to take risks; in fact, they often thrive on it. This is what it takes to be part of a research and development project using technology in a public school. This article gives highlights of our experiences working with a large, multi-year effort to transition advanced computer technologies from defense applications to commercial products for public education.
The Fundamental Skills Tutoring Project (FST) is a timely collaboration among the military, business/industry and educational community to address one of the most pressing challenges facing our nation: teaching the thinking skills necessary to participate in a complex, modern society. R-WISE (Reading and Writing in a Supportive Environment) is one of three Fundamental Skills Tutors developed by the Air Force and field tested at MacArthur High School in San Antonio, Texas, using ninth-grade classes.
Moving Beyond Word Processing
The potential for computers to aid in writing instruction has moved beyond mere word processing. With this in mind, the Air Force's Armstrong Laboratory formed a steering committee in early 1991 to draw up a plan for designing, developing and evaluating interactive software for mediating reading and writing. The finalized plan called for a suite of three intentional learning environments based on the theories of Carl Bereiter and Marlene Scardamalia (authors of The Psychology of Written Composition, 1987).
Each of the workspaces included (1) structured planning and goal-setting, (2) visual algorithms that depict the writing process, (3) adaptive, interactive instruction based on the student's individual situation, and (4) just-in-time modules that present a cogent review of basic principles. In September of 1992, after eight months of programming, the software was ready for a beta-test site release.
Presenting Authentic Writing Tasks
The task modeled by R-WISE involves a type of writing familiar to managers, investigative reporters, engineers and researchers ¬ to name just a few. In issues-based writing, source documents become the raw materials in composing a position paper, an evaluative summary or an interpretive response.
The software is designed to be a comprehensive, integrated verbal skills learning environment providing cognitive support for learners. It differs from many other computer-aided literacy software in that it is an end-to-end development package. It assists the learner throughout the process by providing instructional statements adapted to the individualized needs of the student.
In short, R-WISE offers a way of learning that is different from the traditional classroom: students don't have to wait for the teacher to prompt them on paragraph structure, idea development or whole paper organization. The software embeds adaptive advice into the very process of composing, so students get real-time guidance rather than after-the-fact grading.
Fostering Higher-Order Thinking Skills
When integrated into the curriculum, the tutoring program provides individualized practice with concepts introduced in daily classroom activities. Expert writers use appropriate strategies to write effectively for a variety of audiences and purposes. Novice writers need guided practice with each of these stages in order to develop the metacognitive skills of expert writers. (Metacognition is defined as thoughts about thought, knowledge about knowledge, or reflections about actions.)
In the writing process, novice writers lack the ability to access appropriate information when it is needed. An analogy is to think of the brain of an expert writer as a file cabinet containing labeled folders that are easily accessed when needed. However, the brain of a novice writer is like an overflowing junk drawer from which nothing can be easily retrieved. Metacognitive activities serve as reinforcement for students learning to make suitable choices about their own writing.
Serving as a Test Site
Douglas MacArthur High School, with a current student population of over 3,000, is the oldest high school in the North East Independent School District. In the early 1990s, the school drew from a middle- to upper-middle class population; now transitioning neighborhoods and re-districting have changed the demographics to predominately middle- to lower-middle socio-economic groups. The ethnically mixed population contains approximately 60% Anglo-American, 34% Hispanic, 4% Afro-American and 2% Asian. The instructional staff includes many highly experienced teachers, most of whom have spent their entire teaching career at MacArthur.
The project required teachers to alter their instructional styles and make adjustments between the traditional classroom and the electronic classroom. Additional challenges were presented because this was a research and development project and, as such, presented unusual circumstances for classroom teachers and their students. In all, nine MacArthur teachers, 28 classes and approximately 650 ninth-grade students participated in the pilot study.
That first year required the teacher to deal with many variables: integrating R-WISE into an already demanding curriculum, coping with software changes, maintaining a positive attitude in the students when software problems arose, and working collaboratively with other teachers as well as the developers from Armstrong Laboratories at Brooks Air Force Base. Overall, it was a rewarding experience.
Getting Started
Teaching English is a demanding profession: every year the curriculum expands and new mandates arise. Adding advanced technology to the fullness of these concerns meant new learning curves to climb both for teachers and for students. Not the least of these hurdles for some was gaining a familiarity with computer basics.
Addressing staff development of teachers first, we held an intensive three-day summer workshop in which participants reviewed the basics of the software and discussed how best to work with the functionality and the features of R-WISE.
Other issues, more on the level of "housekeeping," also had to be decided. A set of rules for behavior in the lab was written up and distributed to students. An initial lesson was developed for all students to review keyboarding and mouse skills. Since that first year, the teachers have become more confident about composition instruction using R-WISE. This has come with experience, staff development and improved, more reliable software.
Coping with Change
Logistics posed some constraints for the teachers and their students. For example, the traditional classroom allows teachers to organize and address things in their own ways; on the other hand, teaching in the computer lab requires some things to remain constant. Getting accustomed to teaching in a laboratory setting was one of the first adjustments teachers faced. With 30 workstations grouped in four parallel rows, the physical layout of the room keeps the teacher moving, constantly circulating in the room, observing and offering encouragement to students.
Characteristics of the room and usage logistics created further constraints. The snug quarters meant student movement had to be limited. With the lab rules, however, the dilemma of student movement was soon minimized, enabling teachers to control students and their activities. The rules were the same for all English classes, so if students transferred from one teacher to another, the transition was smooth for everyone.
The sound of the computers presented a problem to teachers who wanted to use whole-class teaching while in the lab. This forced some to alter their presentation styles. Almost all of the teachers began preparing students more thoroughly for the R-WISE lessons prior to going to the lab. A portable public address system was available for those teachers whose voices would not carry over the noise level.
Furthermore, teachers had to use the lab according to a designated schedule that did not necessarily correspond with their regular classroom curriculum, requiring both patience and flexibility.
A partnership between the teacher and the pedagogy of the software is vital to producing performance gains. | Integrating Software with Curriculum Fortunately, R-WISE lends itself to a wide range of educational environments and student populations. The software d'es not dictate course content or teaching style. Since no "canned" lessons are involved, teachers use the Authoring Tool to create lessons that (1) are appropriate to their students' ability level, (2) capture student interest or accommodate situational requirements, and (3) foster individuality and creativity for both students and teachers. Teachers design their lessons choosing from three major tools: Cubing (invention), Idea Board (drafting), and Revision (strategic editing). The tutoring session serves as guided practice to assist students in developing the higher-order thinking skills used by expert writers. The R-WISE tools are related to each other in structure and instructional approach. However, they function as three independent tutoring environments that support the recursive writing process. The independent nature of the tools assures the student writer of access to adaptive feedback throughout the process. The tools may be used separately or in conjunction with each other. The versatility of R-WISE encourages teachers to view it as an adjunct to their own creativity in the classroom. This is important because research into the effectiveness of advanced educational technology indicates that establishing a partnership between the teacher and the pedagogy of the software is vital to producing performance gains. While curriculum pacing was a constant problem, teachers were able to cope and make it work. Coordinating the classroom curriculum with appropriate writing assignments for the R-WISE lab forced teachers to become aware of approaching lab dates. They also had to arrange the classroom curriculum so that students were at an appropriate place in the literature in order to write an essay on an assigned topic. Sometimes students began an essay in the lab and then finished it at a later date when the reading of the literary piece was completed. Since the R-WISE tutor is not intended to be a product-oriented tutor, having students begin writing in the lab and finish in the classroom is not a problem. The tutor provides help for students to write a finished essay; however, completing an essay is not the sole aim of the tutor. The biggest responsibility for the teacher was in following Pre-lab, In-lab and Post-lab procedures. R-WISE relies on teachers to instruct students on the writing process and on computer software terminology in a "pre-lab" setting. While in the computer lab, teachers must partner with the software by monitoring and nudging students along so that they get the most out of their session with the software. Lab days are hectic. Possibly the biggest factor in motivating students for the "in-lab" experiences is using the writing the student has produced with R-WISE for follow-up lessons. Many teachers find this kind of follow-up time-consuming, yet most recognize its validity. Mediating Software Design Changes The R-WISE project used an integrated team approach in developing the software, drawing from the expertise of classroom teachers, instructional designers, cognitive psychologists, rhetoricians and software engineers. Having classroom teachers balance theory with practice smoothed the way for actual implementation in the classroom. During the pilot study, MacArthur teachers who were members of the design team mentored the other participants, holding weekly meetings to discuss lesson plans, to formulate software change requests, and to plan curriculum innovations. Therefore, many changes occurred during the academic year of 1992-93. Even now, modifications in the software are necessary, but not nearly as often as in that first year. From the beginning, teachers offered suggestions and criticism as they observed their students at work on R-WISE. This was a wonderfully empowering and motivating aspect of the experience. With each software change came pride in knowing that the students had helped improve a piece of technology. Students, however, did not always see things the same way. When a recurring bug created problems, students became frustrated, sometimes angry. Teachers had to intercede and help them realize that even with the problems ¬ which are typical in Beta versions of software ¬ they were still learning and improving not only their writing and thinking skills, but also their computer skills. This worked better with some students than with others. The occasional system crash was frustrating for everyone, but overall relatively few students lost their work. Now after three years, the problems are minimal and the R-WISE software has become much more student- and teacher-friendly. It is important to emphasize that even when problems occurred in that first and most difficult year, students still wanted to go to the computer lab. |
Innovative teachers know how to take risks; in fact, they often thrive on it. | Harvesting Results To provide an empirical measure for the software's effect, a second high school ¬ within the same district and demographically similar to MacArthur ¬ was selected as a control. A pre-test writing sample was given at the beginning of the second semester (January 1993) and an equivalent post-test writing sample was given at the end of the second semester (May 1993). We devised a rubric for holistic scoring of the papers on a 1 to 6 scale, and then had the approximately 2,200 samples evaluated by an independent, commercial scoring agency. Inter-rater reliability for the standard two readings was. 79. The differences in means on pre- and post-test for both control and treatment were deemed statistically significant using t-tests. We are encouraged by the approximately 18% gain of the treatment over the control group, but are cautious in our interpretations given that the software was still in development during the study. Today, R-WISE has just completed its third year of field evaluation. The number of test-bed sites has grown from the single school in San Antonio to include eight additional public schools, one alternative school, one community college and one four-year college. These sites are located in six states: Colorado, New York, Pennsylvania, New Mexico, Ohio and Texas. In academic year 1994 -95, over 3,000 students used R-WISE. MacArthur teachers continue their active role in the project by developing training materials for R-WISE, participating in site visits to other schools, and sharing their insights with colleagues in the project. As for the future of R-WISE, after careful analysis of the findings of these three years, the software will enter into a technology transfer phase: meaning that a commercial partner will "productize" the software and make it available on the open market. The promise of advanced technology in the classroom is great. However, much of what is offered today has not undergone a rigorous process of iterative development based on actual use in the classroom. We are pleased to have been part of a project that recognizes the crucial role of the classroom teacher in the success of computerized instruction. Patricia Carlson was the Principal Designer and Technical Lead for R-WISE, working with the U.S. Air Force's Intelligent Training Branch, Armstrong Laboratory since 1989, while on leave from Rose-Hulman Institute of Technology. Her areas of expertise include the cognitive design of multimedia and the integration of advanced media into the classroom. She is now back at Rose-Hulman. E-mail: [email protected] Elaine Hitzfelder, a veteran 22-year teacher, approaches new teaching techniques with enthusiasm. She has conducted numerous workshops on cooperative learning and modeling teaching methods. Contracted by the North East Independent School District to write innovative English curriculum, she was called on to be a subject matter expert at the inception of R-WISE in 1991. Timothy Hudson, having taught at MacArthur for four years, incorporates technology in both his English and his media courses. He has designed a new English course, Film Analysis, which will be offered to 11th and 12th grade students throughout the district. Hudson has been a member of the R-WISE development team since 1992. D'Anne Redmon has spent 17 of her 20 years teaching English and/or journalism in San Antonio's North East Independent School District. Playing an instrumental role in incorporating computer technology in her district's journalism classrooms, she was equally excited at the prospect of using computer technology in the English classroom. Redmon has presented workshops on using media in the classroom at state and national conventions. She has been working with the R-WISE project since 1992.
|