Assessment | September 2012 Digital Edition
Solving Formative Assessment's Catch-22
Teachers often have a hard time embedding assessment in their instruction, but some technologies are making it a little easier.
- By John K. Waters
- 10/10/12
Illustration by Anne Kobayashi |
There are few challenges in K-12 education quite as problematic as formative assessment. This process, through which teachers evaluate how well their students are learning while they are teaching them, is almost universally regarded as a best practice. And yet, as researchers Neil T. Heffernan and Kenneth R. Koedinger so eloquently expressed it in their recently published paper ("Integrating Assessment Within Instruction"), it is a practice that can put teachers in a bind: "They are being asked to assess students to drive instruction, but every minute spent on assessments is a minute lost to instruction."
This pedagogical Catch-22 might explain the relatively slow evolution of tech tools in this category. Even as online learning environments and instructional systems add increasingly sophisticated features for summative assessment (periodic evaluation used to measure student growth after instruction--in other words, tests), the technologies that support true formative assessment are advancing more slowly.
But they are advancing. A case in point: the student response system (SRS), better known as classroom clickers. Clickers started out as modest student polling devices, but the latest generation of the technology is integrated with what amounts to formative assessment platforms that include everything from interactive whiteboards to iPads. Some vendors are even supporting students' responses to interactive questions via their own mobile devices. Companies like i>clicker, Smart Technologies, Turning Technologies, iRespond, Engaging Technologies, Renaissance Learning, and GoSoapBox, among others, are part of this trend, along with what has been called "data-driven differentiated instruction."
Ingrid Oyen, a third-grade teacher at Stella May Swartz Elementary School in Oakbrook Terrace, IL, has been using one of the latest incarnations of the SRS in her classroom for the past year: eInstruction's Insight 360, which integrates a mobile interactive whiteboard, the Mobi 360 for iPad application, some advanced software, and two versions of student response pads.
"They're very powerful tools for formative assessment," Oyen says. "They allow me to respond to a (missed) concept, teach to that, and move them on immediately, instead of two days later when I get through grading a stack of papers. What's the question that most kids missed? I know that right now and can teach to that right now."
Mark Hupp, technology director for Oyen's district, Salt Creek School District 48, says the SRS has come a long way in recent years. "The new system we're using from eInstruction is a lot more accessible and easier to use," he says. "It has two ways you can use it: a kind of ad hoc, verbal mode that requires almost no preparation from the teacher, and script mode that allows you to prepare your lessons by using your old digital content, and then tag a keynote slide or a PDF file with an answer.
"You can line up all your digital content in the sequence in which you want to teach your lesson in a highly scripted way, and embed questions strategically throughout there. Teachers like Ingrid will use the verbal mode a lot; others in higher grades may script their lessons much more," Hupp says. "This is a logical conclusion (to the evolution) of the traditional clickers."
Oyen says using the clickers has improved her students' test scores, especially in math, and reduced the amount of time she wastes in class. "I think it has taught them to read carefully, commit to an answer, enter it, then adjust, learn it, commit again, and just keep going," she says. "They get immediate feedback, and I get immediate feedback. It's not just convenient, it impacts the teaching directly."
A noteworthy feature of the eInstruction product is one that's becoming more common: integration with a substantial database. Insight 360 comes with a CD containing approximately 9,000 teacher edition textbooks, says Eric Rohy, eInstruction's vice president of product management, with lists of assessment questions.
"There's definitely an understanding among teachers that formative assessment is a best practice and that there's a lot of value in doing it," Rohy says. "But what we've been hearing from teachers for a while now is that the available technology for doing it is not easy to learn, or asks too much of the teacher. To conduct formative assessments, you really need great assessment questions. Sometimes teachers are great at that; some are not, and they're looking for some guidance or some standard content at their fingertips ready to use in their classes."
The Teacher Response
But recent advances notwithstanding, there's an inherent problem with clicker systems when it comes to effective formative assessment, says author and teacher Harry Grover Tuttle: Even with the real-time, in-the-trenches knowledge they can provide, teachers aren't necessarily doing anything with that knowledge.
"The problem I see with most clicker systems is, frankly, with the teachers using them," Tuttle says. "They get some feedback and say, okay, I see that many of you aren't getting this, and then they keep right on going. They don't implement corrective formative strategies right then and there, so the clickers serve no purpose."
Tuttle, who has been a classroom teacher (English and Spanish) and a technology coordinator at the district level, has written several books, including Formative Assessment: Responding to Your Students (Eye on Education, 2009)--and he's no fan of formative assessment systems embedded in learning management systems.
"Most large LMSes call the testing of students 'formative assessment,'" Tuttle says, "but it's really summative, because they only show how well the student is doing. They don't provide specific strategies for improvement. Students can be tested to death without being given ways to overcome their learning gaps. Most don't even allow you to put in data that isn't about a grade. To me, that's a fundamental weakness of these programs. To me, they miss the point."
And the point is?
"It's more of a question, really," he says. "Is the most important thing what the teacher is teaching or what the students are learning? If the answer to that question is what students are learning, then we must monitor what the students are learning constantly to find out if they're learning, and if they're not, help (them) to learn. That's why I think formative assessment is critical."
He adds: "All of the literature on formative assessment says that the longer the time between when the students do something and when they find out whether they're right or wrong and how to do it better, the less likely they're going to make any change. If that's true, then our classrooms really have to be places where, as soon as students do something, they get feedback."
And yet Tuttle eschews even the latest formative assessment products in favor of a student-centered approach that combines cloud-based applications, social media, and students' personal technology. The result is more of a peer-based formative assessment. "I believe that formative assessment works better with a student-centered classroom in which the teacher is a guide on the side, and not the sage on the stage," Tuttle says. "We want our students to be responsible for their own learning."
And perhaps even their own assessments. In Tuttle's Spanish classes, for example, students record their Spanish speaking using Voki, a free, web-based service that allows users to create customized speaking avatars. The avatars are then posted to his class's wiki page, which becomes a kind of e-portfolio. The students then "self-analyze" what they've done well and what they can do better, and post they're going to do better. "It's very easy to use," he says. "They can even do it from their smartphones."
Tuttle also employs Google Forms, an online app for creating questionnaires and surveys. The app allows him to post summaries of student responses to the web, to keep track of students' scores with just-in-time scoring, and to quickly analyze trends for any student. He customizes games from the Quia web-based game and quiz application, which students can play on their own time. He generates PowerPoint quizzes that explain "why the correct answer is correct." And he uses Quick Response (QR) codes that direct students to different ways of learning a concept (links to YouTube videos, websites, and so forth).
A Social Approach
Tuttle isn't the only one embracing these kinds of technologies for formative assessment. Matt Wingfield, managing director at TAG Developments, sees social media and web-based tools as signposts for developers of assessment tools.
"I think it's fair to say that the classroom clickers are at one end of the spectrum," Wingfield says. "This isn't what you would call rich assessment, but the technology can be used effectively for managing a class based on real-time feedback. And teachers do feel quite comfortable with that technology."
He believes teachers can go at least a step or two further with real-time assessment by employing social media, something students are already very comfortable with anyway. "That's very new, and I don't think there's a great deal of awareness of the value there," Wingfield says. "This stuff is just second nature to them, and we should be developing mechanisms that are able to support assessment within that context."
TAG is the e-assessment division of Sherston Software, a British publisher of software for schools. Among other products, the company makes an e-portfolio platform called MAPS, which is widely used in schools in the UK, Australia, Singapore, and Sweden. The product has only recently gained a foothold in the US.
MAPS was created in 2001 as an asynchronous--essentially summative--assessment tool, but its evolution has been toward formative assessment. At first, the overall goal of the product was to build up a picture of the progression of the student over a period of time, and to give the teacher the ability to give that student formative feedback, more or less after the creation of a particular piece of evidence.
But increasingly, as the technology for collaboration and more dynamic types of interaction have evolved, and particularly as social media have taken hold, Wingfield says he's seen demand from customers who want MAPS to support that kind of assessment, too.
Currently the company is developing a product called E-scape that employs a model Wingfield calls "storyboard e-portfolioing."
"The key thing with this project is to be able to see the journey the student has taken, and be able to zoom in on the different pieces of evidence the student has created," he says, "as well as be able to provide feedback to the teacher in real time to guide the student in the different steps where it's appropriate."
E-scape uses the MAPS e-portfolio system to manage collaboration among students during a particular project or activity, and to provide feedback in real time. The students capture their own evidence as it is being generated, dynamically, creating in the process a "storybook" portfolio to show the learning process.
"So you're not losing the 'eureka' moment the students have as they're evolving their project," Wingfield says. "When a teacher comes to assess this work, they can see how the student's thinking has been influenced by other students. Call it real-time evidence with reflection."
Or maybe summative-slash-formative assessment.
The company has also been working on a project for the government of Singapore, which involves using social media tools to create a platform for peer-to-peer formative assessment that links the MAPS platform with Twitter and Facebook.
"We're using latent semantic indexing as a mechanism to understand both the syntax the students are using, the meaning of the words they're using, and relating that back to the skills and competencies the teacher is looking for," Wingfield explains. "But it's also coupling that mechanism with social networking analysis that measures the dynamics of the interactions from student to student."
That analysis can present, for example, a graphical representation of how all the students are interacting with each other, Wingfield says, allowing teachers to see who is interacting with whom, where the conversation is heading, and who is dominating the discussion. It also measures the frequency of collaboration, the length of the interactions, and their quality.
The company has yet to name this project, Wingfield says. In-house they're calling it Collaborative Online Learning Tools, or COL Tools for short.
"This idea of getting feedback in real time from a classroom experience has been a longtime aspiration of the field," observes Larry Berger, cofounder and executive chairman of Wireless Generation. "But embedding assessment into the learning experience can be challenging, because we're asking them to assess while they teach."
Over the Shoulder Assessment
Berger's Brooklyn, NY-based company is a provider of educational technologies for preK-12 environments, including a range of assessment and analysis tools. The company's flagship product, Burst:Reading, is a new approach to curriculum publishing designed to enable teachers to differentiate instruction through analysis of students' formative assessment data. The system groups students based upon shared learning needs, and produces lessons designed for each group, which are delivered by the teacher.
The company is currently testing a new formative assessment system based on something teachers have always done: walking around behind their students, looking at their work, and giving them feedback.
"We're just adding a little bit of instrumentation," Berger says. "Put a tool like an iPad in a teacher's hands with the right software, and suddenly this becomes a data-capture moment. But it's also a moment where the teacher is being supported in giving students feedback. Sometimes it's hard to remember which kid is where in the development of his or her writing skills, and sometimes it's hard to remember what kind of feedback a kid at that stage needs."
The new system will support what the company calls "over the shoulder conferences," Berger says. The system will include an iPad or other mobile device with software designed to allow teachers to record what they see a student working on, and then suggest what the student's next writing experience ought to be to develop the next skill in the developmental sequence.
"There are many times when giving the right, highly personal feedback to a student is a very valuable thing for a teacher to be able to do," Berger says. "If some tech tools can support that and highlight some opportunities--some of the best thinking about what kinds of feedback a student might need at a certain level and stage of development--then that's something a teacher will be willing to click a button to get while they're teaching."
Parents Like Formative Assessment--and Their Kids' Teachers Parents want classroom teachers to be the primary decision-makers when it comes to what their children learn--and they want them to have the opportunity to use formative assessment when they make those decisions. That is one of several conclusions that the Northwest Evaluation Association (NWEA) came to after it commissioned a research project on what both parents and educators want from K-12 assessment. In a national survey of more than 1,000 parents, 1,000 teachers, and 200 district administrators, 40 percent of the parents said they believed classroom teachers should make most decisions about what their students learn. Not surprisingly, even more teachers (50 percent) agreed, while only 20 percent of district administrators thought the same thing. "That's because they're the ones who actually know something about the kids," says NWEA President Matt Chapman of the parents' sentiments about their children's teachers. A sizable portion of parents surveyed (84 percent) said they believed formative assessment was either "extremely" or "very" useful, compared with only 44 percent who said the same thing about summative assessment. "Parents were far more aware and supportive than we expected," Chapman said. "That, to me, is news." What's more, parents believe that the sooner the formative assessment takes place, the better. Almost half of the parents surveyed (43 percent) said they believe assessments begin to lose their relevance within one month of the time that students covered the material to be assessed in the classroom. On the other hand, 28 percent said it would not lose its relevance until two to three months later and a mere 12 percent believed it was possible to wait six months before students could be tested. "Parents are aware of the shelf life of these assessments," Chapman said. The research was conducted for NWEA by Grunwald Associates. NWEA is a nonprofit organization that provides assessment and professional development to school districts, primarily in the Northwestern United States. Grunwald Associates provides research and consulting services focused on education. The complete report is available online. |