An 'A' in Abstractions
With assessment attracting increased focus in the push for 21st century skills learning, a new issue arises: How do you measure the immeasurable?
If a district head or state superintendent wanted to intensify the focus of his curriculum on 21st century skills, he would do well to pay a visit to Tucson’s Catalina Foothills School District (CFSD). There, on the Arizona city’s northeastern edge, educators and administrators have decided to go all in on 21st century skills, and in so doing have revamped every subject area from K through 12.
Core subjects and courses remain, but the traditional emphasis on content mastery has had to clear some space for the teaching of a dozen higher-order skills the district’s new strategic plan has identified as critical to students’ footing in the new century, such as teamwork, self direction, and leadership. As an example, to become more adept at “systems thinking,” defined by the district as the ability to “see a whole, a web of relationships, rather than focusing only on the detail of any particular piece,” students are taught to recognize patterns of change over time. In literature, that translates to identifying variations in character traits; in math, it means learning to see changes in a bank account balance and then graph them.
“These skills aren’t new,” says Mary Jo Conery, CFSD’s assistant superintendent for 21st century learning. “Systems thinking, critical thinking, use of technology—they’re not new. Teachers have been incorporating them into their curriculum for a number of years. But what’s different for us this time around is we are going to ensure that all students receive this curriculum in a formal way, that all students have access to developing 21st century skills during the 13 years they are in our system.”
NEW TECHIES Monica Martinez (left) and Dan Liebert favor performance-based assessments.
The backbone of the district’s redesigned curriculum is a meticulous system of performance rubrics that describes what is expected of students across every subject in every grade in each of the 12 designated learning skills, Conery explains. A former principal in the district, she entered her current post in 2005 charged expressly with helping develop and then put into practice the new instructional framework.
According to Conery, the rubrics establish what a capable critical thinker or problem solver or self-directed learner “looks like.” But how can a student demonstrate that he is one? Devising assessments that align to these 21st century skills is “the next logical step,” in Conery’s view. It’s an ongoing effort that has little use for bubble sheets and true-false questions. “What we’re training teachers to do,” Conery says, “is create assessments that use real-world tools and are relevant and purposeful to our students.”
“It’s a dilemma,” says Kathy Boone, assistant director in the West Virginia Department of Education’s Office of Instructional Technology. West Virginia is seen as a leader in 21st century skills assessment. “How do you really assess 21st century skills? If they are things like problem solving and creativity, then you’ve got to give the kids an opportunity to show that.”
Performing Authentic Tasks
The attention being given to assessment represents a push forward for the 21st century skills movement, from explaining and advocating for 21st century skills to figuring out how to test and report on how well students are grasping them.
| Catalina Foothills School District (AZ) assesses students in 12 learning skills, divided among three categories: |
| Personal and Social Responsibility |
Learning and Thinking Skills Digital Age Literacy |
Digital Age Literacy |
| Leadership |
Data Analysis |
Cultural Competence |
| Productivity |
Systems Thinking |
Global Awareness |
| Self Direction |
Scientific Inquiry |
Interactive Communication |
| Teamwork |
Critical and Creative Thinking |
Technology and Tools |
“This whole area is really emerging,” Boone says. Witness US Secretary of Education Arne Duncan’s pledge last June to reserve $350 million in American Reinvestment and Recovery Act (ARRA) funds to support states in creating new assessments tied to a common set of standards. Duncan called for a new way to evaluate students: “We need tests that measure whether students are mastering complex materials and can apply their knowledge in ways that show that they are ready for college and careers….We need tests that go beyond multiple choice.”
But creating them is a major undertaking that rides on a critical question: How do you assess abstractions like collaboration and leadership while still maintaining state content standards? Or put another way, how do you do two kinds of assessment—content and elevated learning?
The answer that many school systems have arrived at is: You don’t. You integrate assessments that measure both simultaneously. That requires a kind of evaluation that no longer takes place after instruction is completed, but is performance based and embedded organically into projects.
“What we mean by performance-based assessments is that rich, authentic tasks are being performed; these are open-item responses,” says Valerie Greenhill, director of strategic initiatives for the advocacy group Partnership for 21st Century Skills.
Boone says that traditional test taking just doesn’t fit with West Virginia’s statewide implementation of what she calls a “triangulated” curriculum, melding content and 21st century skills with the use of technology tools to support the learning of both. “Can you really take a one-time, multiple-choice objective test to determine if you’re creative and if you can communicate?” she asks.
Clearly not, the state has decided, so instead its students are evaluated by their performance on several projects during the year. For example, eighth-graders might be asked in a math class to develop a model to predict real estate prices. They have to collect the data online (information and communications technology); analyze it using math (content); then, working in teams (collaboration), use Microsoft Excel spreadsheets to form a scatterplot of the data (technology) and draw conclusions based on it (creative thinking). Finally, they present a report to the class (communication). In another math project Boone describes, critical thinking is assessed through an exercise in which students look at “how various pieces of data relate and what kind of correlations and conclusions can be drawn based on the data.”
Similar practice is followed by the 41 schools of the New Tech Network, a cross-country association of high schools whose curriculum is based largely on the teaching of 21st century skills through project-based learning. “To assess 21st century skills you have to have something to assess other than pen-and-paper tests,” says Dan Liebert, principal of Tech Valley High School in Rensselaer, NY, one of the New Tech sites. “Students have to do something.”
Liebert uses a freshman math and science project titled “Is the Hudson Nasty?” as an example: Is the Hudson River polluted? Would you eat fish from it? Drink water from it? To determine the answers, students must collect and display data and write a report.
“They’re writing, they’re doing scientific investigation,” Lieber says. “They’re going out into the river with their boots on and collecting soil and water samples, and then they bring them back to the lab. Along the way, they have to take some traditional tests and quizzes. So it’s not like we’re throwing out traditional assessment altogether. It’s within the goal of accomplishing a project, not separated or segregated from its use. We’re trying to show the kids that what we’re teaching actually has utility.”
For the assessment to mean something as well, it has to reach well beyond conventional grading. “Whether they’re producing an artifact or a presentation or some sort of solution to a problem, that whole process has to be assessed,” Liebert says. Tech Valley students receive scores in each of eight different learning measures plus content mastery.
“It’s not like we’re throwing out traditional assessment altogether. It’s within the goal of accomplishing a project. We’re trying to show the kids that what we’re teaching actually has utility.”
“A kid could get a 95 in collaboration,” Liebert says. “He’s the cheerleader on the team, he’s getting everybody to get their stuff done. And maybe he’s really good with communication; he’s a great presenter or he’s a great writer. But maybe he struggles with the algebra and gets a 72 when he takes his traditional test on the content. But he’s being rewarded for being a great collaborator and a great communicator and doing the things he does well.” At the end of a semester, students get a single course grade that factors in their performance in all the learning skills.
The many-sided assessments have come to take their place in student life. “I hear students walking around all the time saying, ‘I’m doing well in collaboration, but I need to work on my oral communication,’” says Paul Curtis, the New Tech Network’s assistant director of school development. “We’re trying to get away from assessing tasks and more onto assessing skills: What does that assignment tell us about a student’s ability in multiple areas?”
All-Access Gradebooks
Another important differentiator between traditional and 21st century assessments is transparency. Gradebooks aren’t kept under lock and key. Performance rubrics and students’ scores are visible online for quick access by students, faculty, and parents.
CFSD uses a web-based tool it calls the Parent Internet Viewer. “Users get to see how students are doing on every assignment, project, and assessment that they have had to date in their class,” Conery says. Clicking on an individual assignment drills down a level to the component learning skills attached to that assignment, revealing how a student fared on each one.
The New Tech Network makes its curricular resources available through a learning management system it developed in-house at its home base in Napa, CA, and named PeBL (pronounced pebble), a slightly contorted acronym for “electronic project-based learning.” Gradebooks are accessed via a separate tool called GradePortal, where students can see their overall scores in each of their courses as well as how they are performing in the different learning skills. Plans are for PeBL and GradePortal to merge into one application in the fall.
Will We Pay the Price?
When US Secretary of Education Arne Duncan announced he was setting aside funds for the creation of 21st century assessments, he conceded that cost was a consideration. “We know that these kinds of tests are expensive to develop,” he said.
Harvard University’s Tony Wagner believes the greater cost would be to not upgrade state testing methods. “It’s insane to give less expensive assessments to every student and think you’re getting more accurate information from them,” he says.
Wagner is co-director of the Change Leadership Group at Harvard’s Graduate School of Education and author of The Global Achievement Gap: Why Even Our Best Schools Don’t Teach the New Survival Skills Our Children Need—And What We Can Do About It (Jossey-Bass, 2008). He says that better, though costlier, assessments could be given to a demographic sample, rather than to every student, with little net cost increase while giving districts a clearer picture of where help is needed. “We have to get rid of the idea of punitive-based testing systems,” Wagner says, “but instead give diagnostic and auditing ones, then determine what districts need improvement and what kind of improvement.”
He offers the example of a US history or social studies assessment that might simply provide the Bill of Rights and give students two hours to answer on paper the question, “Which right is the most important for the preservation of democracy?”
Wagner, a former high school teacher and school principal, argues that this is a more authentic assessment than traditional methods. “Teaching to that kind of test would result in better thinking and better writing,” he says, than the multiple-choice exams most states employ. He doesn’t reference those skills arbitrarily: In a recent survey, “writing in English” and “written communications” were the basic and applied skills, respectively, at which 431 employers representing more than 2 million US employees found their hires deficient.
“All of the teachers’ projects are housed there, the performance rubrics and scoring assessments are housed there—you name it, it’s there,” says the New Tech Network’s president, Monica Martinez. “Students see what it’s going to take for them to get an A, versus what will limit them to a B or a C.”
As an example, when Tech Valley students check online for a breakdown on a project merging algebra and environmental science, they’ll find a description of what they must do to get a grade of 90 to 100 in self direction: All benchmarks are completed on time. Group digital and print folders include almost daily posting of works in progress. Group has posted revisions to the contract and pacing chart to improve them as the project proceeds. The project is on time and complete.
“One 21st century skill is being responsible for your own learning,” says West Virginia’s Boone. She notes her state’s implementation of TechSteps, a tech literacy program from software provider SchoolKit, to support technology use in all areas of its curriculum. The scoring criteria for all TechSteps-related exercises are accessible to students at the program’s website. “Students can look at a rubric, work toward it, and compare their work against it. They become responsible for self-directed learning.”
‘Reprofessionalizing’ Teaching
Just as students are being asked to upgrade their toolbox, 21st century skills assessment places new demands on individual teachers. They are no longer simply instructing and then measuring absorption rates, and so are expected to develop and implement assessments that work for their own classrooms.
“The professional development aspect of this is huge,” says Bernie Trilling, a member of the Partnership for 21st Century Skills’ Strategic Council and the co-author of 21st Century Skills: Learning for Life in Our Times (Jossey-Bass, 2009). “We’re talking about the ‘reprofessionalization’ of teaching so teachers are skilled in how to do these assessments.”
The California-based nonprofit Buck Institute for Education conducts workshops ranging from three days to three months to teach teachers “best practices in PBL [project-based learning] design, assessment, and management.” The organization gets high marks from Carla Williamson, executive director in the West Virginia Department of Education’s Office of Instruction, which has worked with the Buck Institute for two years. “It’s very effective,” she says, “which you can see when you look at the project designs our teachers are creating under its tutelage.”
You Can Look It Up
Curriculum-embedded assessment: “If [instruction is] properly designed, students should not be able to tell whether they are being taught or assessed.”
Source: New Horizons Learning (newhorizons.org) glossary
Tech Valley High School uses a training program it calls “critical friendship.” Every Friday, teachers present to their colleagues about projects they are preparing to roll out to their students: their purpose, how they conform to state standards, the performance rubric, and the assessment piece. The group then provides feedback and analysis.
“A teacher can write a rubric that he thinks is crystal clear,” Liebert says, “but then a 14-year-old looks at it and says, ‘Eclectic? What does that mean?’ Oftentimes it will be the language and clarity that need to be worked on. We want to make sure that the work we’re doing, the projects we’re producing, actually develop the kinds of learning outcomes that we say we want.”
Liebert says the training sessions are key to bringing structure to assessing largely intangible skills. “We want evaluations of self direction or communication in one course to be as similar as possible to evaluations of self direction or communication in another. The only way you can get more consistency is to develop consistent habits among the professionals who are doing them. You have to have this kind of professional collaborative culture among your staff because this is hard. It’s not easy. The easiest thing to do is hand out textbooks and say, ‘Let’s do Chapter 1, and then let’s do Chapter 2, and then let’s do Chapter 3.’”
What still needs to be worked out is how these newfangled assessments will be treated on college-bound transcripts. Conery says CFSD has chosen to move a little slower as a nod to higher ed admissions policies. Though its primary and middle school students receive summative scores that correspond directly to the performance rubrics—4 for advanced, 3 for proficient, 2 for basic, 1 for novice—the district still uses letter grades for high school students.
“There was concern that students might be disadvantaged because colleges and universities wouldn’t be able to interpret the rubric scoring system,” Conery says.
At Tech Valley High, the issue is still a ways off, but it is being worked on. The school is only in its third year, so its inaugural class of ninth-graders will graduate in 2011. “We’re anticipating giving our students a lot of assistance, explaining what kind of school they came from,” Liebert says. “They’ll also get narratives and explanations as part of their numbers. We’re not just going to send them off with an 85. We’re going to send them off with what that 85 means.”
Ultimately, Liebert argues, it means a student’s readiness for real-world circumstances. “You can get a 97 in AP history, but if you go to the Times Union and you’re supposed to help a journalist with a piece of writing and you can’t do anything, you don’t know how to collaborate, you don’t ask questions well, you’re kind of arrogant, what good are you?” 