Ed Games Need To Fill in Assessment Gaps To Be Fully Useful

• By Dian Schaffhauser
• 02/23/15

A joint study on game-playing in K-12 has found that educational game developers could improve the learning impact by making a few changes. The "A-Games Project" found that the tracking of student learning and linkages between game activities and curriculum could be better. The project is a joint effort of the University of Michigan School of Information and School of Education and New York University's Steinhardt School of Culture, Education and Human Development.

A-GAMES stands for "Analyzing Games for Assessment in Math, English language arts, Social Studies and Science." Among the games used in the study are these: Do I Have a Right? from iCivics; the Sports Network 2 from Classroom; Citizen Science from Games+Learning+Society; and Guts & Bolts from BrainPOP. All are freely available online.

"We looked at how various features support educators in knowing what students know," said Barry Fishman, a U Michigan professor in both the schools of information and education. "Our report is a call to the game development community about where they might focus next to make educational games even more useful."

A previous study — a nationwide survey of teachers about how the use of games intersected with formative assessment — found that 57 percent of educators use digital games at least weekly in the classroom. Formative assessments are those tests or quizzes given to measure student learning along the way; a summative assessment measures learning at the end of a unit or school term.

The latest study was based on observations and interviews with 30 middle schools teachers (grades 5-8) in public and private schools in New York. Here the focus was on how teachers used or ignored various features within each game that could be used for formative assessments. Those included:

• Points, scores and stars;
• Player feedback;
• Dashboards of player progress;
• Screen capture and annotations;
• Essential questions;
• Review questions; and
• Quizzes.

Some features were less useful than others in helping teachers understand a student's progress. For example, teachers often had trouble configuring dashboards, and scores weren't necessarily linked to learning goals. The most surprising finding for Fishman was that the "common mechanisms" in games for reporting progress, such as points and stars, aren't all that useful to teachers. "For many of the teachers, it was hard to tell from these progress markers what the students were learning. So a student has 100 points. Does that mean they are learning addition?"

However, the researchers also found that teachers use functionality outside of the game to help them with formative assessments. All of the teachers in the second study were given access to GameUp, a game portal from BrainPOP. That portal included SnapThought, a "photo reflection" or screen capture tool. Teachers could assign students to grab images at certain points in the game to record their progress and in some instances explain what they were thinking as they played. Nearly all the teachers who tried SnapThought said they valued it. However, its use also tended to interrupt the "flow of excitement."

"Not all features were equally useful, and often, utility was related to how the feature was implemented," concluded the report's authors. "Our observations and conversations with teachers revealed both promise and pitfalls, but in general the shortcomings faced by teachers in our study can be repaired through either improvements in game design, improvements in support around the game, or both."

The complete study is available online at the A-Games Project Web site.