Research: Young Students Learn Better with Mix of Virtual and Real Worlds

• By Dian Schaffhauser
• 05/11/15

Young learners do up to five times better when instruction combines the real world with the virtual world. That's the finding from Carnegie Mellon University, where researchers came up with a test to figure out how technology could best contribute to learning.

"NoRILLA," as the testing platform is called, is a mixed-reality set-up that bridges physical and virtual worlds. The system includes software and hardware components, including a computer depth camera (Microsoft's Kinect for Windows) to provide personalized feedback while experimenting in the real world.

The researchers designed an experiment to see if 92 children aged six to eight learned better in a mixed reality or on a screen-only educational game. The test also explored what impact the addition of a physical component (like shaking the screen) had on the students' enjoyment of the activity.

In the experiment students played one of four versions of "EarthShake," a game featuring a friendly gorilla and towers made from blocks. The game teaches students basic physics principles such as stability and balance.

In a mixed-reality version of the game, the gorilla asked the students which of two block towers on an earthquake table in front of them would fall first. The students used a mouse to select their choice on the projected interface. After discussing why they thought the tower would fall, the students used a mouse to press the shake button on the projected screen, and the earthquake table was activated with a switch. Once the tower fell, the game recognized the results in the real world and gave interactive feedback accordingly.

When students chose the correct tower, the gorilla congratulated them and asked why they thought it fell first. If they chose the wrong tower, the gorilla would inform them. Then each group was asked to select the reason why they thought the given tower fell from a list of possible explanations.

Then the researchers offered three variations on that approach. In one version, the students played the mixed-reality game but could control the earthquake table themselves with a physical switch. In a second version students played a screen-only version of the game on a laptop, but answered questions and pressed the shake button with a mouse; there was no physical interaction with the game. In the third version, students played the screen-only version on a tablet and could shake the tablet at earthquake time. Testing was done before and after the game to measure learning.

The mixed-reality variations improved learning by almost five times more than the screen-only alternatives. On top of that, students enjoyed the game more in the mixed-reality conditions. However, the addition of the physical controls — shaking the tablet or pushing the earthquake button — did little to improve either learning or enjoyment of the game.

As the researchers concluded in a paper on the experiment, "Mixed-reality games that support physical observation in the real world have a great potential to enhance learning and enjoyment for young children."

Next, they expect to extend the experiment to understand how learning and enjoyment results compare when more elaborate hands-on activities are added, such as children building with the blocks. They also expect to expand the game for different content areas, such as using a balance scale rather than an earthquake table.

PhD student Nesra Yannier is working on the project with Kenneth Koedinger and Scott Hudson, two professors at the school's Human Computer Interaction Institute. The team recently presented a paper on their findings at the Association for Computing Machinery's Conference on Human Factors in Computing Systems in Seoul.