Vanderbilt Touch App Aims To Help Visually Impaired Students

• By Dian Schaffhauser
• 03/07/12

The same technology that makes the Android game Teeter buzz and vibrate in your hand as the little rolling steely knocks into walls on its way to the green hole may help visually impaired students learn math and other subjects, if haptics research going on at Vanderbilt University in Nashville, TN succeeds. Jenna Gorlewicz, a graduate student in the institution's Initiative in Surgery and Engineering, has created an Android app that uses haptics technology to help high school students without sight to feel math shapes on a tablet. Haptic technology provides touch feedback through device motion, such as vibrations.

In a project funded by the National Science Foundation, Gorlewicz purchased and programmed tablet computers to vibrate or generate a specific tone when a student touches a line, curve, or shape on the screen. The devices can generate vibrations with a number of different frequencies and hundreds of different sounds. This allows the researcher to assign different tactile or audio signals to different features. For instance, in an exercise with an X-Y axis, she can set the horizontal and vertical lines to vibrate at different frequencies and set points to make a certain tone. That allows the user to tell the difference between gridlines and points on the grid.

Graduate student Jenna Gorlewicz works with high schoolers as they try out her haptics-based app for teaching math concepts. (Pat Slattery/Vanderbilt University)

When she began the project 18 months ago, Gorlewicz had to spend $2500 to obtain a tablet primarily used in industrial and commercial applications. It was the only one that provided tactile feedback at the time. Now, haptics is surfacing in consumer-style tablets selling for $300. Some industry observers are predicting that Apple's upcoming iPad 3 may include haptics as well.

"When I began reading articles about haptic technology being incorporated into these new touchscreen devices, I realized that the people who really need haptics are people with impaired vision because they heavily rely on their sense of touch to ‘see' the world around them." Gorlewicz said. "I love math and I love teaching, so I immediately thought of using them for math education, because it has such a strong visual component."

Gorlewicz is testing her app with two students attending Nashville-based Hillsboro High School. In a video demonstrating the technology, Kira, one of the students, runs her fingertip across the touchscreen of a tablet and listens to the muted clicking that results. She announces, "Well, it's a line."

Currently, said the students' teacher Ann Smith, she uses pipe cleaners and pushpins on a large grid board to help the students visualize with their fingers the lines, shapes, and grids that are being discussed in their math classes. But, she noted, "It's slower and it's not as accurate."

The Android app includes a sketching program that allows the student to draw the shape and then adapt it on the fly, so the student can work in real time and follow it with his or her finger.

Kira said she believes the app would be a "really good graphing calculator, if you pair it with the right software." That would allow her to work on problems, "like everybody else in class." She and Quinn, the other student, have told Gorlewicz that they could use her technology in other ways too, from laying out a tactile periodic table of elements to playing video games.

The app is currently in the design phase.