Artificial Intelligence: Is the Future Now For A.I.?

• 02/01/09

The feds' reliance on standardized testing excludes the use of artificial intelligence, so school districts are forging ahead with the technology to power their own formative assessments.

IN THE STEVEN SPIELBERG MOVIE A.I. ARTIFICIAL INTELLIGENCE, the hero, an android, wants to become human because he believes it is a better state than his own. But sadly, he discovers there are some things that artificial intelligence just can't do. AI's condition is much the same in real life. The computer science visionaries of the 1950s and '60s predicted with almost total certainty that machines would soon be able to understand and think like humans. But the evolution of AI has been far more prosaic-- rather than priming a generation of robots to take over the world, it has become a pervasive tool that drives much of the technology behind finance, manufacturing, medicine, and a host of other industries.

In education, however, AI hasn't made much headway. In the one area where it would seem poised to lend the most benefit-- assessment-- the reliance on standardized tests, intensified by the demands of the No Child Left Behind Act of 2001, which holds schools accountable for whether students pass statewide exams, precludes its use. Standardized tests are graded consistently, with no allowance for individual student abilities or styles of learning, and no place for artificial intelligence's ability to decipher and shore up weaknesses.

This climate of broad-based accountability will make it difficult for AI programs to achieve acceptance at the federal level, says Kathy Mickey, senior analyst and managing editor of media research firm Simba Information's Education Group. Mickey has covered the educational testing market for more than 10 years, and most recently she edited Simba's report, "Academic Testing: PreK-12 Materials Market Analysis and Forecast 2008." She says that there's virtually no intelligent software being used in large summative testing programs.

"A number of products are available that feature AI, but they're highly personalized, very individualized, and focus on intervention and individual education. States have tried to use them, but that isn't what the federal government is looking for."

Critics insist that despite government resistance, one-size-fits all testing will soon be gone. For example, one theory held by some educators uses the concept of disruptive innovation to explain why standardized tests are doomed. The term was coined by Harvard Business School professor Clayton Christensen to describe a new technology, process, or business model that replaces, or disrupts, an earlier one by being much more affordable or simpler to use. In his latest book, Disrupting Class: How Disruptive Innovation Will Change the Way the World Learns (McGraw-Hill, 2008), Christensen and his colleagues, Michael Horn and Curtis Johnson, assert that the US public school classroom model will be "disrupted" by technological innovations based on personalized instruction; as this occurs, student assessment will take place entirely in real time and be tailored to individual needs, phasing out traditional classroom testing.

"AI can't replace human teachers, but if it is done well, it has a role in the classroom. Educators can use all the help they can get."

That breakthrough may be a long time coming. "There's a difference between having a conversation about it and actually installing a testing program that meets the accountability standards of a federal law," Mickey points out. "Your framework is narrowed. There's been so much talk about individualized programs. The technology is there and available, and it has a lot of potential. But I doubt it will be used at the summative level."

Schools, however, aren't waiting on the feds to change course, as they continue to conduct their own individualized assessments. In fact, in 2008, Simba projects a 10 percent leap in the purchase of formative tests, whereby teachers monitor students' progress on a day-to-day basis. That surge, along with the expansion of online testing and increased testing in high schools and for English language learners and special education students, Simba says will propel a K-12 testing market increase from $2.06 billion in 2008 to $2.52 billion by 2011.

Hints of Disruption

Encouraged by the market growth, software builders are making notable enhancements to their AI technology-- to the point where it may be verging on disruption. One vendor sparking buzz is Lexia Learning Systems. While Lexia's AI-based reading programs have been around for years, its latest version of Lexia Reading, launched in September 2007, is more sophisticated than its predecessors. Previous formative testing applications scored student tests in full; Lexia is able to stop on each question to identify problem areas. It harnesses the "intelligent" component of AI by continuously assessing a student's skill level and understanding as the student works on an activity. If the student has not understood the concept, the software identifies the area of weakness and offers additional practice before moving on. During each session, the program generates reports for teachers and administrators about the student's skill level and areas in which he or she needs more instruction.

Based on cognitive psychology theory that increasing phonological awareness-- the ability to identify the sound structure of spoken language-- is a critical component of improving literacy, Reading offers individualized instruction and practice on the phonics and word-recognition skills critical for reading fluency and comprehension. For Hall County Schools in Gainesville, GA, with an enrollment of about 25,000 students, it's the "intrusive nature of the program that's so useful," says Aaron Turpin, Hall County's executive director of information and technology. "Every single response accelerates the student to a higher level of questioning or education, or suggests a remedial path. The difference from other software is that Lexia responds to every single question and makes adjustments. It then takes the information and provides it in a clear format for the teachers to use so they can learn from it as well and modify their instruction." The software has benefited both English and non-English speakers, Turpin adds. "We've seen equal change in both."

Another pioneering approach to AI-based testing comes from Pearson Education, whose Knowledge Technologies group has developed WriteToLearn (WTL), a program that enables students to practice essay writing and summarization skills and have their work measured by Pearson's proprietary Knowledge Analysis Technologies engine. KAT analyzes whole passages of text not just for grammar or tone, but for meaning. The WTL program is based on latent semantic analysis (LSA), a language learning theory and computer model developed at Bell Labs and turned into an educational technology component by the University of Colorado and Pearson's Knowledge Technologies group. LSA is a statistical technique for extracting and representing the similarity among meanings of words and passages by analysis of large bodies of text.

Karen Lochbaum, one of the scientists who developed LSA and now vice president of software engineering at Pearson, believes that WTL is cutting-edge in its judging of writing and reading skills. "What we do for essay scoring is train the KAT engine on papers that are already written and have been scored by human scorers," she says. We analyze the papers in lots of different ways to determine how they got the score they did."

Lochbaum explains that the program evaluates a new essay by using old papers on the same topic as yardsticks. "WTL is unique because it looks at the content and development of what the students are writing. Are they addressing the prompt? Are they stating the main idea and building it up with supplementary ideas? For reading assessments, students read an article and write a summary of what they've read. The program gives feedback on their comprehension of what they've read, and they can revise their work based on that feedback. WTL offers the practice in writing that teachers would like to give with more frequency. It really is about the practice."

WTL has been used successfully at Minnesota's Dunwoody Academy High School, a charter school that provides academic and career-focused training for students from depressed areas of Minneapolis and St. Paul. The impact of the school's adoption of WTL in January 2008 was felt on the state assessment last spring. Seventy-three percent of its ninth-graders-- compared to the state average of 67 percent-- met the requirements of the writing portion of the Minnesota Comprehensive Assessment-II Graduation Required Assessments for Diploma (MCA-II/ GRAD) test, says Duane Dutrieuille, dean of academic and student affairs. Plus, he says, "A very high percentage of our 10thgraders passed the test, with a score above those of the state and urban schools in Minnesota."

While Dutrieuille credits the school's language arts teacher, Beverly Davis, for providing the guidance that made the program so successful, he also considers WTL very beneficial. He cites its outstanding capacity to improve reading and writing abilities. "The key component is that it breaks down all the different categories of writing," Dutrieuille says. "Maybe the students need a little bit more help with their sentence structure. Or maybe they have great thoughts, but have a hard time with grammar or the flow and structure. If you're a writer, the program gives you confidence. And then you can build on it."

Beyond Assessment

What we are seeing in these new software programs is the marked advancement AI has brought to formative testing technology, which now is capable of tutoring students in the fundamental understanding of concepts-- building knowledge in addition to assessing it. Another example is ALEKS, a web-based assessment and learning system from the company of the same name. Developed by scientists at the University of California-Irvine, ALEKS (Assessment and Learning in Knowledge Spaces) asks students personalized assessment questions; the choice of each new question is based on responses to all previous questions. When the assessment is complete, ALEKS delivers a report with a detailed graphic representation of the student's knowledge. But the system can do more than merely reveal what a student does and doesn't know about a specific topic. It also can help a student learn. For example, if a student has answered a math practice question without adding the appropriate numerical units, ALEKS may suggest that the student check the units. The program will offer more practice questions until it determines that the student has mastered the concept.

Schools using ALEKS report significant improvement in students' grasp of concepts. At Monte Vista School District in Colorado, 15 students in grades 2 to 5 who had failed both math and English were placed in a remediation class that uses ALEKS. When students entered the program, their average mastery was 53 concepts. Their goal was to master 50 new math concepts in six weeks. By the end of six weeks, several of the students had met that goal; some had mastered more than 100 new topics. And at Westview High School, part of the Tolleson Union High School District in Avondale, AZ, all ninth-graders were pretested using the district's criterion-referenced test for Algebra 1 at the start of the 2008 spring semester. The initial CRT scores showed an average mastery of 30 percent; that number rose to 47 percent following the use of ALEKS.

Impressive results like these are fueling further experiments with AI tools. Last June, the US Department of Education awarded a $2.8 million grant to WestEd, a nonprofit research and development agency, and Quantum Simulations, a maker of AI-based tutoring, assessment, and professional development software, to evaluate the impact of Quantum's chemistry tutoring software on student learning. About 70 schools in California will participate in the study, whose results will be released in 2012. Eventually, the study will be expanded nationwide.

The WestEd study hypothesizes that students who use the Quantum software alongside their classroom studies will understand concepts better and improve faster than students who learn the same curriculum without supplementing with the technology. "Quantum's software models the thinking or conceptual development that the students should have," says Steve Schneider, director of WestEd's Mathematics, Science, and Technology Program. "For example, the students enter the equation they want to balance, the tutor asks a series of questions in text, the students provide answers, they get prompts, and then the software corrects their work and gives them feedback on what they should do."

So far, the response has been encouraging. "We're cautiously optimistic about test scores and also retention," Schneider says. "In the first six weeks of chemistry, a lot of kids usually drop the course, but that hasn't been the case with the test group."

Schneider believes, overall, the experiment is a "very interesting way to use technology. I think the AI movement is seeing a quantum leap from what we've seen in the past 20 years. AI can't replace human teachers, but if it is done well, it has a role in the classroom. Educators can use all the help they can get."

Perhaps the biggest benefit of AI-based software, in Schneider's view, is that it is readily available and accessible in the classroom. "Teachers like using technology if it can do the job quickly and efficiently. They don't like using technology if it means having to drag the kids to a computer lab."

AI's ability to diagnose and solve individual learning problems is likely to earn it a lasting place in K-12 education. Ironically, Spielberg's movie may be the best commentary on its limitations-- and its vast potential. John McCarthy, professor emeritus of computer science at Stanford University and an AI pioneer credited with coining the term artificial intelligence more than 50 years ago, notes this in his blog: "The movie A.I. illustrates one disadvantage of having robots with emotions or which elicit human emotions. Unless you make them really human, they will not fit into human society. Better just make them suitable as a kind of tool."

::WEBEXTRAS ::
If you would like more information on artificial intelligence, visit our website at www.thejournal.com. In the Browse by Topic menu, click on artificial intelligence.

Rama Ramaswami is a freelance writer based in Wilton, CT.