Artificial Intelligence: Is the Future Now For A.I.?
The feds' reliance on standardized testing excludes the use
of artificial intelligence, so school districts are forging ahead
with the technology to power their own formative assessments.
IN THE STEVEN SPIELBERG MOVIE A.I. ARTIFICIAL INTELLIGENCE, the hero, an android,
wants to become human because he believes it is a better state than his own. But
sadly, he discovers there are some things that artificial intelligence just can't do. AI's
condition is much the same in real life. The computer science visionaries of the 1950s
and '60s predicted with almost total certainty that machines would soon be able to understand and
think like humans. But the evolution of AI has been far more prosaic-- rather than priming a
generation of robots to take over the world, it has become a pervasive tool that drives much of the
technology behind finance, manufacturing, medicine, and a host of other industries.
In education, however, AI hasn't made much headway. In the one area where it would seem poised
to lend the most benefit-- assessment-- the reliance on standardized tests, intensified by the
demands of the No Child Left Behind Act of 2001, which holds schools accountable for whether
students pass statewide exams, precludes its use. Standardized tests are graded consistently,
with no allowance for individual student abilities or styles of learning, and no place for artificial
intelligence's ability to decipher and shore up weaknesses.
This climate of broad-based accountability will make it difficult for AI programs to achieve acceptance
at the federal level, says Kathy Mickey, senior analyst and managing editor of media research
firm Simba Information's Education Group. Mickey has covered the educational testing market for
more than 10 years, and most recently she edited Simba's report, "Academic Testing: PreK-12
Materials Market Analysis and Forecast 2008." She says that there's virtually no intelligent software
being used in large summative testing programs.
"A number of products are available that feature AI, but
they're highly personalized, very individualized, and focus on
intervention and individual education. States have tried to use
them, but that isn't what the federal government is looking for."
Critics insist that despite government resistance, one-size-fits all
testing will soon be gone. For example, one theory held by
some educators uses the concept of disruptive innovation to
explain why standardized tests are doomed. The term was coined
by Harvard Business School professor Clayton Christensen to
describe a new technology, process, or business model that
replaces, or disrupts, an earlier one by being much more affordable
or simpler to use. In his latest book, Disrupting Class: How
Disruptive Innovation Will Change the Way the World Learns (McGraw-Hill, 2008), Christensen and his colleagues, Michael
Horn and Curtis Johnson, assert that the US public school
classroom model will be "disrupted" by technological innovations
based on personalized instruction; as this occurs, student
assessment will take place entirely in real time and be tailored
to individual needs, phasing out traditional classroom testing.
"AI can't replace human teachers, but if
it is done well, it has a role in the classroom.
Educators can use all the help they can get."
That breakthrough may be a long time coming. "There's a
difference between having a conversation about it and actually
installing a testing program that meets the accountability standards
of a federal law," Mickey points out. "Your framework is
narrowed. There's been so much talk about individualized
programs. The technology is there and available, and it has a lot
of potential. But I doubt it will be used at the summative level."
Schools, however, aren't waiting on the feds to change
course, as they continue to conduct their own individualized
assessments. In fact, in 2008, Simba projects a 10 percent leap
in the purchase of formative tests, whereby teachers monitor
students' progress on a day-to-day basis. That surge, along with
the expansion of online testing and increased testing in high
schools and for English language learners and special education
students, Simba says will propel a K-12 testing market increase
from $2.06 billion in 2008 to $2.52 billion by 2011.
Hints of Disruption
Encouraged by the market growth, software builders are making
notable enhancements to their AI technology-- to the point
where it may be verging on disruption. One vendor sparking
buzz is Lexia Learning Systems. While Lexia's AI-based reading
programs have been around for years, its latest version of Lexia
Reading, launched in September 2007, is more sophisticated
than its predecessors. Previous formative testing applications
scored student tests in full; Lexia is able to stop on each question
to identify problem areas. It harnesses the "intelligent"
component of AI by continuously assessing a student's skill
level and understanding as the student works on an activity. If
the student has not understood the concept, the software identifies
the area of weakness and offers additional practice before
moving on. During each session, the program generates reports
for teachers and administrators about the student's skill level
and areas in which he or she needs more instruction.
Based on cognitive psychology theory that increasing phonological
awareness-- the ability to identify the sound structure of
spoken language-- is a critical component of improving literacy,
Reading offers individualized instruction and practice on the
phonics and word-recognition skills critical for reading fluency
and comprehension. For Hall County Schools in Gainesville,
GA, with an enrollment of about 25,000 students, it's the "intrusive
nature of the program that's so useful," says Aaron Turpin,
Hall County's executive director of information and technology.
"Every single response accelerates the student to a higher level
of questioning or education, or suggests a remedial path. The
difference from other software is that Lexia responds to every
single question and makes adjustments. It then takes the information
and provides it in a clear format for the teachers to use so
they can learn from it as well and modify their instruction." The software has benefited both English and non-English speakers,
Turpin adds. "We've seen equal change in both."
What Is Artificial Intelligence?
THE TECHNICAL DEFINITION of artificial intelligence is the science
of designing machines that can simulate human intelligence by
showing conversational capability similar to that of humans (and, to
some extent, the ability to "understand" human thinking). The best known
gauge of whether a machine is intelligent is the Turing
test, proposed by the scientist Alan Turing in 1950.
The test measures whether a human judge can tell
the difference between a machine and a human
while engaging in a natural-language conversation
with both in a situation where all participants are
isolated from one another. Today, however, the term
artificial intelligence is used to describe anything
from pattern and voice recognition to genetic programming.
In addition to its grounding in computer science, AI
borrows heavily from disciplines such as cognitive psychology,
mathematics, semantics, linguistics, and philosophy.
Another pioneering approach to AI-based testing comes from
Pearson Education, whose Knowledge Technologies group
has developed WriteToLearn (WTL), a program that enables
students to practice essay writing and summarization skills and
have their work measured by Pearson's proprietary Knowledge
Analysis Technologies engine. KAT analyzes whole passages of
text not just for grammar or tone, but for meaning. The WTL
program is based on latent semantic analysis (LSA), a language
learning theory and computer model developed at Bell Labs and
turned into an educational technology component by the
University of Colorado and Pearson's Knowledge Technologies
group. LSA is a statistical technique for extracting and representing
the similarity among meanings of words and passages by
analysis of large bodies of text.
Karen Lochbaum, one of the scientists who developed LSA
and now vice president of software engineering at Pearson,
believes that WTL is cutting-edge in its judging of writing and
reading skills. "What we do for essay scoring is train the KAT
engine on papers that are already written and have been scored
by human scorers," she says. We analyze the papers in lots of
different ways to determine how they got the score they did."
Lochbaum explains that the program evaluates a new essay by
using old papers on the same topic as yardsticks. "WTL is
unique because it looks at the content and development of what
the students are writing. Are they addressing the prompt? Are
they stating the main idea and building it up with supplementary
ideas? For reading assessments, students read an article and
write a summary of what they've read. The program gives
feedback on their comprehension of what they've read, and they
can revise their work based on that feedback. WTL offers the
practice in writing that teachers would like to give with more
frequency. It really is about the practice."
WTL has been used successfully at Minnesota's Dunwoody
Academy High School, a charter school that provides academic
and career-focused training for students from depressed areas
of Minneapolis and St. Paul. The impact of the school's adoption
of WTL in January 2008 was felt on the state assessment last
spring. Seventy-three percent of its ninth-graders-- compared
to the state average of 67 percent-- met the requirements of the
writing portion of the Minnesota Comprehensive Assessment-II
Graduation Required Assessments for Diploma (MCA-II/
GRAD) test, says Duane Dutrieuille, dean of academic and student
affairs. Plus, he says, "A very high percentage of our 10thgraders
passed the test, with a score above those of the state and
urban schools in Minnesota."
While Dutrieuille credits the school's language arts teacher,
Beverly Davis, for providing the guidance that made the program
so successful, he also considers WTL very beneficial. He
cites its outstanding capacity to improve reading and writing
abilities. "The key component is that it breaks down all the
different categories of writing," Dutrieuille says. "Maybe the
students need a little bit more help with their sentence structure.
Or maybe they have great thoughts, but have a hard time with grammar or the flow and structure.
If you're a writer, the program
gives you confidence. And then
you can build on it."
What we are seeing in these new software programs is the marked
advancement AI has brought to formative testing technology,
which now is capable of tutoring students in the fundamental
understanding of concepts-- building knowledge in addition to
assessing it. Another example is ALEKS, a web-based assessment
and learning system from the company of the same name.
Developed by scientists at the University of California-Irvine,
ALEKS (Assessment and Learning in Knowledge Spaces) asks
students personalized assessment questions; the choice of each
new question is based on responses to all previous questions.
When the assessment is complete, ALEKS delivers a report with
a detailed graphic representation of the student's knowledge. But
the system can do more than merely reveal what a student does
and doesn't know about a specific topic. It also can help a
student learn. For example, if a student has answered a math
practice question without adding the appropriate numerical
units, ALEKS may suggest that the student check the units. The
program will offer more practice questions until it determines
that the student has mastered the concept.
Schools using ALEKS report significant improvement in
students' grasp of concepts. At Monte Vista School District in
Colorado, 15 students in grades 2 to 5 who had failed both math
and English were placed in a remediation class that uses
ALEKS. When students entered the program, their average mastery
was 53 concepts. Their goal was to master 50 new math
concepts in six weeks. By the end of six weeks, several of the
students had met that goal; some had mastered more than 100
new topics. And at Westview High School, part of the Tolleson
Union High School District in Avondale, AZ, all ninth-graders
were pretested using the district's criterion-referenced test for
Algebra 1 at the start of the 2008 spring semester. The initial
CRT scores showed an average mastery of 30 percent; that
number rose to 47 percent following the use of ALEKS.
Impressive results like these are fueling further experiments
with AI tools. Last June, the US Department of Education
awarded a $2.8 million grant to WestEd, a nonprofit research
and development agency, and Quantum Simulations, a maker of
AI-based tutoring, assessment, and professional development
software, to evaluate the impact of Quantum's chemistry tutoring
software on student learning. About 70 schools in California
will participate in the study, whose results will be released in
2012. Eventually, the study will be expanded nationwide.
What's Next? An A.I. Pioneer Looks Ahead
RAY KURZWEIL IS A LEGEND in futurist circles. An award-winning
inventor and computing prophet, he's an artificial intelligence guru
whose theories about the potential of machine intelligence have generated
much debate over the years.We caught up with him to get
some blue-sky thoughts on the technology's future.
T.H.E.: Early forecasts for AI were wildly optimistic. Why hasn't the
field evolved as predicted?
RK: There's a lot more artificial intelligence out there than people realize.
It's like when people go into the rain forest and ask, "Where are all the
species?" The answer is that they're hidden, you can't see them. The
same is true of AI. Every time you use cell phones, search engines, and
so on, you're using AI. It has profound implications for humans. There are
hundreds of examples of AI all around us. There's a product for dyslexic
kids. It's a 4-ounce cell phone that reads out loud in 16 different languages
and displays on screen what it's reading. The kids are reading for
a compensatory purpose, but they're also building reading skills.
How can AI be useful in K-12 assessment?
It is getting more sophisticated. Earlier tests were just rote programs,
but now there are tests that create an effective model of what the
student understands and where the breakdown in understanding is. It's
basically what a good teacher does. There are now adaptive tests where
the test will quickly get more difficult as needed, as opposed to 50
questions that are too easy and 50 that are too difficult. You can
assess where the student is more quickly. There is a whole field of
adaptive testing, and when that works well, that's the best use of AI.
Where do you see AI going a few years from now?
I think where we're going is first understanding human
language-- to actually model semantics, not mimic
syntax. That's a necessary condition for AI systems
to work. Were kind of at that threshold now. In five
years we will have a system of instruction that's not at
a human level, but will understand what the student is
saying and personalize the instruction for that student. But
computers, even if they're not our bodies and brains, are
already very close to us.We should use technology to make
ourselves smarter and extend our horizons. The role of the
teacher is to mentor and guide students to use increasingly
sophisticated tools. The ongoing role of humans
will be to provide moral, social, and emotional
intelligence. By 2029, computers will be
indistinguishable from human intelligence.
The WestEd study hypothesizes that students who use the
Quantum software alongside their classroom studies will understand
concepts better and improve faster than students who learn
the same curriculum without supplementing with the technology.
"Quantum's software models the thinking or conceptual development
that the students should have," says Steve Schneider,
director of WestEd's Mathematics, Science, and Technology
Program. "For example, the students
enter the equation they want
to balance, the tutor asks a series of
questions in text, the students provide
answers, they get prompts, and then the software corrects
their work and gives them feedback on what they should do."
So far, the response has been encouraging. "We're cautiously
optimistic about test scores and also retention," Schneider says.
"In the first six weeks of chemistry, a lot of kids usually drop the
course, but that hasn't been the case with the test group."
Schneider believes, overall, the experiment is a "very interesting
way to use technology. I think the AI movement is seeing
a quantum leap from what we've seen in the past 20 years. AI
can't replace human teachers, but if it is done well, it has a role
in the classroom. Educators can use all the help they can get."
Perhaps the biggest benefit of AI-based software, in Schneider's
view, is that it is readily available and accessible in the
classroom. "Teachers like using technology if it can do the job
quickly and efficiently. They don't like using technology if it
means having to drag the kids to a computer lab."
AI's ability to diagnose and solve individual learning problems
is likely to earn it a lasting place in K-12 education. Ironically,
Spielberg's movie may be the best commentary on its
limitations-- and its vast potential. John McCarthy, professor
emeritus of computer science at Stanford University and an AI
pioneer credited with coining the term artificial intelligence
more than 50 years ago, notes this in his blog: "The movie A.I.
illustrates one disadvantage of having robots with emotions or
which elicit human emotions. Unless you make them really
human, they will not fit into human society. Better just make
them suitable as a kind of tool."
If you would like more information on artificial intelligence, visit
our website at www.thejournal.com. In the
Browse by Topic menu, click on artificial intelligence.
Rama Ramaswami is a freelance writer based in Wilton, CT.
This article originally appeared in the 02/01/2009 issue of THE Journal.