The Coming of the Intelligence Age: Enhancing Education Through Assessment
I chuckle inside when I hear discussions about what the next 'age' in history will be. I'll admit, my ideas on what this means for education may be perceived as fanciful as well. But, after nearly three decades of technology experience and research work in artificial intelligence and cognitive science, I hope I've learned enough to render a reasonable opinion.
Thus far, we've seen the Industrial Age and the Information Age; now, I believe that we are heading toward the Intelligence Age. I'm not talking about robots running our schools; rather, systems that can intelligently gather and analyze data. In education, there is an interesting convergence developing between assessing and evaluating student capabilities and progress, legislation, technology, and communications. This convergence will naturally lead to an increased use of intelligence systems to collect, manage and report data.
There is no question that to succeed, K-12 education must remain hands-on with live classroom interaction between teacher and student, students and their peers, as well as parents and the community. I also know that intelligence systems can enhance education tremendously, and I believe assessment is the vehicle for achieving this.
Embracing Technology
I've marveled at the fact that education does not appear to have embraced technology to the same degree that other industries have. In banking, for instance, complicated models determine whether or not a loan applicant should receive funds based on historical and personal characteristics. Also, in an effort to prevent fraud and abuse, credit card companies now alert you when your purchasing patterns have fallen out of the norm. In addition, where 20 years ago computer-related jobs were a small minority, now studies report that somewhere between 60% and 80% of those actively employed use technology in the workplace (Borghans 2001; Handel 2003).
Yes, there are student information systems, library systems, the Internet, skills-based software, online classrooms and e-mail; however, these systems are designed to operate on the periphery of the education process, behind the scenes, or to serve isolated duty. Wouldn't it be nice to be able to collect test results, cognitive traits, comprehension and rich skill analysis data so that teachers, and even parents, could understand the strengths and weaknesses of students on an individual basis. This concept would be unthinkable without technology.
The good news is that there are interesting assessment technologies — either just around the corner or available right now — to detect students' abilities, gather and analyze the data, and communicate the results effectively to both teachers and parents. Assessment systems, whether online, local network or PC-based, can yield real benefits beyond 'training for the test.' One would certainly expect these systems to determine whether students can demonstrate proficiency in the specific skills tested, but the data from periodic testing can also be used to adjust curriculum in real time.
For instance, using an assessment software system, students are given a short unit quiz that tests a few skills. If the class as a whole has difficulty with one of the skills, then the assessment results will inform teachers and administrators about areas that deserve more attention. The curriculum is then adjusted, more time is spent on an important skill, and instruction g'es on.
Intelligent Systems
So, where does intelligence come into play? I suggest that these are potentially intelligent systems because they can perform logical and analytical tasks faster than we can, and with fewer mistakes. Scoring tests is an easy and very useful application of this technology. Educators might say that there is no replacement for paper-based tests wherein students show their work; I agree.
For this reason, I still believe that paper-based tests have a place interspersed throughout periodic computer-based assessment. In these tests, students can show their work, which may be analyzed by hand to detect, for example, errors in the intermediate steps to solving a math problem. This is a very valuable tool. Having said this, there are companies bringing systems to market that fairly accurately score essays by computer — the intelligence is building.
Yet, these intelligent systems can be set up to do much more. In research completed in 2001, I was able to demonstrate that pretty simple interactions between students and computers could capture impressive amounts of cognitive data about a student's capabilities. In our study, we collected behavior data that could be used to understand students' cognitive or multimodal skills beyond mastery of the standards-based skills that were being tested. We detected metacognitive traits — an important cognitive skill — and whether students responded differently to images or sounds (if they were available) than to text or complex sentences. In addition, we timed almost every interaction the student had with the computer to ask questions like: 'Did they spend more time on word problems?' or 'Are there specific words that they had problems comprehending?'
Because of the study's outcomes, we attempt to capture this rich data at every turn in our Assessa line of products — from our Web-based ASP model system to our free, customizable, public source assessment system that can be used on school intranets. Even our Palm and Pocket PC/.NET handheld systems, which beam small quizzes or group participation games back-and-forth between the handheld and a teacher's computer, have this capability.
The Power of Data Collection
Once the student data is collected, it can be used in connection with other data. An intelligent system can actually gather and analyze this data to predict individual and group performance on future standards-based tests. As the data is added to a central data store, educators can track student progress over time. So, instead of one teacher passing on handwritten cards or notes about students to the next grade teacher at the end of the year, these systems enable teachers to look at a student's progress historically, taking advantage of comprehensive characteristics and details that will enable individualized instruction going forward. What amazes me about the ability to collect this data is that the assessment vehicle is noninvasive.
In addition, it's important to remember that personal interaction is so vital to a student's behavioral and cognitive development. I've embraced the intrapersonal aspects of the classroom, as I have never been an advocate of students sitting in front of PCs for hours on end. Instead, short, periodic computer-based assessments, and occasional Web research or remedial skill building, is fine. Our systems, and a number of others, can be customized to deliver 15-20 minute quizzes or full-blown end-of-semester two-hour tests. Short quizzes or other assessments may be given every few weeks, once a month, quarterly or a few times a year. In relation to the aggregate of classroom time, systems that can collect this rich data so quickly are very efficient and will become invaluable.
Conclusion
A befuddling artifact that continues to come up in presentations, seminars and in my meetings with school officials, is the notion that there is no off-the-shelf technology to meet the needs of their specific district or state. In many cases, the decision is made that since the institution has so many dissimilar systems, there is neither an efficient nor effective way to capture and report on the kinds of data that are required to comply with state and federal needs, as well as the needs of teachers and parents.
Often, in-house technology gurus will craft their own solutions by creating one-off applications, which I believe can be costly in time and resources, closing systems off from potential expansion or evolution. In this regard, I am hopeful that as intelligent systems move forward, standards, common solutions and approaches will evolve, and systems and services offered by researchers, vendors and service providers will be embraced. If we are to enter and embrace the Intelligence Age, then assessment and analysis tools may be the place for educators to start.
References
Borghans, L. 2001 'How Computerization Changes the U.K. Labour Market: The Facts Viewed From a New Perspective.' SKOPE Research Paper No.13. Spring.
Handel, M. 2003. 'Complex Picture of Information Technology and Employment Emerges.' Issue Brief from SRI International Technical Report. July.
Contact Information
EyeCues Education Systems Inc.
(619) 299-2255
www.eyecues.com/assessa
This article originally appeared in the 04/01/2004 issue of THE Journal.