KARMA: The Knowledge Acquisition Reference Multimedia Aid

• 02/01/96

According to The Chronicle of Higher Education,[1] only about 4% of those surveyed use multimedia and CD-ROM materials in the classroom. This is a surprisingly low percentage, especially with the rapidly growing interest in multimedia use in education and business.

Part of the reason for this lack of use is an unwillingness of some faculty to inject this new technology into classroom or supplemental material outside of class. Some faculty suggest that multimedia is all hype and no content.

Granted that multimedia is now being touted as the current fad and multimedia educational material has a long way to go, a number of successful projects incorporating learning theories into educational multimedia software are being developed and used, such as those at the Institute for Learning Sciences at Northwestern University.[2] Even without explicit incorporation of these learning theories, multimedia could be very stimulating for students and faculty.

The College of Integrated Science and Technology (CISAT) at James Madison University is developing multimedia software, as part of its charter, for helping students learn various strategic sectors and underlying concepts. One of the strategic sciences emphasized at CISAT is "knowledge management," of which expert and knowledge-based systems play a critical role.

The first multimedia software developed for CISAT and for its knowledge management discipline was a CD-ROM called "Developing Your First Expert System." It is now being used in over 25 countries, distributed by the non-profit organization, International Society for Intelligent Systems (ISIS).[3] This CD has been used at James Madison University, George Washington University, the US Army War College and over 60 institutions and organizations. The thrust of this multimedia aid is to step the student through the knowledge engineering life cycle (i.e., the expert system development process).

Due to the success of this CD project, another multimedia program was constructed in the knowledge management area to help students learn knowledge acquisition techniques. A previous hypermedia version (mainly hypertext) of this program, called KARTT (Knowledge Acquisition Research and Teaching Tool), was developed by Liebowitz and Bland.[4] KARTT lacked some pleasing user interface designs, so KARMA was constructed to address some of these deficiencies.

KARMA has been developed using Authorware Professional and includes "lessons," "models," "knowledge acquisition sessions" and "exercises" on knowledge acquisition. The first version has just been completed (December 1995) after four months of development. It will be used in the knowledge acquisition component of the Analytical Methods course at James Madison University and the Military Applications of Artificial Intelligence course at the US Army War College in the spring 1996 semester.

Knowledge Acquisition

Knowledge acquisition is the process of acquiring knowledge from an expert or multiple experts for developing the knowledge base of the expert system. Knowledge acquisition is still the biggest bottleneck in expert systems development due to a number of reasons.

First, it is fairly labor- and time-intensive to interview the expert(s) and handcraft this acquired knowledge into a knowledge base.

Second, many knowledge engineers may not be well-skilled and experienced in interviewing or using other knowledge acquisition techniques (like learning by analogy, observation, protocol analysis, etc.), thereby making the knowledge acquisition (and knowledge representation) process an arduous task.

Third, the "knowledge engineering" paradox says that the more expert an individual, the more compiled the knowledge and the harder it is to extract. Thus, the knowledge acquisition process is a difficult one, especially exposing the rules of thumb (i.e., heuristics) that the expert has compiled over many years of experience.

The process of knowledge acquisition transcends the "mere" expert system project development. Knowledge acquisition is a methodology rooted in cognitive psychology, individual and group dynamics, and other interdisciplinary disciplines. By being a "good interviewer," this should enable someone to not only be useful as a knowledge engineer, but also helpful in other professions (like news reporting, medical diagnosis, etc.) as well.

Surprisingly, there are very few knowledge acquisition courses offered at universities and industry.[5] In order to lessen the difficulty of performing knowledge acquisition and to better educate students on this subject, multimedia aids and automated knowledge acquisition tools are needed to support the knowledge engineer.

Towards this goal, KARMA has been developed as a multimedia aid for helping the student learn about knowledge acquisition. (Note: it is not a tool for automating the knowledge acquisition process.)

Developing KARMA

The development of KARMA was principally a two-member team effort. Christine Letsky, the lead multimedia specialist in the Multimedia Lab at CISAT/JMU, served as the multimedia specialist for KARMA. The content specialist was professor Jay Liebowitz, who has taught courses on knowledge acquisition and has written numerous papers on this subject and expert systems.[6] This team also worked on the first CD-ROM project, "Developing Your First Expert System," and had a good working relationship.

Authorware Professional was selected as the multimedia authoring language for KARMA. By having an easy-to-use, yet powerful scripting capability, Authorware Professional provided important features for user interaction with KARMA. The ability to include hyperlinks, nice graphical user interface designs, ability to allow for interactive questioning in exercises, a glossary feature, and effective video/audio capabilities were some of the essential features we wanted in KARMA.

Because the content specialist was located several hours away from JMU, a good amount of the necessary "knowledge" to be included in the first version of KARMA was provided to the multimedia specialist during their first meeting. Subsequently, the two would meet at JMU once a month, with e-mail and telephone messages serving in the interim. Letsky, the multimedia specialist, devoted a day a week to the encoding of this knowledge, using Authorware Professional. She would have liked to have spent more, but her other duties and projects monopolized the rest of her time. Some of the lab assistants in the Multimedia Lab also provided assistance on the KARMA project where possible.

An important part of the development effort was determining the framework for which the content would be based and presented.

KARMA currently has five sections: (1) Introduction to Knowledge Acquisition, (2) Common Problems in Knowledge Acquisition, (3) Interviewing Methods and Helpful Tips, (4) Other Knowledge Acquisition Techniques, and (5) Models and Examples.

Exercises, in the form of multiple-choice questions, and glossary terms accompany the first four sections. Navigation devices, various sounds, graphics and user-sensitive messages are encoded as part of KARMA.

The fifth section on "Models and Examples" currently provides audio dialogues with a knowledge engineer using different knowledge acquisition techniques with a domain expert during knowledge acquisition sessions. The next version of KARMA will have these scripts in video.

Lessons Learned & Future Work

Several lessons were learned from this first version effort of KARMA. These include:

• Multimedia development is not a trivial task; to do it well requires many talents from the team (content knowledge, multimedia authoring knowledge, video and audio production capabilities, graphics and user interface design skills, etc.).
• One person normally d'esn't have all the necessary skills to develop the multimedia program &emdash; the synergistic relationship of the faculty member (as the content specialist) working with the multimedia specialist is a fruitful and risk-reducing approach for building a successful multimedia application.
• Good project management techniques (i.e., deliverables, milestones, schedules, meetings, etc.) must be done in order for the project to stay on schedule.
• User feedback, testing, and evaluation are extremely helpful for refining the multimedia program.
• An infrastructure, such as a Center for Multimedia Development, is greatly needed to assist faculty in developing joint multimedia programs and for climbing the learning curve on multimedia development.

Future work on KARMA includes having the students at James Madison University, George Washington University and the US Army War College use KARMA for day-to-day tasks and projects. Then we plan to incorporate the results of their comments into the next version. Additionally, more content will be included in the five major sections. And finally video segments of knowledge acquisition sessions will also be incorporated into KARMA.

Overall, we feel that there has been little work, if any (besides our other CD), in developing and distributing multimedia software to help students learn about the expert systems development process. We hope that these projects will stimulate others to build and apply multimedia for in- and out-of-class usage in the expert systems education field.

Jay Liebowitz is the Chair in Artificial Intelligence at the US Army War College/Center for Strategic Leadership. He is also a professor of Management Science at George Washington University. He is the Editor-in-Chief of the international journal, Expert Systems With Applications, founder of the World Congress on Expert Systems, and has published about 200 journal articles and 18 books.
E-mail: [email protected]

Christine Letsky is Director of the Multimedia Laboratory in the College of Integrated Science and Technology at James Madison University. She has developed a number of multimedia programs, including the "Developing Your First Expert System" CD-ROM mentioned in this article. Her background is in art and multimedia programming.
E-mail: [email protected]

References:
1. "Using Technology in the Classroom," (1994), The Chronicle of Higher Education, November 19.
2. Schank, R. (1995), "Active Learning: Integrating Multimedia and Artificial Intelligence," Keynote Address at the International Symposium on Artificial Intelligence, Monterrey, Mexico, October 18.
3. Liebowitz, J. & Letsky, C. (1995), "Developing Your First Expert System" CD ROM. Distributed by the International Society for Intelligent Systems, PO Box 1656, Rockville, MD 20849, (301) 770-2978.
4. Liebowitz, J. & Bland, K. (1994), "Using Multimedia to Help Students Learn Knowledge Acquisition," Multimedia Computing in the 21st Century (S. Reisman, Ed.), Idea Group Publishing, Harrisburg, PA.
5. Liebowitz, J. (1993), "The Need for Knowledge Acquisition Courses," Journal of Computer Information Systems, Association for Computer Information Systems, Oklahoma State University, OK.
6. Lee, J.K., Liebowitz, J. & Chae, Y.M. (Eds.) (February 1996), Proceedings of the 3rd World Congress on Expert Systems, Cognizant Communication Corp., Elmsford, NY.