Teaching Executives in China
Since 1991 we have been using a distance learning technology to teach MBA students in China. Our university contracted with two firms in China to deliver MBA programs to selected executives. Each student would take 42 semester hours of instruction (17 separate classes), following exactly the same sequence of courses as those students enrolled in our regular on-campus MBA program.
Our objectives for engaging in these distance learning programs were:To enhance the internationalism of our faculty.To increase the facultys experience with executive education.To enhance our understanding of a distance learning technology.To improve the overall quality of our graduate students.
Prior to these programs few of our faculty had ever been to Asia and none of the faculty had prior experience with distance learning. After a review of distance learning technologies (e.g., correspondence education, live TV, etc.) we chose Tutored-Video Instruction (TVI), a technology developed by Stanford Universitys School of Engineering in the early 1970s for the purpose of delivering graduate engineering education to Hewlett-Packard employees 100 miles distant from Stanfords campus. Stanford relied upon research of the various distance learning technologies that suggested that the use of television as a means of delivering instruction to students, regardless of grade levels or subject matter, resulted in student performance that was equal to that of live classes.
In the TVI format, live lectures are videotaped and viewed later at remote locations by groups of students where discussions, directed by live tutors, are then fed back to the instructor. A local tutor is trained to direct the students learning at the remote location and serves as the faculty members local representative, distributing materials and collecting assignments. The tutor is also charged with providing answers to student questions. To do so requires that the tutor stop the videotape when questions arise, direct a discussion towards providing answers by other students and, when necessary, forwarding unanswered questions directly to the faculty. Thus, the tutor provided the instructor with a direct link to the remote location experiences. Together the tutor and the students were expected to manage the learning experience by actively engaging the information presented to them on the videotapes.
Research on Stanfords success with this program concluded that:the grades of the TVI students were (statistically) higher than those of the on-campus students whose class had been taped for subsequent viewing off-campus;the grades of the on-campus students were (statistically) higher than those students who were learning the same material by watching television with no opportunity for interacting with other students or the instructor;tutors who answered questions directly were less effective than those tutors who drew students into discussion when the tape was stopped; andin the most effective classes, as measured by their average grade, the students stopped the videotape to discuss a point or raise a question approximately every 5 minutes.
A follow up study based on a cross-sectional analysis of TVI distance engineering education at eight U.S. universities strongly supported the results of the pioneering Stanford study. Student performance in TVI classes was (statistically) higher than that of students enrolled in the same classes on campus regardless of whether the instruction was within a degree program or merely non-degree, technical training. Further, the older the students the higher their performance was in TVI classes, in direct contrast to on-campus classes where age and performance are not related. Research comparing the use of TVI to other distance learning technologies in delivering engineering courses in Norway was also in agreement with the Stanford results.
On the issue of cost-effectiveness, TVI has been found to cost less to deliver than live instruction. Stanford estimated that their initial experiment cost approximately 9.5% less than the cost of delivering similar coursework to live students. Research on the use of TVI to deliver distance learning across a multi-campus system suggests that it costs from 10-15% less to deliver.
A Mid-Stream Evaluation
Our faculty felt that the basic TVI model needed to be augmented with live instruction. We videotaped live classes for viewing by our Chinese students. Our faculty traveled to China to hold from 9-11 hours of live discussions with the students either at the beginning or towards the end of each academic term. For accreditation purposes our faculty agreed to use a common final examination to evaluate the TVI students directly against the on-campus students whose classes served as both videotape subjects and as a control groups. Students were recruited from a list submitted by their company and were personally interviewed by a faculty member prior to their acceptance. They were then given English language and math exams to test their basic skills. The students entering these programs were in their mid-30s and each was employed full-time in an executive position at their company.
The students attended classes (watching and discussing videos) one night during the week and on Saturday. In the early days of these programs the videotaped classes were held in a small, windowless room located in the Universitys instructional media complex. Live students shared a small video monitor where they saw what the instructor was writing on a pad (and taped via an overhead projector). Microphones were placed between each set of on-site students so that both the students questions and the instructors responses were captured on the videotape. Each faculty was required to submit discussion questions for their tutors and to require formal weekly feedback from the tutor via fax. Further, a formal faculty feedback mechanism was established in the form of a videotaped interview with each of the faculty who taught in the program (thereby permitting faculty who had not taught to get on the learning curve quickly).
An interim analysis of the student evaluations of the TVI courses in the first MBA program and a survey of the faculty teaching in the program identified significant technical problems (e.g., poor quality video, inability to hear student questions or comments, etc.). Chinese students clearly preferred more live-time with the faculty. Therefore, for the second China program the amount of live time was increased to 12-15 hours, and new overhead cameras, additional microphones, and more powerful multidirectional microphones were installed. Faculty was encouraged to post their e-mail addresses on their TVI course syllabi to facilitate greater interaction between themselves and their TVI students.
Despite these changes, further faculty responses were discouraging. They felt that the TVI technology and the enhanced e-mail-based feedback loop was not working, and desired even more live time with their Chinese students. We also discovered that most of the faculty had no common means of assessing the performance of the Chinese students directly against that of their on-campus students and that the faculty were not using the tutor for feedback purposes or providing adequate directions for the tutor. Finally, despite these problem areas the faculty felt that overall the TVI experience was a rewarding one for themselves and their Chinese students.
Unfortunately, no formal evaluation of the student perceptions of the program or their performance was undertaken during this interim evaluation. However, Chinese students course evaluations continued to note both continued technical difficulties and renewed requests for more live time in place of TVI.
A Comprehensive Evaluation
In 1997 a formal survey of current and past Chinese students was undertaken, together with an analysis of their performance in the TVI classes and their student evaluations of those classes. Approximately 40% of the students that completed the TVI programs responded to a program performance survey and an attitudinal survey. We continued to collect student performance data and the course evaluation data and updated them on an ongoing basis to see if there were any changes in either of these measures.
Student Performance: TVI students received slightly lower grades than did the average on-campus MBA student. Comparing each of the TVI student classes directly against the on-campus class that was taped, the TVI students failed to perform as well. This was contrary to the findings of earlier research on the TVI technology.
Student Course Evaluations: At the completion of each course, the TVI students were asked to fill out a specially developed course evaluation form. The form focused on three specific areas that dealt with course content, technology (tapes and taping process), and on-location teaching by the faculty. We found out that the results vary slightly among different teaching areas. We attribute this finding primarily to the type of course and teaching methodology used in that course (e.g., management courses were taught using high student in-class participation vs. accounting courses primarily being lecture based). The student course evaluations were slightly higher in primarily lecture-oriented courses.
Attitudinal Responses: In order to determine the perceptions of the off-site students on our programs and to see if these perceptions had changed over time (from one program to the other), we conducted an attitudinal survey of the graduates of three China cohorts that had completed their MBA studies. The questions focused on four major areas:whether the students developed the desired managerial skills;whether the tutored-video format was effective;whether the institutional support (Chinese companies) for the program was adequate; andwhether the program was valuable (impact on career progress, benefit to the organization, and recommend the program to colleagues, respectively).
Results in all cases showed high levels of satisfaction and general improvement in the degree of satisfaction over time.
The results of our evaluation of these programs conclude that our Chinese (off-site) students are very happy with the TVI technology and proud of the education that they received. Our faculty also highly valued their experiences with this technology, as they discovered that they had to be better prepared to teach a course that was going to be taped and be shown to students in China. Further, they highly valued the opportunity to gain firsthand experience in a foreign country (through on-site visits). Although our TVI off-site students performances did not exceed those of our live students, we feel that this is due to the fact that we did not devote sufficient attention to the role that the tutor played in each of our classes.
However, it also important that we learned that the TVI technology is a low-investment, medium-maintenance, high-leverage means of delivering distance learning. It d'es not necessarily require high-grade video productions. We also found that one of our existing classrooms could easily be modified for videotaping, and that an investment of less than $50,000 resulted in a sufficient number of cameras and microphones. A trained student assistant was hired and was utilized for videotape production and editing and duplication. Since the success of this technology rests primarily with the tutor, and the faculty use thereof, program directors at both ends are very vital. At the production end, the program director must make certain that the faculty isusing the technology correctly,using the tutor effectively,employing the same evaluation methods as in his/her live class, andsupplying the necessary handout materials so the students at the viewing end are able to participate in a learning environment structured similar to a "live" class.
At the viewing end, the program director must make certain thatthe students view the tapes as a group,sufficient time is allotted for discussion,the integrity of the student evaluation process is insured, andthe tutor, in fact, meets his/her responsibilities for providing feedback to the instructor.
L. W. (Bill) Murray, Ph.D., is a professor of international business and finance and directs the international programs for the McLaren School of Business, University of San Francisco. He negotiated the first contract with the firm in China, taught the first TVI course, directed the initial research leading to the selection of the TVI technology, and regularly teaches in both TVI programs.
Alev Efendioglu, Ph.D., is a professor of management for the McLaren School of Business, University of San Francisco. Professor Efendioglu currently serves as director of one of the two Chinese MBA programs and regularly teaches in both TVI programs.
This article originally appeared in the 01/01/2000 issue of THE Journal.