Student Assessment Comparison of Lecture and Online Construction Equipment and Methods Classes
##AUTHORSPLIT##<--->
Many university programs are beginning to integrate online classes into their curriculums. The newness of the Internet delivery method raises many questions about class administration and quality assurance. Issues of online class quality are just being explored. Comparisons to traditional lecture delivery classes are just being documented. It has been found "that the learning of the online students is equal to the learning of inclass students," comparing pre and post-tests of knowledge for both groups of participants (Schulman and Sims 1999). This quantitative comparison is a direct indicator that the online learning environment can be as effective as the traditional lecture environment.
Many university programs are beginning to integrate online classes into their curriculums. The newness of the Internet delivery method raises many questions about class administration and quality assurance. Issues of online class quality are just being explored. Comparisons to traditional lecture delivery classes are just being documented. It has been found "that the learning of the online students is equal to the learning of inclass students," comparing pre and post-tests of knowledge for both groups of participants (Schulman and Sims 1999). This quantitative comparison is a direct indicator that the online learning environment can be as effective as the traditional lecture environment.
A convenient means of gathering online class assessment would be to use criteria already in place for lecture classes. Success can be evaluated by comparing lecture and online class assessments and final outcomes. However, this raises a primary question. "Can online classes be judged by the same quality standards as traditional lecture classes?" There has been little documentation concerning student perceptions comparing online to traditional lecture class quality.
At the April 1999 National Construction Industry Education Forum (NCIEF) in Las Vegas, Nevada, discussion about online construction courses focused on quality assurance. During this discussion it was recognized that online construction classes should meet the same quality standards as traditional lecture classes and these standards can be evaluated the same way. Part of the NCIEF discussion focused on the Construction Equipment and Methods classes offered Spring 1998 and 1999 by the University of Oklahoma (OU) Construction Science Division (CNS). Twenty-five construction students from nine geographically separate universities successfully completed the online classes. As part of the lecture and online class evaluations the instructor used the OU College of Architecture (CoA) Non-Studio Course Evaluation to gather student ratings concerning the course instructor, course content and student performance.
The CNS 4913:
Construction Equipment
and Methods Classes
The 1998 and 1999 Classes
During the Spring 1998 semester, a Web-based construction equipment and methods class was taught in conjunction with a 26 student lecture class meeting twice a week at the OU. The online class was administered from a custom-built Web site designed, implemented and maintained to replace the traditional class methodology (http://www.ou.edu/architecture/dcns/cns4913/). The online teaching model required students to check the site regularly in the same fashion as the traditional teaching model required traditional class students to attend lectures. The site was formatted like a "book" of organized information to be used as a class information resource. Class participants were required to use Windows 95 and Office 97 for assignments. They used e-mail, the telephone, the chat feature and limited desktop video conferencing for communication between each other and the instructor. Participating construction programs included Oklahoma State, Texas A&M, Auburn, East Carolina and Cincinnati.
Participants in the 1998 OU CNS 4913 lecture class did not have access to the class Web site this semester, but course content and sequence were the same for both classes. Students in both classes were given the same homework assignments and exams at approximately the same times during the semester. Class assessment for both groups was mostly closed-book and required a faculty proctor for all examinations. The purpose of this parallel delivery was for direct performance assessment and administration comparison between the two instructional strategies.
An interesting observation was made as the semester progressed and the instructor became more familiar with the class administration. The lectures typically contained and followed information in the Web site. Traditional participants took notes during these lectures to be used for homework and exams. The Web students did not attend lectures, but by printing the Web site pages for reference, actually had a better hard-copy set of notes than the participants attending the traditional lectures. The trade-off, however, is that the Web participants miss the class interaction determining what information is the most important. This interaction also provides a convenient format in which to answer questions.
Based upon this experience, as well as course evaluations and subjective assessment from participants, the Web site and course were revised and updated. The class was advertised to all ASC programs and review of a preview Web site was suggested to potential participants (http://www.ou.edu/architecture/dcns/cns4913/preview/). During Spring 1999 fourteen participants from Oklahoma State, Texas A&M, Texas Tech, Auburn, University of Wisconsin-Stout and Cal Poly-San Luis Obispo participated in the online class along with 26 OU students in the lecture group. During this class all participants used the Web site for information, communication and to obtain homework exercises.
Class Quality Comparison
Methodology
Paralleling the CNS 4913 class deliveries provided a unique opportunity for comparison of students’ quality perceptions of traditional lecture and online strategies. The author believed that if the online participants’ quality perceptions and the outcome assessment grades were similar to those of the lecture class, then it could be assumed that the online class was as effective as the lecture class.
The 1998 lecture and Web student groups were compared to determine whether the participants were similar in background, knowledge and attitude. The Test of Logical Thinking (TOLT) was administered between groups comparing basic reasoning ability, and was given once as a pretest. The Test of Construction Attitude (TOCA) was used to investigate empowerment, attitude and motivation toward professional construction management and used to evaluate student attitudes and motivation toward learning and instruction within and between groups.
The OU CoA Non-Studio Course Evaluation was used for the 1998 and 1999 classes to gather students’ perceptions of instructor performance, course content and self-evaluation at the end of each semester. Additional information was also collected from the online classes using a participant survey at the conclusion of the course offerings, instructor observations and online participant communications.
Results
Results of the TOLT instrument indicated that the difference between the two groups was not significant. They showed that the online class considered themselves to be more professional than the lecture class. These instruments were not used in the Spring 1999 class due to the known similarities of these students compared to the group participating in the lecture class.
The course evaluation instrument used in the OU CoA investigates the course instructor, course content and student performance divided into four categories. The first category deals with the qualities of the instructor and administration of the class. The second category deals with prerequisite knowledge and class assignments. The third category deals with the number of exams. The fourth and last category deals with course comparison of the instructor and the student.
Upon initial review, it appears that several of the evaluation questions are not applicable because of the online delivery of the class. For instance, the questions dealing with instructor communication and with the comparison of the instructor to other faculty members do not seem applicable in the traditional format. They are still, however, applicable when used in the context of online class delivery. Effective communication and availability of the instructor are still an integral part of a successful class experience, regardless of the delivery method. The instructor’s performance can be compared to other construction faculty in the participants’ programs.
Observations
The actual numerical ratings are not the focus of this discussion. They are, however, consistent for both methods of class delivery for both semesters the classes were offered. There are no trends or evidence of different quality perception between the classes based upon answers to the questions contained in the survey. The final grades for online and lecture participants were not significantly different for either course offering. To the author, these comparisons signify that teaching techniques and styles are suitable for both methods. For further discussion about these topics go to http://www.ou.edu/architecture/dcns/ryan/bestpractices.htm.
The Web class ratings of Questions 2 and 4 (2. Expressing clearly and concisely by instructor, and 4. Availability and helpfulness outside of class by instructor) are very close for both semesters and are consistent with the lecture class ratings. It should be noted that of the 26 students completing the non-studio class evaluation during the two class offerings, three participants marked "not applicable" on the assessment form for these questions.
Though still in the same range as the Spring 98 class, the ratings of Question 12 (rating instructor compared to other faculty members) Spring 99 were 1.24 for the lecture class and 1.91 for the Web class. A possible reason for this rating difference is that lecture delivery offers participants an easier opportunity to interact with the instructor and form a basis for comparison to other faculty. Interaction with the online instructor using e-mail, telephone or chat demands greater efficiency than open oral discussion and therefore is more limited. This is perhaps the greatest limitation of the online delivery method. Almost all online participants felt that this was the greatest weakness of the class and suggested that mandatory times for interaction be included in the class format. It should be noted that the availability of the class on the Web all the time was the greatest strength of the class.
The opportunities to work independently at one’s own pace and to communicate anonymously are two of the primary advantages of working online. When preparing an online class the instructor must decide how strongly these advantages are to be embraced in the course administration. The amount of collaboration and interaction required of participants will be determined by this decision. Extensive collaboration and interaction will require greater effort by the instructor to motivate participants to communicate using Web capabilities such as e-mail or chat.
It should be noted that not all content may be suitable for teaching online. One of the determinants of this suitability is the degree and method of testing that is necessary to assess participants’ understanding. "Open book" testing requires the least amount of faculty intervention. Greater effort must be made to find proctors for exams at the participants’ locations if more controlled examination is necessary. The major disadvantage to "open book" testing is the reliance on the participant to work independently. The assessment method and rigor of examinations must match the level of desired demonstrated learning. The anonymity of the distance learning format also places a large responsibility and reliance on the participant to follow specified guidelines and to work independently when required. "Developing effective and reliable assessment methods for online class participants perhaps will demand the greatest effort for innovation and departure from traditional practices." (Ryan, http://www.ou.edu/architecture/dcns/ryan/bestpractices.htm, 1999)
Conclusion
Should online classes be considered as "good" as lecture classes, or treated as an alternative or the "next best thing"? The quality of online class offerings will influence this perception. Class administrators and participants must ask themselves, "Should there be a trade-off for class quality and the convenience of the delivery method?" Class quality should not be compromised for the sake of posting the class online for business reasons, because others are doing it, or for the convenience that the Internet brings. There is great potential for exceptional classes to be industry-sponsored productions, combining all emerging capabilities of the Internet.
A necessary step toward online class quality assurance is determining how classes are to be evaluated by participants. Regardless of the delivery method, issues of quality are the same. Recognizing that online classes should be evaluated in the same way as lecture classes is a necessary step to establish standards for quality.
Richard Ryan is an Associate Professor of Construction Science at the University of Oklahoma. He has used the Web to supplement construction class experiences since 1996. He has consulted with construction companies pertaining to marketing, training, and project documentation utilizing new information technologies. Currently he is investigating methods of online testing that can be constructively used to supplement and promote the online learning experience.
E-mail: [email protected]
References
Ryan, R. 1999. "Best Practice Suggestions for Custom Building a Technology Class Web Site and Administering the Class," WWW document. URL: http://www.ou.edu/architecture/dcns/ryan/bestpractices.htm.
Ryan, R. 1999. "Construction Equipment and Methods Class on the Web," Syllabus99 Proceedings, WWW document. URL: http://www.syllabus.com/syll99_proceedings/CONSTRUC.HTM.
Schulman, A. H. and Sims, R. L. 1999. "Learning in an Online Format versus an In-class Format: An Experimental Study." T. H. E. Journal, 26 (11), 54-56.
This article originally appeared in the 01/01/2000 issue of THE Journal.