Leveraging Student Feedback to Improve Teaching in Web-based Courses
- By Sunil Hazari, Donna Schno
- 06/01/99
##AUTHORSPLIT###- -->
Web-based course materials are being used extensively to provide supplementary as well as standalone instruction in educational settings. One of the most powerful elements of using the Web for teaching is the ability to engage learners in an interactive format. Although there are many technologies available that allow instructors to develop interactive Web course materials and elicit feedback from students, many educators choose to use the Web only for distributing static documents. This severely limits the potential of the Web to improve the teaching/learning process.
Feedback and Assessment can be used to monitor student progress, control the pace of learning, and evaluate teaching strategies. Using HTML Forms is a simple but powerful technique to collect student feedback and leverage it to improve teaching. Forms are used to collect data from the students who use Web-based course materials. Student feedback can provide the instructor formative evaluation of teaching during the semester. By using this evaluation method, instructors can get immediate feedback on course material, teaching style and student progress in order to make necessary adjustments. This article explores cognitive aspects of learning and using sample Educational Psychology courses to demonstrate how student feedback can be applied in education to improve teaching.
Many educators have recognized the potential of using the Internet for instruction. Although many Internet technologies such as e-mail, listservs, ftp and conferencing can be used to assist with teaching, the World Wide Web remains the most popular medium. It provides a user friendly front end and easy access to text, graphics, audio and video materials that may be used in a common and consistent format. Most education Web sites provide basic course information such as syllabus, schedule, announcements and reading lists. Others go beyond static materials to include synchronous or asynchronous communication, online testing, discussion groups, conferences, whiteboards, streaming audio and video. These type of materials are being made available in courses that meet in classrooms regularly and use Web materials as supplementary tools, as well as courses that are delivered entirely over the Web without traditional classroom meetings. The popularity of the Web for use in education can be seen from sites such as The World Lecture Hall (www.utexas.edu/world/lecture) that contains links to hundreds of courses created by educators worldwide. Course topic areas on this site range from Accounting and Advertising to Women's Studies and Zoology.
Web Interactivity
Chickering and Ehrmann (1998) suggest that while using technology to teach, educators should eschew materials that are didactic and instead search for technology-assisted solutions that are interactive, problem oriented, and that evoke student motivation. Web interactivity helps engage students in active application of knowledge, principles and values, and provides them with feedback that allows their understanding to grow and evolve. As compared to just accessing a static Web page and either reading or printing it, interactive components involve participation by having the user in some way interact with the Web environment. The interaction can be with content, other students, instructor, participation in a discussion group, quiz questions, simulation program, conferencing, live chat, or by filling out a feedback form.
Web environments can make use of one or more of these interactive components at any time. This type of interactivity may seem directly related to the constructivist theory. In designing learning environments, researchers (Honebein 1996; Lebow 1993; Knuth and Cunningham 1993) have recommended using constructivist theory for effective learning. The constructivist approach incorporates pedagogical goals in the knowledge construction process by providing appreciation for multiple perspectives, embedding learning in relevant contexts, encouraging ownership in the learning process, embedding learning in social experience, encouraging use of multiple modes of representation, and encouraging self awareness of the knowledge construction process (Vygotsky 1986; Bruner 1990).
Gagné and Briggs (1979) have stressed the need for evaluating students' understanding, providing feedback during evaluation and assessing complete understanding of each concept as part of the effective learning process. Interaction and feedback have significant impact on the learning process since they add value that results in improving quality and success in Web courses. Moore and Kearsley (1996) and Cornell and Martin (1997) have specifically identified interaction and feedback components as factors that influence student motivation in completing a course. In a research study, Comeaux (1995) also found that interaction and involvement lessened the psychological distance for students at remote learning sites.
With educators designing and customizing Web learning environments, it is necessary to determine if these environments are meeting the needs of learners. Mechanisms must be incorporated in Web-based environments to evaluate the medium, content, format, design and structure so timely intervention can occur if a problem is identified. Riel and Harasim (1994) indicate user feedback is one way of examining if the learning environment is successful in meeting learning outcomes. Although there are two categories of evaluation: Summative and Formative, as identified by Harasim, Hiltz, Teles and Turoff (1995), formative evaluation is best suited in a Web-based environment because of the advantage of instructor intervention in a timely manner. Moore and Kearsley (1996) also state that evaluation should be a continuous process and effectiveness of materials and media should be routinely assessed.
HTML Forms
In Web-based courses one way of collecting assessment and feedback data is by using HTML forms. Similar to paper forms, HTML forms have designated fields in which the user enters information that is sent back to the course instructor via the Web. In most cases students fill out the feedback form and press a "Submit" button that automatically forwards student-entered data to faculty as an e-mail message. Faculty can design and customize questions based on course content or outcomes to be measured. In most cases since the design d'es not go through reliability and validity statistical analysis, and is used for the sole purpose of improving teaching, it can be considered as Informal Assessment.
Questions in HTML forms may consist of check boxes, radio buttons, selection lists, and text input. Although the same effect of forms can be achieved by having students e-mail the requested data by clicking links on the Web course page, there are certain disadvantages associated with e-mail. Some of these disadvantages are lack of set customized responses, individual users' browser-specific quirks, unclear instructions on purpose of collecting data, and lack of structure to respond to information. Forms on the other hand can be integrated on Web pages, are easier to fill out, more focused, do not depend on browser-specific commands, and also provide a structure within the course context as displayed with other course Web materials. Many HTML editor programs (such as Microsoft FrontPage, Netscape Composer, Claris HomePage, and Adobe PageMill) include features to easily create forms using buttons and pull down menu options. A sample feedback form that can be used to collect student responses is shown in Figure 1. This form consists of instructions to the student and four questions that are indicators for assessing learning outcomes which in this sample case include pace of instruction, instructional strategy, and any open-ended student comments.
Application
This type of informal assessment technique has been successfully used in two university courses, ECLG 632 and ESEC 443 at California State University, San Bernardino. These are Educational Psychology courses that deal with learning theory. Because of the nature of these courses it was appropriate that the use of such techniques served as a powerful social cognitive modeling tool for the future teachers and counselors who were taking these courses.
The primary goal of using informal assessment techniques through the Web was to provide opportunities for the students to truly master the concepts being discussed in class. Mastery learning implies that "it is the task of instruction to find the means which will enable our students to master the subject under consideration" (Bloom 1968) and, according to Carroll, (1963) more time may be needed for students to master content. The use of an online feedback form enables students the opportunity after every class lesson to describe what they are learning such that it focuses their attention on and reinforces the key elements of the concepts they do know. Moreover, in describing what students do not understand, the course instructor is given useful information regarding concepts that still need reinforcement on both an individual basis and as a whole class. The use of the feedback form extends beyond in-class time constraints and is a non-threatening way for students to communicate their uncertainties regarding course content because it conveys the message that it is normal to understand some concepts and not understand others.
Regarding student feedback as it relates to the instructor's provision of enough examples and the pace of the class, the instructor is given an extraordinary opportunity to modify the course on an ongoing basis, or to maintain the flow of class as it is currently being presented. As one student indicated through the use of the feedback form, "I had a good understanding of the information that was taught in class last week. Yet after listening to the lecture I have been able to link my reading to the lecture. This has helped me sort out a couple of misunderstandings that I had about Behaviorist and Constructivist principles."
Course instructors can also create a template on their interactive Web module to pose questions before and after every lesson that align with levels of thinking as outlined by Bloom's taxonomy of the cognitive domain. Students fill in the form and submit it to the course instructor. This aligns well with Carroll's outline for a successful mastery approach in that it considers the most important points that will have later value or are basic to later learning, provides specific questions relating to the objectives as well as a format that students can use to organize their notes, and offers to students several exam questions based on each course objective (1971).
The critical thinking questions used feedback forms to provide opportunity for students to think about and thus reinforce the concepts discussed in class. In addition, the learning process can be spiraled for students as they are able to analyze the intricacies of class concepts on an ongoing basis, reflect upon previous concepts, and make connections among seemingly disparate components of theory. Furthermore, the course instructor can give praise and encouragement for responses that demonstrate strong critical analysis and reasoning, which has been noted as an ingredient of a successful mastery approach (Carroll 1971).
The course instructor can also use an idealized version of another student's response to a particular question as a model for students to follow, if their response is missing depth of analysis in some way. This serves to support the quality of students' work and represents a component to scaffolding where the instructor has presented an idealized version of critical analysis as a guideline for students to follow (Good and Brophy 1995). Students are able to explore other responses and regulate their understanding by recognizing that their response may lack such depth as the provision of enough examples to support their statement. This form of learning exemplifies how the course instructor can value the students' zone of proximal development, and engage students in the evaluation level of Bloom's taxonomy of the cognitive domain where they are given external criteria by which they can regulate their own learning.
This Web-based informal assessment represents a cognitive behavior modification technique designed to help students develop goal setting behavior, planning and self-monitoring (Good and Brophy 1995), and provides the opportunity for students to master the concepts (Bloom 1981). For example, students can regulate and monitor their own learning throughout the course in a sequential and constructive fashion as they respond to the questions and receive ongoing feedback.
As an example in response to a student's entry, the course instructor was able to give the following feedback: "good job identifying the primary differences [between Piaget and Vygotsky]. Your response is right on target. Should this be an exam question, you might want to elaborate a bit on each difference (e.g. give an example of each of these differences related to a teaching strategy for example). Thank you for taking advantage of this opportunity." The Web-based assessment form creates an iterative process to learning and growing in that the student is then able to provide a new response for further feedback. "Thanks for the feedback. You suggested that I may need examples of the differences of Vygotsky-vs- Piaget, and I'd like to run some by you to see if I am applying the theory correctly."
As can be seen, the course instructor becomes fully aware of the students' reasoning skills as related to their question responses, and can scaffold their learning process in a non-threatening fashion.
Additionally, the course instructor used the students' thought process as a guide for the next class discussion, truly integrating their ideas with class content and continuing to bridge the gap between what they know and still need to learn. The Web-based assessment enables true scaffolding because the course instructor can create critical thinking questions that represent the fluid nature of learning (i.e. the questions posed online follow the classroom discourse rather than devising predetermined questions that may or may not be related to the students' line of reasoning). These technology tools also enable students to analyze the course content in a direct and focused manner. Furthermore, the feedback and critical thinking forms provide the opportunity for students to demonstrate what they are learning, and how to grow in specific areas if needed, in a way that d'es not count against them through formal measures.
Benefits
At the beginning of a course, feedback forms can be used to familiarize students with the interactive Web technology while at the same time collecting data on initial student competencies. This may identify potential issues with students and/or technology. Once the course has started, similar to traditional courses, the feedback received can help faculty gain a perspective of the range of attained learning as well as individual competence (Rudmen 1983). This type of informal assessment can also be considered beneficial as it helps the instructor evaluate knowledge acquisition and retention rate of students in class. The instructor can get a better idea of what students are understanding as well as concepts that still need clarification to help plan subsequent classes.
Faculty are continuously able to monitor students' learning. It is also a means of engaging students in construction of knowledge. Since data is collected during the course there is a better chance of faculty intervention by identifying and taking corrective action on issues that affect learning. Depending on the way the feedback and assessment mechanism is set-up it can be delivered quickly and at regular time intervals (e.g., after each lecture, every two weeks, during midterm, etc.). If students fill out the required information outside the classroom, they get a better chance to reflect on the material before replying.
Also, the exercise d'es not take up valuable class time. Forms can also be set up so students are not required to identify themselves while replying to questions on the form. Anonymous feedback offers an opportunity for students to be able to bring forth comments they would not ordinarily have mentioned openly in face-to-face meetings or in group settings. This type of environment encourages students to take learning more seriously. Angelo and Cross (1993) suggest that after feedback has been collected, faculty can complete the loop by providing students with results of assessment data. As the approach becomes integrated with everyday classroom activities, this communications loop connecting faculty and students –; and teaching and learning –; becomes more efficient and effective.
Conclusion
In traditional as well as Web-based courses there is a gap between what is taught and what is learned. Reid (1997) mentions that methods of assessing the teaching and learning experience in online education are in high demand but short supply. No measurement yet exists that would adequately evaluate how well a faculty member performs in a virtual classroom. Angelo and Cross (1993) observe that by cooperating in assessment, students reinforce their grasp of course content and strengthen their own skills at self-assessment. Furthermore, student motivation is increased when they realize that faculty are interested in their success as learners. To achieve pedagogical improvements in interactive Web environments for assisting teaching and promoting learning, faculty can empower themselves by using technologies to facilitate a proven educational process of receiving and acting on feedback from learners.
References: Angelo, T., and Cross, K. 1993. Classroom Assessment Techniques, a Handbook for College Teachers (2nd ed.). San Francisco, CA: Jossey-Bass Publishers. Bloom, B.S. 1968. "Learning for mastery." Evaluation Comment 1(2): 1-12.
Bloom, B.S. 1981. All Our Children Learning. New York, NY: McGraw-Hill.
Bruner, J. 1990. Acts of Meaning. Cambridge, MA: Harvard University Press.
Carroll, J.B. 1963. "A model of school learning." Teachers College Record 64: 723-733.
Chickering, A., and Ehrmann, S. 1998. "Implementing the seven principles: Technology as a lever." Retrieved November 3, 1998 from the World Wide Web: www.tltgroup.org/ehrmann.htm.
Comeaux, P. 1995. "The impact of an interactive distance learning network on classroom communication." Communication Education 44: 355-361.
Cornell, R. and Martin, B 1997. "The role of motivation in Web-based instruction." In B. A. Khan (Ed.), Web-Based Instruction, pp. 93-100. Englewood Cliffs, NJ: Educational Technology Publications.
Gagné, R. M. and Briggs, L. J. 1979. Principles of Instructional Design (2nd ed.). New York, NY: Holt, Rinehart and Winston.
Good, T. L. and Brophy, J. 1995. Contemporary Educational Psychology (5th ed.). New York, NY: Longman.
Harasim, L., Hiltz, S. R., Teles, L. and Turoff, M. 1995. Learning Networks, A Field Guide to Teaching and Learning Online. Cambridge, MA: MIT Press.
Honebein, P.C. 1996. "Seven goals for the design of constructivist learning environments." In B.G. Wilson (Ed.), Constructivist Learning Environments: Case Studies in Instructional Design, pp. 11-24. Englewood Cliffs, NJ: Education Technology Publications.
Knuth, R. A., and Cunningham, D. J. 1993. "Tools for constructivism." In T. Duffy, J. Lowych, and D. Jonassen (Eds.), Designing Environments for Constructivist Learning, pp. 163-187. Berlin: Springer-Verlag.
Lebow, D. 1993. "Constructivist values for instructional systems design: Five principles for a new mindset." Educational Technology Research and Development, 41(3): 4-16.
Moore, M., and Kearsley, G. (1996). Distance education: A systems view. CA: Wadsworth Publishing.
Reid, J., Jr. 1997. "Measurements in online study: Items useful in quantifying pedagogical, social and technical issues." Retrieved November 3, 1998 from the World Wide Web: www.caso.com/articles/reid05.html.
Riel, M., and Harasim, L. 1994. "Research perspectives on network learning." Machine-Mediated Learning, 4(2-3): 91-113.
Rudman, H. 1989. Integrating Testing with Teaching. ERIC Document Reproduction Service No. ED 315 432
Vygotsky, L. 1986. Thought and Language. Cambridge, MA: MIT Press.
This article originally appeared in the 06/01/1999 issue of THE Journal.