Facts and Assumptions of Assessment:Technology, The Missing Link
##AUTHORSPLIT##<--->{\rtf1\ansi\deff0{\fonttbl{\f0\fnil MS Sans Serif;}{\f1\fnil\fcharset0 MS Sans Serif;}}\viewkind4\uc1\pard\ri5520\lang1033\f0\fs16
\par \par \par
\par Many political leaders at the national, state and local levels have been promoting the value of large-scale assessment. Whenever changes in top government officials occur, education is usually one of the primary focal points. National reports such as \f1\ldblquote A Nation at Risk\rdblquote and \ldblquote What Work Requires of Schools: A SCANS (Secretary\rquote s Commission on Achieving Necessary Skills) Report for America 2000\rdblquote brought public attention to education inadequacies for a changing world, as well as set new directions for schools that required major changes for schools and school systems.
\par
\par Along with reforming curriculum, assessment and accountability took on a new meaning. Former President Bill Clinton utilized the SCANS Report as a pillar for his education plan while in office. Both Clinton and former U.S. Secretary of Education Richard W. Riley argued that voluntary national tests of reading in the fourth grade and mathematics in the eighth grade would have positive consequences for education. During hearings by the U.S. House of Representatives Subcommittee on Early Childhood and the Youth and Families, and the Committee on Education and the Workforce, Riley said, \ldblquote [National tests] will have national benchmarks to measure against, as they seek to refine and define state and local standards of excellence. I believe these tests are absolutely essential for the future of American education.\rdblquote
\par
\par President George W. Bush, along with U.S. Secretary of Education Rod Paige, had a similar theme of preparing children for the 21st century workforce. During both presidencies the goal has been to establish programs where no child would be left behind and all would have access to a well-rounded education. At the Summit on the 21st Century Workforce, Paige said that the culture needs to change from supporting educational systems to supporting the needs of children. An assurance of achievement by measuring the success of the students is needed. Paige\rquote s statement is very prolific, but the means of measuring success need to be further developed.
\par
\par According to Bush\rquote s No Child Left Behind Act of 2001, states must institute annual testing in grades 3-8 in basic skills to receive federal education money. If, after three years, the school d'es not show progress, federal funds would be cut and given directly to the families for alternative education. Mandated testing is a method to provide statistical information on the performance of a school, but tends to reveal only whether students can recognize or recall information.
\par
\par It is important that the government as well as school districts understand the importance of challenging students to be effective performers with the acquired knowledge. According to the SCANS Report (1991): \ldblquote Look beyond your discipline and your classroom to the other courses your students take, to your community, and to the lives of your students outside school. Help your students connect what they learn in class to the world outside.\rdblquote Educators, political forces, administrators, parents, the community and students all want the same thing: individuals who are prepared to be effective citizens in today\rquote s workforce.
\par \par
\par Technology Infusion
\par
\par The infusion of technology in schools has opened the door for opportunities to challenge governing bodies to use the technology to provide student assessment that will measure their abilities for connecting knowledge learned with real-world applications. Evidence indicates that when used effectively, \ldblquote technology applications can support higher-order thinking by engaging students in authentic, complex tasks within collaborative learning contexts\rdblquote (Means et al. 1993). Students should be able to use technology to deepen their understanding of the content in the academic standards and also advance their knowledge of the world around them. The SCANS Report provided the educational arena with the foundation to incorporate necessary real-world skills. In addition, technology has provided us with the tool to link academic standards with higher-order thinking skills that are necessary in properly assessing the competency of the learner.
\par \par
\par Alternative vs. Traditional Assessment Techniques
\par
\par The purpose of this article is to provide a comparison of alternative and traditional assessment techniques for teachers, administrators and political leaders to assist in making informed decisions relating to student success and evaluation. In addition, this article incorporates how technology can be used as a tool in student assessment. A review of secondary research was conducted to determine student performance differences through both traditional and alternative assessment, as well as methods of incorporating technology into the assessment factor.
\par
\par According to the Kentucky Department of Education (1999), assessment, in educational terms, is a test, planned procedure, method or task used to guide decisions about student learning, curriculum and instruction. In an article by Black and Wiliam (1998), assessment is described as a \ldblquote black box.\rdblquote Inputs from the outside such as students, teachers, administrators, parental anxieties, standards, tests and other variables are all fed into the box. The outputs that should follow are students who are more knowledgeable and competent, test scores that are better, as well as teachers who are satisfied with the outcomes. The authors question what is happening inside the box. How can validation take place on what inputs produce better outputs?
\par
\par Teachers are responsible for producing knowledgeable and competent students, but with additional variables the measurement tends to become less qualitative. Varied reports of assessment exist; but they generally fail to give clear accounts as to the other variables that may play a role in the success of an assessment method. For example, available research is often silent about the actual classroom methods used, the motivation and experience of the teachers, the nature of the tests used as measures of success, or the outlooks and expectations for the students involved (Black and Wiliam 1998).
\par \par
\par Curriculum and Instruction
\par
\par What d'es the research show with regard to curricular and instructional effects of performance assessments? A research report by Khattri, Kane and Reeve (1995) visited 16 schools across the United States that were developing and implementing performance assessments. They interviewed school personnel, students, parents and school board members; collected student work; and conducted observations. The authors concluded that, in general, \ldblquote our findings show that the effect of assessments on the curriculum teachers use in their classrooms has been marginal, although the impact on instruction and on teacher roles in some cases has been substantial\rdblquote (Khattri, Kane and Reeve 1995). Furthermore, when teachers are involved in the changing of curriculum and assessment, they tend to be more detailed in the planning and dissemination of learning.
\par
\par Chudowsky and Behuniak (1997) used teacher focus groups from seven schools representing a cross section of schools in Connecticut. These focus groups discussed their perceptions of the impact of the Connecticut Academic Performance Test (CAPT), which includes multiple-choice, grid-in, short answer and extended response items. Of all the teachers, 99% reported that preparing students and aligning their instruction to the test \ldblquote resulted in a narrowing of the curriculum.\rdblquote
\par
\par Teaching to the test has been a major disadvantage to standardized testing. Content that is not a critical part of the standardized test tends to be given less importance since the educational goal is to improve test scores. Traditional (standardized) testing is limited in measuring both cognitive and psychomotor skills. Although standardized tests are somewhat limited in measuring skill development, they are also ineffective in ascertaining student attitudes and behavior changes (Travis 1996). As long as schools use traditional tests to determine student outcomes, true measurement of abilities will be incomplete.
\par
\par Mertler (1999) completed a descriptive study of classroom assessment practices of Ohio teachers. He found some significant differences between assessment methods of teachers based on the different school levels and length of teaching experience. Traditional assessment techniques were used more by middle and high school teachers than by elementary teachers. Subsequently, teachers at the elementary level used alternative assessment techniques more frequently than secondary level teachers. The demographics of the study showed that teachers in suburban settings used alternative assessment methods more frequently than teachers in rural settings. Finally, the study showed teachers with fewer years of experience used alternative assessment more than teachers with 30 or more years of experience.
\par
\par Koretz et al. (1996) surveyed teachers and principals in Kentucky regarding the Kentucky Instructional Results Information System (KIRIS) and found that 90% of the teachers agreed that KIRIS caused them to de-emphasize or neglect untested material. Teaching to prepare students for a standardized test has always been one of the major cons in combining test results with variables such as additional funding, special grants and comparable statistics of similar school districts.
\par \par
\par Assessment Perspectives
\par
\par Alternative assessment is a broad term that encompasses several different assessment strategies. Alternative assessment techniques include performance-based assessments, observation techniques, student self-assessment and portfolios. An authentic indicator of student performance, alternative assessment closely resembles what students might be required to perform in class or on a real-world task. In contrast, traditional assessments consist of standardized and classroom achievement tests that are principally close-ended items. Traditional assessment is a passive and individual process of learning (Anderson 1998). The traditional assessment is objective, and looks at the cognitive and affective abilities separately. Advocates of traditional assessment argue that close-ended items are more objective and reliable than alternative types of assessment (Bol, Stephenson and O\rquote Connell 1998).
\par
\par Gipps (1994) compares the shift from testing to assessment to a shift from psychometrics to the assessment of learning. With this, a new emphasis on formative assessment is placed on improving learning (Crooks 1988; Sadler 1989). For effective formative assessment to take place it is necessary to set clear goals, develop appropriate learning tasks, as well as assess and give formative feedback to guide learning. Feedback must be used if assessment is formative (Sadler 1989). The context of the feedback, the associated learning opportunities and the broader context of students\rquote assumptions determine its effectiveness.
\par
\par Technology tools can be utilized to provide various approaches for learning the content and learner input in determining appropriate learning outcomes. Too often assessment for grading is overemphasized while assessment for learning is underemphasized. Assess-ment becomes meaningful when learners recognize that a gap exists between their present state of knowledge, understanding or skill and a desired goal. Learning occurs with action that is taken to close the gap (Sadler 1989).
\par \par
\par Contrasting Assessments
\par
\par In contrasting alternative and traditional assessments in detail, more technical differences arise. Alternative assessment, sometimes called authentic assessment, requires students to be effective performers with acquired knowledge (Wiggins 1990). Traditional tests tend to reveal only whether students can recognize or recall what was learned out of context. The criteria measured in these objective tests tend to be confined to knowledge and skills (Travis 1996).
\par
\par Alternative assessments present students with an assortment of tasks that reflect the challenges found in instructional activities, such as conducting research; writing, revising and discussing papers; providing an engaging oral analysis of a recent political event; or collaborating with others on a debate. When well designed and properly implemented, an alternative assessment, particularly a performance assessment, is closely aligned with curriculum and instruction that emphasize knowledge construction and problem solving in authentic contexts (Bol, Stephenson and O\rquote Connell 1998). In contrast, traditional tests are usually limited to paper and pencil, providing one-answer questions.
\par
\par In using alternative assessments, students can thoroughly revise and justify answers or products (Wiggins 1990). Alternative assessments require students to structure the assessment task, apply information and construct responses. In many cases, students must also be able to explain the processes by which they arrive at the answers (Kane et al. 1997). Traditional tests typically only ask students to write correct responses, regardless of reasons; as a result, students are not able to demonstrate their full extent of knowledge.
\par
\par Alternative assessment achieves validity and reliability by emphasizing and standardizing the appropriate criteria for scoring such varied products. Scoring rubrics can be used to judge the quality of student performance on a task. They specify the knowledge and competencies by which student work is to be evaluated and delineate the criteria for determining the quality of student work (Kane et al.1997).
\par \par
\par Assessment Research
\par
\par The research on assessment describes two major proponents: traditional and alternative assessment. Both forms of assessment have advantages and disadvantages. Traditional assessment is a very objective form of measurement that identifies a student\rquote s strengths and weaknesses relating to specific knowledge and skills. A major downfall of traditional assessment is that it d'es not take into consideration the holistic individual and how students can function as part of a team or apply analytical skills in solving problems. Conversely, alternative assessment is more subjective in nature. If it is not evaluated with valid tools, such as a rubric, it may not properly measure the intended objectives. Performance assessment allows for a broader form of evaluation and meets various learning styles of students.
\par
\par For assessment to challenge higher-order thinking skills of students, technology should be integrated to provide relevant and current information. The world outside the classroom has changed significantly due to technology. If academic institutions are to have relevance they must reflect on the effect information technology has had on society and the way students learn. The research significance d'es not assure that one form of assessment is better than the other. It d'es, however, show that to truly measure the ability of students, the assessment should be carefully chosen and not just based on the need for comparison data.
\par
\par Continuous research should occur as to what assessment techniques work best based on subject, age level, purpose of assessment, and the need for additional learning skills. The National Business Education Association\rquote s \ldblquote National Standards for Business Education\rdblquote provides a model for the business curriculum. The model uses the developmental approach for content mastery. The information system\rquote s component of the standards focuses on the \ldblquote how-to\rdblquote of technology. An integration of the information systems standards into academic standards would provide for excellent strategies for assessing students\rquote knowledge and skills. This method is excellent to use in evaluating what assessment technique works best.
\par \par
\par Conclusion
\par
\par Assessing student achievement has always been an area of debate on the benefits and drawbacks of authentic assessment versus traditional forms of assessment. Any assessment of student achievement is unlikely to show significant changes in results unless tangible forms of rewards, sanctions or public comparisons of students or schools are in place. This is why President Bush\rquote s proposal of standardized testing and individual state testing should show specific results.
\par
\par The uncertainty of standardized testing is that it d'esn\rquote t provide a clear picture of a student\rquote s knowledge and skill of subject matter, or the interrelatedness to other content areas. Mehrens (1998) writes that traditional standardized tests don\rquote t immediately generalize to larger domains of knowledge. With traditional testing methods, teachers may end up shaping lessons to the test format without teaching underlying concepts. A statistical measurement of traditional assessment would provide a comparison of results between districts, as well as provide a starting point for measurement. Although this seems to be the best method for showing improvement, it is not the best tool to measure the subjective nature of various learning standards.
\par
\par Alternative assessment integrated with information technology provides learners with the opportunity to prove their learning competency using different methods. The way one student learns is not the same way another student would learn. In today\rquote s workforce, people tend to work as a team and develop end products that take into account more than what would be tested on a standardized evaluation tool. A combination of traditional and alternative assessment techniques with the infusion of technology needs to be incorporated into the curriculum, and both should be used in measuring the competency of the students.
\par \par
\par \par
\par Maximizing Student Achievement Through Technology
\par
\par There is much written on the use of technology in the classroom to support curriculum. According to Market Data Retrieval\rquote s \ldblquote Technology in Education 2001\rdblquote report, access of computers for students is improving. In 2001, there was one Internet-connected computer for every 6.8 students, compared to one for every 19.7 students in 1998. Former Nebraska Sen. Bob Kerrey, chairman of the Congressional Web-Based Education Commission said, \ldblquote Technology offers tremendous potential for improving the delivery of education and we should not squander this opportunity\rdblquote (Trotter 2001). The Web-Based Education Commission\rquote s report, \ldblquote The Power of the Internet for Learning,\rdblquote provides recommendations for technology in education. One recommendation holds very true to learner assessment: \ldblquote New designs [for education] are needed to create the \lquote knowledge workers\rquote who will define the Information Age\rdblquote (2000). Information can be structured in various ways to support effective understanding of the knowledge and skills of the content.
\par
\par An article titled \ldblquote Better Students Through Technology\rdblquote (Conner 2002) provides 10 tips on using technology to maximize student achievement:
\par
\par 1. Get everyone on board
\par
\par 2. Learn from others
\par
\par 3. Utilize assessment tools
\par
\par 4. Determine and provide professional development needs
\par
\par 5. Establish curriculum objectives
\par
\par 6. Explore research-based instruction
\par
\par 7. Encourage parental involvement
\par
\par 8. Publicize progress
\par
\par 9. Locate funding sources
\par
\par 10. Utilize supporting resources
\par \par
\par The 10 tips are important functions for developing a plan for assessment through technology. Conner\rquote s article says that work must be done with teachers, administrators and curriculum developers to investigate, review and select the most appropriate standards-based assessment tools. Multiple measures of assessment should be utilized, including electronic portfolios and presentations (Conner 2002). In addition, both traditional and alternative assessment methods must be paired with technology to provide students with enhanced evaluation preparation as well as various modes of assessment.
\par \par \par
\par
\par Online Rubric Resources
\par
\par Alternative assessment tasks involve flexible challenges that help students rehearse for the roles of adult and professional life. When you combine the philosophy behind alternative assessment with technology integration you begin to see the opportunities for students to use higher-order thinking skills, such as thinking critically, analyzing, making inferences and solving problems. There are many Web sites that assist in creating and sharing rubrics, as well as provide educators with a starting point for developing rubrics that make alternative assessment activities more objective in the evaluation process. These sites include:
\par \par
\par \bullet MindSPACE III: Rubric Creation Process
\par http://kalama.d'e.hawaii.edu/~mspace/sim99/assessment/rubindex.html
\par \par
\par \bullet Transformation of Learning: Rubric Development
\par www.k12.hi.us/~transfor/rubric.html
\par \par
\par \bullet TeAch-nology.com\rquote s Teacher Rubric Makers
\par www.teach-nology.com/web_tools/rubrics
\par \par
\par \bullet Kathy Schrock\rquote s Guide for Educators: Assessment Rubrics
\par http://school.discovery.com/schrockguide/assess.html
\par \par
\par \par
\par References
\par
\par Anderson, R. 1998. \ldblquote Why Talk About Different Ways to Grade? The Shift From Traditional Assessment to Alternative Assessment.\rdblquote New Directions for Teaching and Learning, 74: 5-16.
\par \par
\par Black, P. and D. Wiliam. 1998. \ldblquote Inside the Black Box: Raising Standards Through Classroom Assessment.\rdblquote Phi Delta Kappan. Online: www.pdkintl.org/kappan/kbla9810.htm.
\par \par
\par Bol, L., P. Stephenson and A. O\rquote Connell. 1998. \ldblquote Influence of Experience, Grade Level and Subject Area on Teachers\rquote Assess-ment Practices.\rdblquote The Journal of Educational Research, 91 (6): 323-330.
\par \par
\par Chudowsky, N., and P. Behuniak. 1997. \ldblquote Establishing Consequential Validity for Large-Scale Performance Assessments.\rdblquote Paper presented at the annual meeting of the National Council of Measurement in Education: Chicago, IL.
\par \par
\par Conner, D. 2002. \ldblquote Better Students Through Technology!\rdblquote Education World. Online: www.education-world.com/a_tech/tech149.shtml.
\par \par
\par Crooks. T. J. 1988. \ldblquote The Impact of Classroom Evaluation Practices on Students.\rdblquote Review of Educational Research, 58: 438-481.
\par \par
\par Gipps, C. 1994. Beyond Testing: Towards a Theory of Education Assessment. London: Falmer Press.
\par \par
\par Kane, M., N. Khattri, A. Reeve and R. Adamson. 1997. \ldblquote Assessment of Student Performance.\rdblquote Studies of Education Reform. U.S. Education Department\rquote s Office of Educational Research and Improvement, Washington, D.C.
\par \par
\par Kentucky Department of Education. 1999. \ldblquote Commonwealth Accountability Testing System: Common Terms and Acronyms.
\par \par
\par Khattri, N., M. Kane and A. Reeve. 1995. \ldblquote How Performance Assessments Affect Teaching and Learning.\rdblquote [Research Report]. Educational Leadership.
\par \par
\par Koretz, D., S. Barron, K. Mitchell and B. Stecher, B. 1996. \ldblquote Perceived Effects of The Kentucky Instructional Results Information System.\rdblquote Institute on Education and Training, RAND.
\par \par
\par Means, B., J. Blando, K. Olson, T. Middleton, C. Morocco, A. Remz and J. Zorfass. 1993. \ldblquote Using Technology to Support Education Reform.\rdblquote U.S. Department of Education, Washington, D.C. Online: www.ed.gov/pubs/EdReformStudies/TechReforms.
\par \par
\par Mehrens, W. 1998. \ldblquote Consequences of Assessment: What is the Evidence?\rdblquote Education Policy Analysis Archives, 6(13). Online: http://epaa.asu.edu/epaa/v6n13.html.
\par \par
\par Mertler, Craig A. 1999. Assessing Student Performance: A Descriptive Study of the Classroom Assessment Practices of Ohio Teachers. Education, 120: 285.
\par \par
\par Report of the Web-Based Education Commission to the President and the Congress of the United States. 2000. \ldblquote The Power of the Internet for Learning.\rdblquote Online: www.ed.gov/offices/AC/WBEC/FinalReport/WBECReport.pdf.
\par \par
\par Sadler, R. 1989. \ldblquote Formative Assessment and The Design of Instructional Systems.\rdblquote Instructional Science, 18: 119-144.
\par \par
\par Secretary\rquote s Commission on Achieving Necessary Skills (SCANS). 1991. \ldblquote What Work Requires of Schools: A SCANS Report for America 2000.\rdblquote U.S. Department of Labor, Washington, D.C.
\par \par
\par Travis, J. 1996. \ldblquote Meaningful Assessment.\rdblquote Clearing House, 69 (5): 308-312.
\par \par
\par Trotter, A. 2001. \ldblquote Federal Study Details Major Barrier to Internet Learning.\rdblquote Education Week, 20 (16): 34.
\par \par
\par Wiggins, G. 1990. \ldblquote The Case for Authentic Assessment.\rdblquote ERIC Digest. Online: http://ericae.net/edo/ED328611.htm.
\par \f0 \par }
This article originally appeared in the 01/01/2003 issue of THE Journal.