New Assessment Tools Provide a Closer Look at Students' Mastery
by William Willis While the face and nature of assessment has evolved rapidly the past few years (from traditional methods like letter grades to portfolios and profiles of achievement), the fundamentals driving assessment have stayed the same. Community expectations, accreditation, plus local, state and, increasingly, national standards all demand more accountability. But, while continuous assessment for students and faculty is necessary to both fulfill learning objectives and for accreditation, the funds to meet these goals have shrunk as rapidly as the need has grown. This article profiles some of the many assessment tools that can help educators stay ahead of the curve and within budget. Full-On Systems Attempting to help educators meet all of their assessment needs, some firms offer software that lets educators create, distribute, report on, and analyze any number of various tests. The Curriculum Manager Series from Scantron Quality Computers is a set of software programs and classroom materials that help educators design, evaluate and teach math and science around curriculum objectives set by groups such as the NCTM, NSTA and AAAS. The software creates a test or worksheet from a pre-written database of questions matched to selected objectives. Tests can be administered online in a computer lab or printed out. After students finish online tests, instructors receive an individual progress report with the date, comparing results to objectives selected. Instructors can also edit pre-written questions as well as create their own database of questions for an aligned curriculum. Onscreen graphics help students visualize math and science problems. The NCS ABACUS Test package lets educators easily construct an even wider variety of learner-directed assessments including performance-based, learner-constructed or a combination. One chooses from thousands of professionally developed and field-tested items in one of the many NCS Item Banks available. NCS ABACUS Score, another product, works with the firm's OMR units to scan and score tests created with ABACUS Test. Standard reports generated include Class Ranking, Item Response Analysis, Test Response Analysis, Test Frequency Distribution and Individual Student Test Diagnostic. The NCS ABACUS Instructional Management System seamlessly integrates both ABACUS Test and ABACUS Score for fully automated assessment and instructional management. Another program to create and organize questions for all kinds of student assessment is Question Mark for Windows, by Question Mark America, which integrates sound, video and multimedia. Questions can be randomized and set with time limits and other variables. Instructors can collate, review and analyze answers. QM also offers statistics on the results of tests or surveys and allows answer files to be exported to graphing, analytical and statistical packages. One of the more sophisticated products is MicroCAT, from Assessment Systems, designed to support the entire testing process. The program helps to create and maintain item banks containing text, graphics and digitized images; develop and create printed test forms; and develop and administer computerized tests, from simple traditional exams to sophisticated adaptive tests. It then performs conventional item analysis, Rasch analysis, or two- and three-parameter logistic IRT calibration. Targeting Specific Needs While some packages attempt to address an institution's entire assessment needs, others are more specialized in focus, targeting only specific areas of assessment. One such product is S.T.A.R., from Advantage Learning Systems, Inc., an innovative reading assessment package. S.T.A.R. is a computer-adaptive, norm- referenced reading test that takes under ten minutes to complete and delivers results immediately. When students choose the best word to complete a sentence, the software weighs each answer and then presents the next question with a higher or lower difficulty. S.T.A.R.'s pool of test questions spans from primary to post-secondary levels, and tests can be taken up to five times per year. For mathematics, MathCheck, from ips Publishing, supports NCTM Assessment Standards, to give math and science educators a quick and inexpensive way to produce dynamic, custom-aligned assessments. A unique algorithm-generation system can generate thousands of different-but- equivalent permutations of each learning objective. Even complex story problems can be devised in minutes. Homework assignments, cooperative learning worksheets, daily quizzes and complete performance-based or authentic assessment final exams can all be easily created by MathCheck. On a different front, counseling and advising departments should note Invest Learning's Aptitude & Interest Inventory. This package offers a battery of seven aptitude tests and a 200-item interest inventory. Aptitude Inventory is a one hour and 40-minute computerized test battery that analyzes and assesses a person's aptitudes, abilities and capacities for learning. Interest Inventory, meanwhile, matches individuals to occupations and training based on the kinds of activities they like. All results are presented in terms of potential success in careers, training programs and post-secondary education. Skills Pkgs., Business as Usual Some packages are primarily for business schools, but can also be used in computer labs for accreditation in specific skills proficiency. Individual Software's Skill Assessment Software assesses proficiency in 15 of the leading Windows applications. It includes pre- and post-testing options. Graphical reports display both pre- and post-test scores, plus recommends which built-in CBT lessons will increase proficiency. Also suited for business schools, PASS!, from South-Western Educational Publishing, lets students use the actual business software being tested for, while they are testing. PASS! assesses students' knowledge of: Corel WordPerfect; Microsoft Word, Excel and Access; Lotus 1-2-3; Quattro Pro; dBase and Paradox, with more on the way. The SAT Made Simple With such importance placed on the SAT and other assessment tests, firms have responded with many products to help students prepare, assess their abilities and supply feedback to educators. Score Builder for the SAT, from The Learning Co., is based on SAT expert Dr. Gary Gruber's proven techniques. It uses four special tests to identify not only the skills, but also the test-taking strategies students need to raise their scores. Practice test scores are tracked with immediate feedback to build test-taking confidence. Scores are compared to previous ones, as well as to the average scores for more than 900 U.S. colleges. Straight from The College Board comes One-on-One with the SAT, the only prep program with actual SAT questions. It lets students take a "real" SAT on their computer. Customized study sessions are provided. They feature robust performance tracking, study plans and strategies, question explanations and hints, a glossary, an onscreen calculator and reference books, a pacer and timer, plus functions for guessing, marking and eliminating answers. Students are guided through every type of question encountered on the SAT. Special counselor features allow teachers to monitor and summarize students' improvement, individually or as a class, and help create effective study plans. A demonstration version of One-on-One with the SAT may be downloaded from the firm's Web site. Another prep package, Chancellor & Dean Complete Multimedia Course for the SAT, employs natural voice-over narration, synchronized with graphics and animation, to offer verbal explanations and guidance. It delivers feedback on every test question, within context, at a student's request. Plus math and strategy notebooks are hot-linked to every practice question, to detail the best test-taking strategies for that question. Kaplan offers various packages to help students get ready for the SAT, GRE, LSAT, GMAT or ACT. For example, SAT RoadTrip Multimedia analyzes and determines study paths, identifies strengths and weaknesses, drills students on each question type in Verbal and Math, and gives strategies to help boost scores. Cliffs Notes offers its Cliffs StudyWare CD-ROM featuring onscreen reports and graphs, tests, hundreds of review questions with explanations and more. Cliffs StudyWare is available for ACT, GRE, SAT, GMAT and LSAT tests. Don't Forget Hardware The trend toward more affordable, low-cost OMR scanning means several popular hardware makers are offering lower entry-cost OMR units. Scantron antes up with their new ScanMark 2000ADF, which features a standard 250-sheet capacity automatic document feeder with double-sheet detection. Two programmable output hoppers add to the machine's "set and go" performance by allowing mismarked or incomplete documents to be separated for later correction without interfering with the scanning process. The ScanMark 2000ADF reads a variety of marks and also features options like a bar code reader, an ink-read option and an interactive printer that prints messages on forms. The OpScan 3 is NCS' affordable OMR scanner. It can scan up to 100 sheets automatically, all without any adjustment when changing to forms of a different thickness. It also reads single-sheet as well as multiple-sheet booklets without additional equipment. OpScan 3's rate of 20 sheets per minute speeds data entry, and it can scan both front and back sides with an optional two-sided-read capability. Options are a bar code reader, a printer and an auxiliary port that supports remote use on a multiple-user computer system. For low-volume needs, Scanning Systems offers its SR-607 scanner. It accepts various paper weights and features dual-sided reading, automatic de-skewing and self-test, a snap-open feed path and 16-level mark discrimination. Options let one outfit the SR-607 for particular needs. A 200-sheet auto-feed lets it scan up to 2,200 sheets per hour, while an interactive printer, bar code reader and ink-read head all expand the unit's functionality. "Helper" Applications Finally, there are quite a few "helper apps" that assist in creating forms and more, saving the cost of pre-printed forms and expensive OMR scanners. Scanning Concepts' Forms-On-Command software, for instance, lets one quickly create, print and scan forms using a standard single-sided or duplex laser printer and OMR scanner. Another package, Slugger, is designed to pre-slug information (mark in machine-readable format) onto pre-printed OMR forms via a laser printer. Student names, IDs and grade levels can be added to forms before distribution, increasing completion rates and data integrity. Unlike traditional OMR systems, Principia's Remark software OMR works with plain-paper forms and common desktop scanners. Forms can be created with any word processor and printed by a laser printer; completed forms can be scanned in by most desktop scanners; and data from Remark can be analyzed with packages like SPSS, SAS, dBase, Excel, etc. Remark also includes test grading and survey tabulation functions that produce item analysis, student grade report and class distribution graphs as well as test statistics. With Form Shop for Windows, from Scanning Dynamics, educators can customize forms with text, grids and graphics. Forms can be printed by a laser printer, and the ability to pre-slug forms with various types of data in one pass speeds up data processing. What Else? The Internet Like all other areas of education, testing and assessment is expanding onto the Internet. From software that lets one convert existing tests into HTML code and post them on the Web (QM Web, from Question Mark America), to a comprehensive Web-based, customizable skill-assessment program designed with Java (PREP Web@ssessor, by ComputerPREP), software firms are quickly taking assessment online. And the efforts aren't just commercial endeavors. The University of California at Irvine, for instance, is developing a computerized assessment system for mathematics that can be accessed through the Internet. For more information on this interesting project, see http://www.spaces.uci.edu. And Indiana State University has an interesting Web site (http://www-isu.indstate.edu/dragon/ix-indx.html) that lists descriptions of over 100 assessment tools and links to related information. Finally, ETS (Educational Testing Service) recently opened a Web site (http://www.ets.org) with information on major testing programs such as the GRE, T'EFL, GMAT and others. Choices Abound Mac users, especially those with Newtons, can do comprehensive assessments with Grady Profile, from Aurbach Associates or Learner's Profile, from Sunburst Communications. Both packages run on, and transfer data between, Macs and Newtons. Courseware from the likes of CCC, Jostens and TRO all integrate assessment functions. IBM's new SchoolVista class-learning system sports its own Assessment Suite as well. A range of individual add-ons as well as modules for courseware "systems" now exist. All mean better education for students through a more well-rounded view of their performance and mastery.