New Products Streamline Testing Assessment
As parents and policymakers continue to clamor for educational reform, the issue of assessment has moved toward the center of the debate. Some critics have demanded new assessment techniques that more accurately reflect the capabilities students need to succeed in society. But who decides what students should know and be able to do?
In his State of the Union address, President Clinton proposed voluntary annual tests in English (grade 4) and math (grade 8). The U.S. Department of Education has signed a contract with an alliance of private companies to develop the tests, which will be made available to schools in 1999. Advocates of standardized tests point out that community members can be assured the results are impartial and not affected by grade inflation.
On both K-12 and college campuses, more teachers are experimenting with software packages that tightly integrate instruction and assessment. At some institutions, students can take exams over the Internet from the privacy and comfort of home. This article discusses several hardware and software products that help you create, deliver, score and analyze tests.
OMR Remains Popular
Despite all the evidence that technology enhances learning, few schools can afford to place a computer on every student's desk. Introduced decades ago, multiple-choice paper-and-pencil tests are an inexpensive way to measure the performance of hundreds or even thousands of students. Thus, optical mark reader (OMR) scanners and forms are far from obsolete. Although they may look roughly the same, OMR products these days are cheaper to own and easier to use.
Scanning Systems manufactures a variety of OMR scanners to match your budget. A low-cost model, the SR-601 can scan up to 2,200 sheets an hour (when equipped with a 200-sheet auto-feed). The unit reads any 48-column scannable form and interfaces with most testing applications. For more demanding environments, the SR-360 reads fully marked 8.5" x 11" forms and outputs data to a serial port at the rate of 85 sheets per minute. Because this scanner reflects light from the surface of the form, rather than passing light through the paper, it distinguishes valid marks from erasures. Designed with educators in mind, the OpScan 3 from National Computer Systems accommodates both NCS Mark Reflex and Transoptic forms ranging in size from 2.5" x 5" to 9" x 14". Sheet thickness detection prevents two sheets from being scanned as one, and skew detection ensures that crumpled forms are not misread. An open feed path allows the operator to easily inspect the read head and remove eraser dust, bits of paper or other foreign material transferred from the forms. The firm's latest scanning system, the NCS 5000i processes from 4,000 to 9,000 sheets per hour (see p.40 for a related article).
A familiar name in this field, Scantron remains a leading producer of OMR scanners and test forms. The Model 888P+ OMR machine scores multiple-choice tests by printing an error mark and the correct answer next to each wrong response; the total number of correct answers; and a serial number identifying the form. The unit also lets teachers combine an objective test (true/false, multiple choice or matching) with a performance-based test (essays, special projects, homework and oral reports) to calculate a single score. Scantron publishes a free 20-page color brochure with pictures and descriptions of its most popular testing forms.
From Scanning Concepts comes an innovative product that enables educators to create their own OMR forms for as little as three cents apiece. With Forms-On-Command for Windows, you drag and drop response items inside Microsoft Word, choosing from four bubble shapes and a library of prepared grids. When finished, one- or two-sided forms are printed on plain paper. An automated printer alignment wizard eliminates trial and error. Forms-On-Command also includes modules for scanning marked forms and saving the data.
A similar package, Principia's Remark Office OMR lets you economically produce machine-readable forms in small quantities on a laser printer or photocopier. After designing a form, the teacher scans it and defines how the fields should be interpreted (barcode, image or OMR). Version 4.0, scheduled for release this month, allows teachers to assign test points on a question-by-question basis, set up subjective points areas and more.
Both Scantron and NCS offer sophisticated test generation and scoring applications that work seamlessly with their respective scanners. Scantron's ParSCORE for Windows automatically scores answer forms, then generates detailed statistical reports for monitoring student performance. Its built-in gradebook, encrypted and password protected, stores an unlimited number of student records. With NCS ABACUS Test and Score, instructors may build exams either by hand picking or by randomly selecting items from the test bank. Within minutes of scoring tests, the software generates reports showing class ranking of scores, outcome mastery, frequency distribution and more.
Addressing the Standards
Linking assessment to state and national standards is a top priority at most K-12 districts. Several firms have introduced packages to address this need, including Scantron Quality Computers (The Curriculum Manager), Evans Newton (TargetTeach) and Tudor Publishing (Curriculum Designer). Compatible with scanners from Scantron and Scanning Systems, The Assessor from Software America tracks progress for any academic or skill level. It aligns assessments to national, state, district and course standards, identifying students who fall behind, meet or exceed the specified standards. Reports can be printed for any demographic subgroup, such as female athletes.
Although this article focuses narrowly on testing and assessment, it should be stressed that many related products help teachers evaluate how well students have reached their learning objectives. Electronic gradebooks from Jackson Software (Grade Quick) and Excelsior (Pinnacle) put an end to the tedious paperwork normally associated with grading. In addition, student information systems often contain modules for managing curriculum and assessing student progress. The McGraw-Hill Classroom System, for instance, comes with the WinScan scanning utility and InteGrade electronic gradebook.
With Acces (one "s") from EduCaide, teachers prepare pre- and post-tests by selecting problems from a printed catalog and entering the corresponding codes on their computer. If desired, the program will pick problems at random, scramble their order, or generate an alternate version of a test. EduCaide currently offers modules for Pre-Algebra, Algebra I, Algebra II/Trigonometry, Calculus and Canadian Math Grades 11 & 12. Additional modules cover the math portions of standardized exams such as the SAT, New York Regents and Texas Assessment of Academic Skills.
Another product tailored for mathematics courses, IPS Publishing's MathCheck lets one create paper tests in any format -- open ended, multiple choice, word or story problems. Items are keyed to outcomes in leading textbooks and to standards set by the National Council of Teachers of Mathematics (NCTM). The software even produces practice worksheets that reflect the objectives assigned to each student. For $295, customers get the main application (for Mac or Windows) plus one library title.
Instructors can add or modify questions using a built-in equation editor. Kevin Scott, a math teacher at Bellflower High School in Long Beach, Calif., says that by quickly generating "separate but equal" tests, MathCheck thwarts would-be cheaters. "This is important because I am teaching 175 seventh graders who might be called interactive and co-dependent."
The NCS Mathematics Assessment Series (for use with ABACUS) contains 4,000 professionally developed items in three formats (approximately 80% multiple choice, 10% short answer and 10% performance). Divided into three strands (K-4, 5-8 and 9-12), the series emphasizes "real world" problem solving. Teachers print items, answers, exemplars and rubrics from the CD-ROM at their discretion. Of course, math is not the only subject for which test item banks are available. NCS also offers the Essential Skills Banks, comprising over 31,000 test items in the areas of math, language arts, science and social studies.
Paperless Solutions
From Advantage Learning Systems comes S.T.A.R. (Standardized Test for Assessment of Reading), which employs "computer-adaptive testing" technology. Unlike the products previously mentioned, S.T.A.R. administers totally "paper-less" tests. Teachers no longer need to pass out questionnaires, scan forms and scrutinize data. Sitting in front of a Macintosh or Windows PC, the student selects the best word to fit in a sentence. The software analyzes each response and then presents the next question at an appropriately higher or lower reading level. Most students complete the test in less than 10 minutes. At Lincoln Elementary School in Twin Falls, Idaho, Principal Kay Jones reports increased motivation among students: "They look forward to their next S.T.A.R. test to see how much their reading ability has improved."
With Electronic Bookshelf, published by a firm with the same name, students read literature in class or at home, then take quizzes on the computer. Teachers may adjust the difficulty level of the tests, requiring that students get four out of five questions correct, for example. Another package, The Academy of Reading from Autoskill International includes a comprehensive battery of onscreen tests that measure speed and accuracy in phonemic awareness, oral and silent reading comprehension, and word recognition. The software analyzes the results of these tests to prescribe a course of study.
Rather than provide hard-and-fast scores for subjects like math and reading, some computer-based assessment tools measure students' overall intelligence or career aspirations. Virtual Knowledge's Ultimate IQ Test, based on the Test Your Intelligence series by Norman Sullivan, puts users through a "mental workout" covering four categories: verbal reasoning, numerical reasoning, visual-spatial reasoning and general knowledge. The Educational Testing Service (ETS) offers SIGI PLUS, a self-directed program to explore career options and locate colleges that can prepare graduates for those careers. ETS' Center for Performance Assessment conducts research dealing with the design and implementation of assessments that interpret complex student reasoning. The Center hopes to develop a quality control framework for so-called portfolio assessments, which document how students perform over time by collecting essays, audiotapes, drawings, etc.
The Grady Profile from Aurbach & Associates "digitizes" the portfolio approach. This Mac program enables teachers to record students talking or reading, copy their word-processed essays, scan their drawings and videotape their class presentations. By playing back the samples, teachers or parents can literally hear and see students grow intellectually. The Grady Profile Companion, for Apple's Newton MessagePad and eMate, allows one to make evaluations or observations on the playground or on field trips; data are transferred to the portfolios via serial cable.
Flocking to the Web
Not surprisingly, perhaps the biggest trend in assessment these days revolves around the Internet. Many observers believe that educators will abandon paper-and-pencil tests in favor of Web-based counterparts. In his book The Road Ahead, Microsoft founder and CEO Bill Gates predicts that, "Testing with self-administered quizzes on the Internet will become a positive part of the learning process."
ComputerPREP, a leading provider of training products, this year upgraded Drake Web@ssessor, which lets you author, manage, deliver and analyze assessment over the Internet or an intranet. The program's point-and-click interface d'es not require any knowledge of HTML or Java. Instructors may create assessment devices in any language and add any amount of pictures, video clips, animation, graphics or sound files. A music professor, for example, could insert a sound clip of Beethoven's Fifth Symphony and ask students to select the multiple choice answer that identifies it.
Chariot Graphics' WebTest resides on the firm's own high-capacity server, eliminating the need for schools to invest in extra hardware or support personnel. Teachers can design their own tests or simply send their question files to Chariot (preferably in MicroTest III format). WebTest staff assign ID codes and passwords, allowing students to securely take the test from any PC with Internet access and a standard Web browser.
In the near future, look for private companies to play an even greater role in educational assessment. Virtual University Enterprises (VUE), a division of NCS, began offering Internet-based testing this summer. VUE utilizes its custom Program Management software, advanced communications technologies and education consultants to remove the burden from training and testing groups of students. Besides "high stakes" assessment -- arranged at dedicated test centers for specific dates -- the firm administers multiple choice exams via the Web into unproctored locations.
This article originally appeared in the 11/01/1997 issue of THE Journal.