Minneapolis District Finds Success With OMR Scanning for Testing & Assessment
In December 1993, after struggling for years with declining student achievement, administrators at the Minneapolis School District turned over operations to Public Strategies, a private, for-profit company. A major challenge was figuring out exactly what students knew and what they needed to be taught.
Many of the districtís 48,000 students come from poor, ethnically diverse backgrounds. The responsibility of demonstrating improvements throughout the school system fell on the shoulders of William L. Brown, director of the districtís department of Research, Evaluation and Assessment.
"Bowling in the Fog"
Brown describes the situation this way: "The problem was much like bowling in the fog. Without getting more specific information, you know you are not hitting the pins, but you donít know how nor what corrections will help you hit the mark."
When Public Strategies arrived, the district was using more standardized tests with strictly multiple-choice questions. Some educators there didnít believe the tests accurately measured studentsí knowledge. Others complained that the method did not allow teachers to receive timely results.
With an eight-person staff for 79 schools, Brown looked for outside expertise to supplement his internal resources. He got in touch with Scanning Concepts, Inc. (SCI), a full-service data collection company that happened to be located in Minneapolis.
"The original reason we went with SCI was that we needed to expand to different types of assessments, and we wanted to reduce the amount of human error in our existing scanning applications," says Brown.
The district outsourced the test forms development, scoring and reporting to SCIís Service Bureau. The firm provided a turnkey solution, including project planning and management, survey design, pre-slugging current student information onto each form, scanning and processing completed tests, performing statistical analyses, and summarizing results into customized reports.
Like many schools elsewhere in the nation, those in Minneapolis had experience with optical mark recognition (OMR) scannable documents. Brown considered adopting computer-based techniques, but decided that scanning offered more control. "We didnít believe that a computerized application would work at this time."
SLUGGER Comes to Bat
Scanning Concepts d'es not ignore the benefits of technology, however. In fact, their proprietary software, SLUGGER, greatly simplifies the task of delivering tests. The program automatically downloads student records from the districtís computers and pre-slugs data on each answer sheet.
"Prior to SCI, we had three staff working part-time for over a week to check for student errors in recording their student record data, and whether they entered their answers into the correct section of the standardized test form," says Brown. He estimates that SCIís Service Bureau saved the district about $4,000 per test.
Brown also called upon SCI to develop tests for writing and math, as well as a standardized achievement test for grades two, three, five and seven. This latter assignment presented unique challenges.
Normally, answer sheets pre-slugged with only the studentís name run about $1 each, plus $5 for scanning, processing and reporting. Shipping the tests to California cost the district an additional $30,000, and results didnít come back until the end of the school year. This cumbersome routine prevented parents from finding out the results until after the deadline for enrolling their children in summer classes.
Another problem with the old tests was the layout of the various sub-sections in the 30-page test booklet. Confusion over where students should mark their answers often introduced unnecessary errors.
According to Brown, SCI addressed all of these concerns and saved the district thousands of dollars. "They created a tailored version of the test for our fifth- and seventh-grade students which cost us only $1.53 per answer sheet. This included forms development, printing and pre-printing our current student record information, scanning, sorting, analyzing the data against the testing companyís normative database and developing two copies of a custom report for each student."
Thanks to SCI, standardized test results last year arrived two to three weeks ahead of schedule. The advantages for Minneapolis Public Schools didnít stop there. These days, teachers are more involved with the tests themselves.
More Ownership, Credibility
"Now, teachers write and select questions for the tests, which many times are more challenging than commercial standardized achievement tests," says Brown. "Teachers have more ownership and are more likely to use the results."
Brown also credits the SCI partnership with increasing the credibility of the assessment results within the community. "Public Strategiesí focus on student achievement brought about the change, but SCI enabled us to have multi-faceted testing instruments that give us multiple views of student performance with a moderately sized staff."
Write 503 on Inquiry Card
See "Internet-Based Testing: Vision or Reality," a Feature article in this monthís issue.