10 Lessons Learned From the Assessment Field Tests

Schools and districts that took part in the PARCC and Smarter Balanced trial runs share their experiences to help you prepare for online testing this spring.

• By Dian Schaffhauser
• 11/18/14

According to Chief Technology Officer Brandt Redd, the Smarter Balanced Assessment Consortium saw “significantly” fewer problems than it had expected during its spring 2014 field tests. “Things went more smoothly than our expectations,” said Redd. “We didn't have any systemwide issues; issues that happened tended to be isolated." That lack of major issues was in large part due to the efforts of educators all over the country who put in the time to make sure everything would work before nearly 5 million students showed up to take the field tests — whether for Smarter Balanced, PARCC or one of the alternative state online initiatives. The lessons that the field testers learned last spring can help you prepare so that your school or district's experience with the real deal goes as smoothly as possible.

Prioritize Your Infrastructure

Even before Dennis Villano, director of technology integration for Burlington Public Schools (MA), had ever heard of PARCC, his district was investing in a broadband and wireless infrastructure that could eventually sustain a 1-to-1 program. “For us, it was about education,” he said. “It was about implementing technology into the environment.”

Lately, though, Villano has been hearing a lot of school tech directors “talk about the devices first.” That's the wrong approach, he asserted. “You have to talk about the infrastructure first. Get that working, get that up to speed, spend the money where you need to there, and then talk about the devices.”

According to Jesús Aguirre, state superintendent of education, local education agencies within the District of Columbia each received a share of a $4 million allotment to help them get ready for the PARCC field testing and the live assessment. (Another round of funding was distributed in October.) DC gave individual schools the choice of spending the money on new devices or on infrastructure since, as Aguirre noted, “Every school is different. They’re going to make the better decision about what they need.” To make the most informed decision possible, it can help for tech leaders and staff to turn their priorities into a shopping list that you can consult when funding becomes available.

Do a Dry Run

The biggest piece of technology advice that came out of the field testing, according to Redd, was the importance of doing a dry run. Redd offered this example: If you're taking a room that's normally used for other purposes and dedicating it to testing, you may find that once you've packed it with “30 computers and 30 people, the air conditioning isn't adequate to keep it cool. It's not the thing you'd think of right off the bat.” Another possible scenario: You find out too late there isn't enough power to run all the systems.

Then there are the IT concerns, he noted, such as the wireless access point that serves that room: “Does it have sufficient capacity for that many computers and bandwidth need? You might have enough Internet bandwidth coming into the school, but it may be that the access point that serves a particular sector of the school was expecting a dozen devices, and all of a sudden you have 30 or 40, and it's not built for that much capacity.”

There’s no substitute, Redd stressed, “for setting up all your computers and getting the equivalent number of people in front of those computers, whether it’s students volunteering to stay after school to play around or a bunch of parents and teachers bringing up a practice test and trying it out and making sure everything works."

Prepare Staff for New Priorities

As the largest district in the state, Burlington Public Schools acted as a proxy for other districts during the field testing. So even though its schools run 1-to-1 programs in grades 1 through 12, for the purposes of the PARCC assessments, it tested not only on iPad 2s, but also on Chromebooks, Windows PCs and iMacs.

The Student Information Office dedicated two months to getting everything ready for the test, including working with Villano to develop a schedule of testing sessions that they kicked back and forth for days on Google Docs. “I didn't have much to do at all with standardized testing before,” Villano said. “Now we were totally dedicating our time to working on it.” Once the schedule was in place, the IT department dedicated about a month and a half to preparing the devices, “something that normally they wouldn't have to do,” said Villano.

Try a “SWAT” Approach

Oxford Middle School (AL) Principal Kyle McCartney said that there’s no such thing as overplanning when a school is undertaking a major initiative such as the transition to online testing. Beyond having a plan, he added, make sure you “have a lot of people to help you out — people who know what they're doing onsite, who can help you avoid problems and prevent them from becoming bigger problems."

Since his school was shifting to 1-to-1 at the same time it was preparing for the new assessments, the district provided a one-person SWAT team: Sherita Hayes, an instructional technology specialist at Oxford City Schools. Hayes, who has a computer science and math background, held weekly open-door sessions onsite for teachers and staff, helping them make the transition to new computers (PC to Mac), new instructional techniques (blended learning) and new tools (such as Google Apps, LanSchool for classroom management and Apple TV). She also went from class to class, answering questions and helping teachers work with students in using their computers.

Over the course of year, McCartney said, “She was able to help us get on our feet, and now we’re able to do it without her, because she did such a good job.” She has now moved on to helping the entire district shift to new ways of learning and assessment.

Aguirre reported that, following the field testing, DC sent a consulting team to visit every school and take a tech inventory. It soon became evident that schools needed other kinds of help. “Initially, it originated as an IT initiative, because we just wanted to make sure we knew what was out there on the technical side,” he recalled. “As soon as we started, we quickly learned that many of the questions the schools had were not just about IT but about how to roll this out. So we supplemented those teams with folks who knew much more about the instructional side and about PARCC itself.”

Next, the consulting team will compile its results and DC will use them, explained Aguirre, “to help guide how we roll out these funds, but also to give it back to these schools to say, ‘Here’s what we’re finding. The standard for us will be the minimum technology requirements that PARCC has. Here’s where you are, and here’s where we recommend you invest these funds we're going to send to you.’ ”

Adjust on the Fly

No matter how well you plan, preparations are bound to be complicated by the unexpected. At Burlington, there were differences to deal with for each device. The iPad uses a Pearson TestNav app from Apple’s App Store, while Chromebooks use a different version from the Chrome Web Store. And the desktops needed to have a specific Java installation that runs in the background.

During the field testing, both the state department of education and Pearson held a daily briefing with district representatives to share problems and possible solutions. Burlington's Villano tuned in as much as possible to stay apprised.

The district ran two rounds of testing, one in March and the other in early May. Both the iPad and Chrome apps were “clearly” tweaked for stabilization between the first round and the second round. The first iteration of the Chromebook app hit a roadblock due to firewall conflicts in the first round but in the second round, Villano said, “We saw some improvements in the way that Chromebooks functioned with the data being synced back to Pearson.”

At Oxford, multiple problems surfaced with software updates and computers going to sleep, based on settings primarily tweaked by the students, who had control of the computers as part of the school's 1-to-1 program. McCartney said, “Apple wants to update. Google wants to update. Safari wants to update. Every time one of those events happened, a student would be booted out of the testing [because] it would think they were cheating.” The district responded by beefing up its image as the test went on.

McCartney explained, “The first image we sent out was exactly the way the state told us to set it up. But we had to make sure we went to a uniform image for every single kid for the testing window. We reimaged all of them so they all had the same settings when it came to screen savers and swipe pads and things like that. It took us two or three days to make sure we had that down just right.”

Get All Hands on Deck

Although teachers acted as proctors during Burlington's field testing, members of multiple departments were enlisted to provide in-school support. Villano said, “If you had an issue and something wasn't working, there was somebody in the building from IT, ed tech or the library that you could call on for help. We brought in as many team members as we possibly could.”

At Folsom Cordova Unified School District (CA), a much larger district, the IT organization wants to figure out how to give remote visibility of desktops to its team of technicians when technology-related problems surfaced during testing. The extra help will, according to public information officer Daniel Thigpen, “cut the time we’re doing troubleshooting.”

Try Out Various Scheduling Scenarios

If you're not sure which approach to scheduling will be best, consider testing different schemes at different schools. (You may need to do this anyway, if you don’t have as many computers as you have students to be assessed.)

For the live assessments at Burlington, where students all have their own iPads, the job of scheduling will be fairly straightforward and Villano anticipates no problems in getting testing done in the time provided. He's not so sure that other districts will be as lucky, particularly if they have to cycle classes through a single computer lab.

During the field tests, Oxford’s schools spread their testing windows out so that no two schools were hitting the district network at the same time. Because online assessments tend to be shorter in duration than paper-based tests, this is easier than you might imagine. For example, said McCartney, if his middle school took the math assessment for two hours in the morning, the fifth- and sixth-grade elementary school would tackle it that afternoon.

At Folsom Cordova's Cordova High School, 11th-graders simply tested in the class they're all expected to take: government. That minimized disruption to the rest of the 11th-grade classes, explained Thigpen.

Redd suggested that, when sorting out the schedule, you should leave time toward the end of the testing window to give tests to students who may have missed them in the first go-around, whether due to illness, logistics or an Internet outage on the day of testing. “We're going to keep our servers up and reliable, but that doesn't mean somebody isn't digging up the street with a backhoe and cutting the school off,” he said.

Deal With Keyboards

Both PARCC and Smarter Balanced have mandated the use of external keyboards for their tests, but some districts have discovered that they should probably be optional. Villano remarked, “The students in Burlington don’t use an external keyboard. They like the on-screen keyboard. The fact that PARCC required it was actually a bit of a challenge for the kids who weren't used to having it.” During the field test, he said, “Many of our kids disconnected the external keyboard and stayed with the on-screen keyboard. We just wanted to make sure they were using it however it was most comfortable for them.”

That preference for the touchscreen also dominated at Folsom Cordova. “We certainly heard from teachers that the kids need some more experience with keyboarding, particularly on the smaller keyboards of the Chromebooks,” noted Thigpen. “Some of our younger students in particular are used to touchscreens. What we’re looking at is really trying to get students more exposure with the technology, making sure that work is done on Chromebooks more frequently. If typing is a major requirement in the assessment, [we want to make sure] we’re providing support to our classroom staff and teachers so they can make typing more routine classroom work; so it's not just for the purposes of the assessments, but it's really helping build that skill.”

Oxford's McCartney found that a bigger problem was simply getting kids accustomed to using keyboards for writing instead of “doing it with paper and pencil and with ‘pretty writing.’ We’re still working through those things,” he said.

Practice the Sample Tests

At least for a while, students will need help finding their way around the online assessments. Sample tests provided by both PARCC and Smarter Balanced can give them the introduction they need. Folsom Cordova’s Thigpen said, “The performance tasks were certainly a new element, and that was probably the biggest change we saw from [California’s state] testing. Part of that was building understanding around which tools the student can use during the assessment, such as dragging and dropping and drawing lines.”

To help students get comfortable, Thigpen said, “We wanted all of our sites to have the time to make multiple practice tests before the field tests, but the reality was, we were working under an extremely truncated timeline. If a site or class did not have the time to do multiple practice tests, we [saw] more questions about which tools to use. It was difficult to figure out what teachers could tell students to do. For instance, the teacher couldn’t tell them necessarily how to draw a line, but they could tell where in the test to click the link that explains to the students how to use that tool.”

DC’s Aguirre pushed “as many people [as possible] to participate in the field tests because we know the general hesitancy in moving from a paper-based test to a computer-based test.” The sample tests can play a similar role. “This is all about ironing out the kinks and highlighting all the challenges that we know are out there, so we could do it well in advance of the real test.”

Put Your Communications Experts to Work

At Folsom Cordova, Thigpen is responsible for communicating internally with teachers and support staff “about the tools that we have to support them and what exactly is changing.” He also handles communications with families. “We have a responsibility to help them understand how they can help their student succeed and how we are helping their student prepare. What is the objective for the new assessments? What will it require of your child? How will it change classroom instruction? And what tools can we provide to help students succeed outside of the classroom?”

At a previous district where Thigpen worked, a communications team put together Web resources explaining the Common Core and the new assessments, including, he said, “videos, tutorials, sample tutorials, frequently asked question sections — really trying to get the word out and trying to build understanding around this large-scale change.”

Although Smarter Balanced and PARCC have robust informational websites, the amount of content they make available can be overwhelming. To help highlight key points, DC is putting out weekly information in condensed forms to its local education agencies. Aguirre said, “We do PARCC FAQ sheets, so they can share with their communities. And we make it short and simple, so they can hand it to parents and community members.”

It’s Showtime!

None of the challenges that came up during field testing were “outside the realm of what you’d expect in such a large-scale shift in the testing environment,” said Thigpen, adding that the trial run “met its objective of being a learning experience…. Making sure we apply the lessons we learned during the field test will ensure a smooth transition into the real assessment this year.”

Once all the planning is done, the infrastructure is in place and the rehearsals take place, IT and instructional technology leaders can play another vital role: that of confidence builder. “The biggest thing for us was to show people that it wasn’t going to be that stressful,” Burlington's Villano said. “Sometimes technology doesn’t work, and sometimes it can stress people out. For us it was [all about] making clear to people that this was going to be okay.”