Making a Comeback
##AUTHORSPLIT##<--->
Lacking the resources companies have to protect their datain the event of disruptions, K-12 technology directors aredevising their own backup solutions.
HE'S GOT THEIR BACK Tech
directorMark Finstrom set up a
disaster recoverysite that can retrieve
his district's criticaldata systems if a
calamity strikes.
YOU'RE AN IT ADMINISTRATOR. When it comes to disasterrecovery planning, you know the drill. Back up everything.Store records in a separate location. Install redundant systems.Trouble is, you're in charge of a K-12 school district, and you haveneither the money nor the staff to invest in and maintain state-ofthe-art data recovery systems, like those your counterparts in theprivate sector are blessed with.
As technology grows more pervasive in education, the data recovery issues that school systems encounter are becoming increasingly problematic, as districts work to accommodate a range of concerns that don't encumber commercial businesses. State and federal laws often require schools to report on student populations and educational progress according to mandated deadlines-- meaning that even if your school district has suffered a disaster, you'll somehow need to recover the data so you can file the necessary reports. You'll also contend with strict privacy regulations that require student data to be kept confidential-- meaning that even if you can't access the data, you'll need to make sure it doesn't fall into the wrong hands. And as remote learning becomes more and more commonplace, teachers and students need access to technology 24/7 and in widely dispersed locations-- meaning that you'll have to keep online access up and running even if the main school site is a disaster area.
"The issue is that you have to protect data and make it accessible at the same time," says Stephen Harris, a commercial building construction expert and principal of Harris & Harris Consulting, a logistics consulting firm based in Lincoln, VT. "Hospitals, for example, want their information kept in a different area of the country and in a different portion of the electrical grid so they're less vulnerable to blackouts. [That means] setting up mirror capacity in a different location. But it's expensive. A data center uses 350 watts of electricity per square foot, compared to 5 or 6 watts for a normal office building. School districts aren't going to be able to afford that kind of thing."
Work Around It
Pinched by tight budgets, many district IT directors have come up with imaginative workarounds that operate within the various mandates and limitations placed on K-12 information infrastructure.
For Mark Finstrom, director of technology services for Highline Public Schools in Seattle, disaster recovery planning is a matter of juggling budget restrictions with that unique set of data protection demands. "We have more data protection concerns than a business has," he says. "We can't release videos of students. We have privacy and family protection laws to follow-- we're very similar to hospitals in that respect." At the same time, Finstrom doesn't have the financial resources that a commercial business usually has. State funding for schools often does not keep pace with costs; public school districts in the state of Washington rely on local bonds and levies to make up for shortfalls.
"Our funding is based on state dollars and some bond or levy money," says Finstrom. "We're funded under [Washington's] Basic Education Act. The amount changes every year because it's based on the number of students we have in the district, which is currently 17,500. I get an allocation based upon BEA dollars. I have to plan with that in mind. I have to walk into a school year knowing I have a regular budget plus the bond or levy money, knowing what projects I have to accomplish and choosing which ones to drop if I have to."
An Insider's Tips for Good Data Defense
NOT EVEN THE SMALLEST of organizations can afford to ignore the threatposed by data disruptions. At the same time, budgetary constraints can force ITdirectors to cut corners. But according to Carl Walker, a product manager for Eaton, a global power management and electrical systemscompany, it's possible to provide a reasonable amount of IT systems protectioneven on a tight budget.
In a report he produced for Eaton,Walker points out that only a very small percentage of IT data loss results from natural disasters or computer viruses. He writes that "the most destructive influences on data centers actually come from much more mundane causes: software error (14 percent), human error (32 percent), and hardware failure (44 percent), frequently triggered by power problems, including power failure, power surges, brownouts, line noise, high voltage, frequency variation, switching transients, and harmonic distortion."
Walker offers several easy-to-follow tactics for safeguarding your data:
Treat any IT equipment location as a data center. Even if you don't have a dedicated data room, all the necessary components for IT power protection, cooling, security, and so on must be housed on one or more racks or in cabinets. Try to invest in enclosures with locking entries, a well-organized arrangement of cables, and a monitoring/management system.
Look beyond generators and surge suppressors. Public utilities are not required to provide computer-grade power-- and they don't. IT equipment is damaged by invisible events such as sags, surges, and spikes. Uninterruptible power supplies (UPSs) are the best form of protection; they come in a range of configurations and prices.
Protect IT equipment from overheating. Sixty percent of hardware downtime is heat-related,Walker says. The most cost-effective solution, in his view, is to use door-mounted fans for the equipment enclosure.
Restrict access to IT equipment. Enclosures provide access-control options such as lock-and-key, electronic control, radio frequency identification (RFID) local readers, and access cards. You can devise an access-control strategy to match the level of security you need.
Manage cables for efficiency and airflow. With IT devices smaller than ever-- often served by dual or triple power supplies-- a single rack of equipment might produce 40 or more power cords to manage, plus associated service cables and patch cords.
Poor management of all this wiring can lead to damaged cables, performance degradation, and costly downtime. Keeping cables out of the way of airflow also maximizes the cooling ability of the enclosure.
Watch for environmental hazards. The performance and life span of IT equipment can be compromised if the environment it resides in is dirty, damp, hot, or insecure, says Walker. How hot is it inside the enclosure? Did a technician leave the door open? Is condensation putting equipment at risk? Did a blown capacitor spill acid? Walker advises investing in a UPS that works with an electronic probe, a plug-in device that can be used with a standard web browser to monitor the temperature and humidity of the environment where the UPS is located.
Complicating issues for Finstrom, the Highline school district is located on a fault, so it's vulnerable to earthquakes. What's more, it's in a heavily trafficked part of the state. "Disaster planning is escalating for us," says Finstrom. "We surround Sea-Tac airport. Planes could crash here, or a school could be used as a disaster recovery site. We have reciprocal agreements with local police, fire departments, and medical centers, and we run drills at our schools. In terms of technology, I'm at the point now where if a Virginia Tech or Columbine were to happen, I can release videos to police and firefighters so they can see our cameras as they go to the site."
Getting to that point, however, took meticulous planning and determination-- and, to some extent, luck. In 2006, the state of Washington passed a levy that provided funding for a number of public school projects. Best of all, it allowed for a new high school that Finstrom set up as his disaster recovery site, with the IT networks constructed to his specifications. The site is on separate phone and power grids and can recover all student information and payroll systems if the main site experiences a disaster. Equipped with its own power generator, the new high school features uninterruptible power supply systems, a backup communications system, voice over IP capabilities alongside standard copper phone lines, and internet access through a separate cable connection as well as through Private Line Transport GeoMax, a high-speed data-networking solution from Qwest. And because the school, like all public K-12 institutions, is subject to privacy regulations such as the Children's Internet Protection Act and the Children's Online Privacy Protection Act, online filters are also installed on school computers, so compliance can be maintained even in an emergency.
Throughout the installation process, Finstrom had to stay aware that he was operating in a cash-strapped district. "Over half of our district is on a free or reduced-cost lunch program," he says. "We have several buildings that are 90 percent freeand- reduced."
As a result, those schools get heavy discounts on telecommunications and internet services through the Federal Communications Commission's E-Rate program, and Finstrom has been able to use a mix of applications from Qwest, Sprint Nextel, Comcast, and other providers. "We bring in vendors that have saturated our area, because of reduced cost and more services," he says.
Finstrom also has to keep distance learning requirements in mind. "Disasters can be manmade or biological. For example, during the bird flu scare last year, the state wanted us to draw up a disaster recovery plan. If there was even one case, they'd actually have closed the school for six weeks. But we'd still have needed to provide education to the students while they were staying home. So we're trying to provide computers to some of our students, as well as working with Qwest to provide broadband DSL lines."
"We have more data protection concerns than a businesshas.We can't release videos of students. We haveprivacy and family protection laws to follow-- we're very similar to hospitals in that respect."
Common sense plays a big role in Finstrom's disaster recovery protocol: One way to ensure backup is to connect with "peer schools that run the same system," he says, citing a neighboring school district with a recently remodeled data center that he can turn to in a pinch. He adds, "If you have a good indexing policy for your data, you can recover the information that you need."
For example, Highline Public Schools has a 50,000-square-foot building that stores records dating back to the early 1940s. "You can imagine the condition of 1940s paper," Finstrom says. "So you bring in a company to review it and see if you can archive it electronically and keep it in a readable format, but also index it. In the past year, we have begun to build that schema, and eventually we will move completely to an electronic process. We won't have paper copy anymore."
All this, Finstrom emphasizes, is pretty impressive for a school district that had no backup systems at all when he joined it three years ago-- and that still has a lean IT department. He cross-trains every member of his staff of 20 people in disaster recovery procedures so anyone can fill in for missing staff members if necessary. "If you lose even one of your staff, say in an accident," he says, "you lose a lot of knowledge."
A Common Storage System
Nearly 2,500 miles away, other school systems face similar challenges but have handled them somewhat differently. About three years ago, school districts in Ohio needed a common data storage system that would also ensure full data recovery in case of a disaster. A network of 23 Information Technology Centers represents the state's 725 school districts, as well as its Educational Service Centers, community schools, and about 1.3 million students. The ITCs provide technology services to all the school districts across the state.
The task of setting up the data recovery system fell to the Management Council of the Ohio Education Computer Network (MCOECN), which oversees the 23 ITCs. After much consideration, the council decided to invest in StorServer's Data Protection Appliances, which provide disk-to-disk and diskto- tape configurations and are platform-neutral. Following the construction of a new disaster recovery site in the Columbus, OH, area earlier this year, most of the ITCs are now using the appliances, which perform backups for 600 school servers, sending them to the new site every night.
The project has been quite a feat for MCOECN to pull off, says its chief operating officer, Duane Baker. "Lack of financial resources and staffing resources are two of the really big issues we face," he says. "But we're also responsible for making sure that student data is safe at all times. We're in a scenario where everything has to be encrypted all the time."
Keeping IT Backup In-House
GIVEN STAFF CONSTRAINTS and the increasing complexity of IT systems, it might seemlogical for schools to follow the route gaining favor in the private sector and farm out ITbackup entirely to a third-party service provider. Money, predictably, is the main barrier.
"It's not a cheap thing," says Chris Loeffler, product manager at Eaton, a global power management and electrical systems company, and previously a member of the IT staff for the Flagstaff Unified School District (AZ). Plus, he says, in many cases schools can make do with legacy equipment and retrofits. "In reality, when you look at IT equipment, it has a long life span-- 10 to 20 years in some cases. In the K-12 environment, speed is not critical. You can manage with a five- to seven-year refresh. Data recovery is not a recurring expense, and recordkeeping, payroll, and curriculum systems don't need top-notch equipment to run on."
Still, Loeffler says, schools need certain basics, such as surge protection devices for a building's entire electrical and telecommunications systems, uninterruptible power supplies, and some form of offsite storage of data. The key, he says, is to keep the backup data in a geographically different area. "Don't put everything in one location."
A series of disruptions, including a blackout in August 2003 that downed several district data centers, brought home the importance of data recovery, Baker says, adding that proper planning for such disasters "has long been ignored in schools and government agencies and medium-sized businesses." Funding for the project came from the ITCs, which get their resources from the school districts and the Ohio Department of Education.
"You can always use more money, especially in the K-12 environment," Baker says. "But we wanted to provide valuable resources and put schools in a position that was not so vulnerable to disaster. When we started the initiative, we had not pulled together the funding for the hot site. In the past, if one ITC was having a disaster, we tried to move applications to a nearby ITC and get them operational. The concept behind using StorServer was to get all our applications in the same format and interchange them easily across ITCs. A statewide implementation of a storage appliance has been unheard of until now. It's a real blessing that we now have a common archival center."
The council selected StorServer because it was one of the few solutions that could handle data from so many different operating systems, which meant less training for IT staff members. The simpler the system, the better, Baker says. "We have learned that in a disaster it may not be just the hardware and network connections that are disrupted-- you may not have access to your people. If their home has been destroyed, they need to be with their family, not with their employer.
"The one thing I would encourage school districts to do is to use standardized technology and try to keep their process simple. If you have standardized technology, you can utilize resources from anywhere-- anyone can follow a disaster recovery playbook, so to speak, and restore operations."
::WEBEXTRAS ::
If you would like more information on disasterrecovery planning, visit www.thejournal.com. Enterthe keywords disaster recovery.
Rama Ramaswami is a freelance writer based in Wilton, CT.
This article originally appeared in the 09/01/2008 issue of THE Journal.