Building a School District's Wide Area Network
##AUTHORSPLIT##<--->
Bismarck Public School District is an 11,000 student K-12 school district in south central North Dakota. We have three senior high schools, three junior highs and 17 elementary schools, plus assorted support and administrative sites. With the exception of terminals and PCs connected to an administrative IBM AS/400 via dedicated phone lines, every school was an island unto itself. We wanted to change that. This article covers our journey from isolation to the Information Superhighway.
WAN Design Goals & Overview
Our goals for the Wide Area Network are:
- To give access to the community/ school library circulation system at all schools;
- To share resources like printers and files whenever possible and practical;
- To deliver Internet-ready e-mail;
- To support Macintosh, Apple II and DOS/Windows operating systems;
- To give district staff network access from their homes;
- To have Internet access available at all district sites; and
- To increase software use in elementary schools while decreasing software costs.
All of these goals had to be met with a modest budget. Indeed, a large portion of the initial WAN project was funded out of a budget that had not specifically included routers, servers and all of the network's infrastructure components.
All sites are connected with Frame Relay packet-switched service from US West. At the time of this writing (February, 1996), 24 sites are networked in whole or in part and are connected to the Frame Relay service. We expect to bring schools online at a rate of six to eight per year, finishing the project in two to three years.
Overall Infrastructure
Because we had almost no network infrastructure in place in any of our buildings, we were not burdened with retrofitting older equipment and network hardware. Refer to Figure 1 for a diagram of our Media Center site and Figures 2 and 3 for diagrams of typical primary and secondary school sites.
The Frame Relay access link (digital phone line) at each site connects to a CSU/DSU and a Cisco Systems' 2501 router, which is programmed to route both AppleTalk and IP protocols. The AppleTalk protocol handles all the district's traffic between file servers and printers, while the IP protocol handles all e-mail and Internet traffic both inside and outside the system.
Some claim that routing more than one protocol is difficult and needs extra maintenance. We have found the opposite: that having two protocols is actually more flexible without requiring more maintenance. Router configuration is only marginally more complicated with AppleTalk and IP versus IP only.
Ethernet was selected as the network media for the system. This was an easy choice. Ethernet equipment is common, inexpensive and widely supported. We use RG58 thin coaxial cable to interconnect our hubs, routers and bridges, with Category 5 unshielded twisted pair (UTP) wire connecting offices and classrooms to the hubs.
We do not follow the network "textbook" when it comes to installing 10Base-T wiring. Ethernet on UTP wire uses only two pairs: one pair for transmit, one pair for receive. Category 5 wire is four-pair wire, thus Ethernet leaves half the wires available in the cable. We use one of those pairs for the LocalTalk network at each site.
The district has some 600 Macintosh computers and 175 DOS/Windows PCs. We have a long history with the simple, but very functional LocalTalk network. A portion of our Macs are not Ethernet capable, so we must have LocalTalk along with Ethernet. Wherever we place an Ethernet network socket, we automatically have a LocalTalk socket. Thus, a user can plug into the WAN anywhere with any Mac or with any Ethernet-capable PC. This boosts the network's usability with little extra cost.
Referring to Figure 1, our Media Center wears many hats. It houses the E-mail, FTP and Web servers, along with an AppleShare server for site use. It is the central point for dial-in access to the WAN. The Media Center is our guinea pig site; we test new equipment and configurations here before we put them into common use.
Each school site is networked as shown in Figures 2 and 3. (These are general diagrams; each school site differs due to individual needs.) The number of Ethernet hubs can be as few as two or as many as seven. Use of Ethernet bridges is limited to sites where there are multiple floors, sites with one or two computer labs on a floor, or sites with administrative offices and large staffs, such as our high schools. Each location has a common AppleShare setup.
As mentioned, we have Ethernet and LocalTalk networks back to back in each location. Apple has two software products that are ideal for this environment. The Apple Internet Router (no relation to "the" Internet) software connects Ethernet and LocalTalk networks, allowing LocalTalk Macs to access Ethernet peripherals or Ethernet Macs to access LocalTalk peripherals.
We also use the Apple IP Gateway software at each site. This program gives LocalTalk Macs, such as a Mac Plus or a Mac Classic, access to IP services like e-mail and the Internet. These programs run on the AppleShare file server at most sites. In high-traffic sites like our high schools, we use a separate Mac to run the Internet Router and IP Gateway software.
Apple Internet Mail Server is a full-featured SMTP/POP mail server for Macintosh and available free. | Implementing Library Access In 1990 a community plan was developed to combine the two local public libraries, a local college library and all the individual school district's libraries into a common circulation system. In 1993 the plan's first phase was completed with the public and college libraries going online with a common circulation system. These three library locations are linked by Frame Relay and use an IBM RS/6000 host. In the fall of 1994, the Bismarck School District made its first connections to this system by activating one high school and one junior high school library. Each school library is being equipped with dedicated computers for patrons and another dedicated computer for circulation and librarians. The number of patron-access computers depends upon the library: high schools get eight, junior highs get five to eight, and elementary schools get two to four. For the library terminals, we are re-using 8088-based IBM Model 25s, original IBM PCs and XTs, a task for which they are well suited. These run ProComm software and are connected to Lantronix and Chase I/O LAN terminal servers, devices that convert serial communications to IP communications traffic on an Ethernet network. The system is simple, reliable and flexible. Since we have an abundant supply of old PCs, we expect to be able to equip most of our school libraries before we run out. We are now adding the contents of our school libraries to the common circulation system. As we progress, the physical card catalog will disappear, as will the physical checkout card in each book. They will be replaced with a bar code on the book, a bar-coded student ID card and bar code scanners in each library. Book checkout becomes a matter of one swipe with a wand across the student ID. This has been in place in our public libaries for over a year and we expect to finish the first total transition in our largest high school's library in time for the 1997 school term. With network access from the classroom, library searches can be done from literally anywhere at anytime. Staff or students can search for books by any combination of criteria and know immediately the availability and location of the materials they seek. Sharing Resources Across Platforms It is a mistake to assume that there must be a homogeneous, one-vendor solution to the cross-platform challenge. We interpret cross platform to mean a common network that is accessible to, and usable by, both Macintosh and DOS/Windows computers, indeed any computer that can generate data packets that the network will accept. In some cases this means that the two major systems are separate but equal; in other cases this means common and equal. The goal with shared resources is to offer print and file services to users across the district. Shared resources should be available on demand, allowing a user the ability to change printers or file servers at will, without the intervention of a network administrator. Thus we chose to use the AppleTalk protocol for all of our print and file server activities, for both Macintosh and DOS-based PCs. With the Macintosh this is very easy; AppleTalk support is part of the Mac and its operating system. (AppleTalk is the network protocol developed by Apple in 1984 and runs over telephone, Ethernet or Token Ring network media.) For DOS-based PCs, it's a little trickier, but not much. In 1987 Apple developed a PC version of its AppleTalk stack. Marketed as a hardware/software combination called PhoneNet PC, it allowed PCs to connect into a LocalTalk network and share Apple laser printers and file servers. In 1991 Apple sold the software and hardware to Farallon Computing who continued development and sales. Their current version of PhoneNet PC works with all network media and it gives Windows users a Mac-like Chooser application for printer and file server connections. The software is Open Datalink Interface (ODI) compliant, which means that a single Ethernet card in the PC can simultaneously handle both the AppleTalk LAN traffic and the IP traffic for Internet and e-mail services. At the time of this writing, Apple is shipping its AppleShare for Windows client software. We are evaluating it in comparison to PhoneNET PC and have found that, with modifications, it compares quite favorably. |
Using freeware, the entire cost for our mail system has been less than $500. | Electronic Mail Initially we didn't think we could include electronic mail in the plan because of cost. With traditional mail software, mail server hardware and Internet gateway software there is a substantial investment. Delay worked to our advantage, however. We use a mail program called AIMS, the Apple Internet Mail Server. Written by Glen Anderson, a computer support specialist at the library of the University of Otago in Dunedin, New Zealand, AIMS is a full-featured SMTP/POP mail server program that runs on a Macintosh and available free from Apple. We run it on an older Mac LCII with 6MB of RAM and a 40MB hard disk. This mail server supports over 200 concurrent users and processes over 700 hits per day, with 90% of this falling between 8 a.m. and 5 p.m. With AIMS no extra server hardware or gateway software is needed for users to send or receive electronic mail anywhere in the district or anywhere in the world. We also run Timbuktu Pro network software on the machine as its remote-access features enable easy access to mail account maintenance while in the field. Because we have Mac, DOS and Windows computers, we use four different mail client programs. Eudora Lite from Qualcomm is used on Macs with System 6, System 7 and on Windows PCs. Minuet, from the University of Minnesota, is on DOS PCs. In our student computer labs (both Mac and Windows) we are deploying POP Mail Lab, also from the University of Minnesota. This client has the unique feature of being non-user specific. In each case, the software is freeware and so the entire cost for our mail system has been less than $500 -- all of that for the Mac LCII. Any district staff member can request e-mail if their building is connected to the WAN. Some elect not to use it; others find it of significant value. Though at first glance it would seem that e-mail is a very simple activity, we have found that training is important for widespread effective use. Many users have a hard time conceptualizing the ideas of mail host, mail client and mail flow. Dial-in Access We use Tribe's TribeLink8 remote networking server for dial-in access to the WAN. The unit supports eight phone lines and uses PPP (Point to Point) protocol for both IP and AppleTalk network communications. Dial-in access is limited to Macintosh or Windows (not DOS) computers only. The TribeLink8 includes Macintosh PPP software and we run Trumpet Winsock (shareware) on the Windows PCs. After extended pre-purchase testing, we determined that low-cost 14.4 modems performed very close to the $200 name-brand products. The difference in throughput was less than 10%, small enough to make us go the low road on modem purchases. We do not have dedicated lines for dial-in access. There are eight phone lines into the Media Center building. During the day they handle normal voice and fax traffic; from 5 p.m. to 7 a.m. they are available for dial-in access. Rather than turn the modems off and on every day, we installed phone/modem switches on each line. The dial-in user appends a two-digit code onto the end of the phone number called. The switch sees this code and sends the incoming call to the modem instead of the telephone. Cost is small, only $75 per line. Don't try this unless you have a location with clearly defined blocks of time when the phone lines are not used. In most cases, this means having dedicated phone lines for dial-in access. Internet Access Internet access proved to be the easiest part of the WAN. The Information Services Division of the state of North Dakota is our local Internet access provider. We have one 56K Frame Relay connection (soon to be upgraded to a T1) that carries all of our Internet traffic. Our cost is $200 per month for the connection. Initially, there was some concern whether this would be adequate for our needs. For the first ten months of operation it was. However now that we have so many sites networked, our 56K Internet link is overloaded. In the course of 15 months, we went from 20 Internet-ready computers to over 400, and growing. The move to a T1 Internet connection is necessary for us. Many small schools however, should be able to function just fine with a 56K link. Adhering to Software Licenses In the elementary schools, demand for software always exceeds the budget. In most cases, it isn't financially possible to purchase enough copies or licenses of different applications to load all of a school's computers. And even if the budget was there, the waste would be significant as at any moment in time only a few of those copies or licenses would be in use. Thus the need for license control. KeyServer, from Sassafras Software, controls the usage of software in our networked elementary schools. KeyServer allows us to buy a program and copy it to every computer in a building, yet still be legal. KeyServer works by modifying a program with a software ‘key.' When a user launches a keyed application, KeyServer compares the number of keys allocated for that program to the number of keys already in use. If usage is less than the limit, the program is allowed to run. If usage has reached the limit, the user is told that all keys are checked out and the program will not run. Keyed software won't run if the computer is disconnected from the network. In a typical installation we purchase a five pack so KeyServer is configured to allocate five keys. KeyServer also logs usage and delivers extensive reports that can help decide if more copies of a program are needed or if there are programs not being used at all. |
Internet access proved to be the easiest part of the WAN. | Shareware/Freeware in the Mix We actively seek out and test shareware and freeware software for inclusion in our software mix. There is just too much high quality shareware/freeware to be ignored. Common objections to this source for software are lack of support and erratic updates. Yet these same charges can be levied at commercial software. Technical support from many commercial vendors is increasingly difficult to obtain and often only at additional cost. Upgrading commercial programs is becoming a nightmare, with too many upgrades as bug-laden as their predecessors or nearly as costly. Our experience with shareware/freeware has been excellent, often superior to commercial products. Yes, there have been problems, but in most every case, the developer has been quick to respond to e-mail requests for help. And where a problem could not be fixed, the developer was not afraid to admit defeat and withdraw. Some shareware/freeware products are also offered in commercial editions (such as Qualcomm's Eudora Pro) that have additional features and technical support that may be of value. In any case, use the shareware/freeware versions for testing to save time and money -- "try before you buy." With the direct Internet access that our WAN gives us, looking for new shareware/ freeware to test or obtaining updates for existing versions is only a matter of a few clicks of the mouse. Unforeseen Problems & Benefits We were surprised at the minimal level of technical support supplied by primary vendors. They were prepared to answer questions and offer solutions only if asked the right questions and if we accepted their solutions without question. Many vendors apparently don't believe that a school system is capable of conceiving, organizing and executing a project this technically complex. A project of this sort is not something that you can hand over completely to an outside vendor. It is essential that you have, or develop, in-house expertise to handle the design, installation and long-term maintenance. We have networked Apple IIgs labs in five of our elementary schools. In addition, we have dozens of IIgs machines around the district. In the labs, the machines boot from a local AppleShare server that is loaded with MECC software for elementary grades. We had never thought about accessing these servers from outside of the labs until one teacher innocently asked if it was possible to access the lab server from his classroom. The answer had to be "why not?" We now have Apple IIgs computers in some buildings booting from servers across town. The ProDOS (the Apple II's operating system) programs from MECC are small in size and easily moved across the 56K access links of the Frame Relay system. One elementary school has a Macintosh equipped with an Apple IIe co-processor card in every classroom. These are able to boot from the MECC server in the local IIgs lab, giving the classroom teacher access to a very wide range of software -- from simple Apple IIe drill-and-practice programs to state-of-the-art Macintosh multimedia products. Monitoring the WAN Monitoring of the network proved to be a much simpler task than originally thought. Expensive network monitoring tools have been unnecessary. AppleTalk is a very informative protocol and the Mac OS's Chooser application will quickly show if something is wrong, if a server has dropped off line or if a zone is unavailable because of a router problem. We use two tools, InterPoll, an AppleTalk network tool, and MacPING, for general AppleTalk and IP protocol testing. Apple's MacTCP PING, included with MacTCP, is also very useful when checking IP protocols. A Telnet application like NCSA Telnet is all that is needed to check on and configure routers. More of our old IBM PCs are used as terminals directly connected to the console ports of the Cisco routers for making quick checks of router operation while at a site. It is difficult to justify the cost of solutions such as SNMP hardware and tools. Nearly all of the network problems we've encountered to date couldn't be fixed remotely. Such things as a router that lost power and didn't reset because of a failed circuit breaker, an Ethernet hub that died, a server that locked up, a network outlet that got torn off the wall by an errant vacuum cleaner, all require an actual visit to a site. In conclusion, we are very satisfied with our Wide Area Network. In contrast to an environment that traditionally has underutilized technology, our network is getting used -- so much so that we are looking at upgrades to handle rapidly growing traffic levels. It is outperforming our expectations and we look forward to getting all of our schools online as fast as possible. n Vern Mastel is the computer technician for the Bismarck Public School District. In addition to the design and installation of the district's WAN, he also installs and maintains the district's computers. He has had over 50 articles on electronic design, programming and software usage published in various magazines and journals since 1978. E-mail: [email protected] Products or companies mentioned in this article:
Apple Internet Router, Apple IP Gateway, AppleShare for Windows client, MacTCP, MacTCP PING; Apple Computer, Inc., distributed by Claris Corp., Santa Clara, CA, (800) 747-7483 (800 # for educators only), www.claris.com
Apple Internet Mail Server (AIMS); Free download (<200K) from www.solutions.apple.com/aims/default.html documentation, support programs, list-serve at: www.freedonia.com/ism/mail/mail.html
Chase I/O LAN terminal servers; Chase Research, Inc., Nashville, TN, (800) 242-7387
Cisco Model 2501 router; Cisco Systems, Inc., San Jose, CA, (800) 553-6387, www.cisco.com
Eudora Lite, Eudora Pro; Qualcomm, Inc., San Diego, CA, (800) 238-3672, www.qualcomm.com
IBM RS/6000, IBM AS/400; IBM Corp., Armonk, NY, (800) 426-3333, www.ibm.com
KeyServer; Sassafras Software, Hanover, NH, (603) 643-3351
Lantronix, I/O LAN terminal servers; Lantronix, Irvine, CA, (800) 422-7015, www.lantronix.com
InterPoll; Dartmouth College, Hanover, NH;
MacPING; True Basic, West Lebanon, NH, (800) 436-2111
MECC; Minneapolis, MN, (800) 685-MECC
Minuet, POP Mail Lab; University of Minnesota, anonymous FTP to: boombox.micro.umn.edu
PhoneNet, Timbuktu Pro; Farallon Computing, Inc., Alameda, CA, (800) 995-7761, www.farallon.com
ProComm; Datastorm Technologies, Inc., Columbia, MO, (800) 315-3282
TribeLink8; Tribe Computer Works, Alameda, CA (800) 77-TRIBE, www.tribe.com
|