The trouble with storing data
By Drew Sullivan, NICAR Staff
It's always difficult to decide what to do with all that great government data you've worked so hard to collect. If only we all had a 500GB hard drive. But a storage plan can be devised using new technologies that make it possible to store data safely and efficiently while still providing quick access to the most needed data. This involves deciding between magnetic, optical, magneto-optical and chip-based storage media. What's best for your organization depends on what trade-offs you are willing to make between money, space and time.
Government agencies still store most data on 9-track tapes or 3480 tape carts. Tapes are inexpensive relative to their storage capacity (generally about 200 MB), but they can present some serious problems, especially for smaller news operations.
For example, tape are sensitive to temperature, magnetic fields and other environmental phenomena.
Tapes also are prone to "read" errors. If the machine that wrote the data has misaligned heads, your tape machine might not read it. Everyone who has used a 9-track machine is familiar with the problem of the tape spinning back and forth over one damaged or poorly written spot.
A new form of tape cartridge on the market called the Digital Audio Tape (DAT) elimates many of the problems of 9-tracks and carts. A DAT is about the size of a standard audio cassette, costs a little more than an 9-track tape and can hold 8 GB or more of data.
Because DAT cartridges use a digital rather than analog storage, the alignment of heads isn't a problem. But DATs suffer from some of the same disadvantages of other tapes. The data is accessed serially. They are unstable, so copies should be made every couple of years. Also, damaging a small part of the tape will destroy large amounts of data.
Nevertheless, DAT's offer inexpensive storage for massive amounts of data. And if you don't mind slow access to your data, they make a good archival medium. DATs, at between $10 and $25 each, are cheap enough to make extra copies for offsite storage. DAT drives are available for about $2,500.
Magneto-optical (MO) drives, a somewhat more exotic storage medium, show great promise for data storage. The drives use a rewriteable glass disk that resembles a large compact disk (CD). For about $130 these disks can store 1.2 GB of data. Although MO disks are more expensive than tapes, they can be accessed almost as quickly as a hard drive (19 milliseconds). An MO acts almost like a removeable hard drive. Data is not read serially, meaning the computer goes straight to the data you want.
Data on an MO disk is virtually uncorruptable and the disks themselves are hard to damage if handled correctly. Also available are WORM (write once read many) disks. Once written, data on WORM disks cannot be changed. This makes them useful for network systems where users can access data without the ability to alter or damage it.
Cost remains the biggest disadvantage of MOs. Drives sell for about $3,000 and the disks are expensive for long-term data storage. If you can afford it, it's the way to go.
The tapes themselves are large and difficult to store. If you don't have money for a climate-controlled room or even a large, safe storage area, 9-tracks constitute an inefficient long-term storage option.
Writeable CDs also have gained ground as a viable medium for archiving data. Although they are not rewriteable, CDs are inexpensive (about $22), hold a lot of data (680 MB) and are virtually uncorruptable. With triple and quadruple speed CD drives on the market, access time has increased nearly to that of MOs.
In addition to space considerations, retrieving data from tapes is time-consuming. Computers access tape-stored data "serially." If you envision your drive like a book, serial retrieval means the computer would have to read every page prior to the page you are interested in before it can access that data. Even with a good 9-track reader, you might have to wait 20 to 40 minutes to read the data you want.
Once a "master" has been created using a writeable CD drive, copies can be made for about $5 each, depending on the number needed. Still, CD writer drives are expensive ($3,500 - $5,800) but they are more cost-effective than they were a few years ago. CD "jukeboxes," which operate much like their musical counterparts, allow ten or more CD's to be online at any one time providing an exceptional network data library.
Flopticals are 3.5-inch floppies but they hold 21 MB of data on an optical disk. Access time is similar to that of a floppy. The drives are inexpensive (less than $350) but the disks are still pricey at $21.
In deciding among the above options, most organizations should consider the levels of storage needed. Frequently accessed data should be stored on hard drives, MOs or CDs. DATs or 9-track tapes are good for off-premises archival storage. By combining the two, a news outlet can continue to make use of the data they've collected and still have room to work.
Have you had some good experiences with the hardware discussed here? Had some nightmares? Share your experiences by calling, e-mailing or writing us at NICAR.