Friday, May 9, 2008
Spencer, a legendary Unix hacker -- a species not exactly known for humility -- is pleasantly understated about his role as Usenet's great early archivist. He's the first to point out that he wasn't really the only one saving those early messages. But the copies he kept of Usenet postings from 1981 to 1991 appear to be the only ones that still exist. "There were several other people who were archiving stuff, but all of them gave up before we did, and as far as I know none of their archiving survived," he says. For instance, legend has it that two guys at Bell Labs kept back-ups as well, but their stores of these ultra-rare posts are nowhere to be found.
"I'm very glad the stuff is finally out there, and I can stop worrying about how the only copy might get lost," Spencer says, now that Google has assured the preservation of the more than 2 million old messages he saved. "I'm just glad that this particular great mass of data is no longer my worry."
One of the early adopters of the computer language C, Spencer is known for his Ten Commandments for C Programmers, as well as for being the coauthor of C News, one of the early programs for transferring and reading Usenet messages.
Now 46 years old, he works as an independent consultant, but back in 1981 he ran the computer facility at the University of Toronto's zoology department. While the geeks over in the university's computer science department were busy with the Arpanet, the Department of Defense's system was too expensive for the zoologists.
"The zoology department may sound like a funny place for pioneering networking work," says Spencer. "But the computer science department wasn't very interested in this inferior networking. It was very low-tech by their standards. But it worked and theirs didn't. Their opinion changed fast when we started providing e-mail."
That's how, in the spring of 1981, with a 300 baud modem, the zoology department at the University of Toronto became a central distribution point for Usenet, when the network was just 2 years old.
Traffic was almost unimaginably lighter in those days. Only about 200 people had access to Usenet: "In the first few years, it was at least plausible to come in in the morning and read all the Usenet traffic that had come in, and 15 minutes later be off doing something useful," remembers Spencer. But even that low level of traffic was too much for the storage requirements of the day. "Pretty soon, it was necessary to think about expiring old stuff," he says.
It wasn't a sense of historical importance that initially led Spencer to think about creating an archive. His motivation was much more pragmatic than that: Most of the conversations on Usenet at the time were very technical, and he was reluctant to see the information in them disappear, because it might be useful to the university's geeks: "A lot of the early traffic was about things like Unix systems bugs, and it seemed unwise to just throw it out."
So the archiving began with 40 megabytes filling up a new mag tape -- each reel one-half inch thick and 10 inches in diameter -- every few months. In this era, messages from the outside world came in at the tortoise rate of 300 baud. ("When we got a 1,200 baud auto-dialing modem, that was just wonderful. Twelve-hundred baud was just total luxury," Spencer recalls.) As Usenet grew, this meant that Spencer and his system administrators had to be selective about which newsgroups they received and archived, keeping technical conversations but throwing away some of the more general discussions that generated a lot of traffic.
"We started dumping stuff that we thought was obviously of no future use, groups that specialized in a lot of talk and no substance, so to speak. For example, fairly early on there was a newsgroup about abortion which specialized in violent arguments."
That's why not only the very earliest Usenet posts, before Spencer started archiving in 1981 (Usenet began in 1979) but even some of the posts in the 1980s are still lost. It's too bad; today, wouldn't more of us rather see what was being said about abortion in 1984 than sift through the arcana of bug fixes in systems that have probably been long since retired? "It was perfectly reasonable from the viewpoint of stuff that we might want to use again, but a little sad from today's viewpoint," Spencer admits.
For 10 years, the nine-track mag tapes piled up, hanging in a huge rack at the zoology department's computer facility. Finally, in the early '90s, with the growth of Usenet outpacing the zoology department's budget for $15-a-pop tapes, the general archiving project ended.
Get Salon in your mailbox!