Sunday, Sep 7, 2008
Kan's optimism, however, is not matched by researchers who have released critical reports in the last month. A recent study by Eytan Adar and Bernardo Huberman of Xerox PARC examined the traffic on Gnutella and discovered that there were an awful lot of "free riders," users who were happy to download files but weren't willing to share their own. In fact, 70 percent of all Gnutella users don't share files, and of the 30 percent who do, the top 1 percent share 40 percent of all files. During a 24-hour period, Adar and Huberman observed 31,395 hosts -- but of that group, 314 hosts were serving the majority of the files.
It's the tragedy of the commons, writ for the digital age: Shared resources are being gobbled by users who get more than they give.
"There's very little reward for you sharing your files, and there's a high cost," explains Adar. "You're anonymous so you can't get credited for doing what you're doing -- no one says 'good job' for doing this. And it's a high cost, because [due to bandwidth limitations] even if you're on a DSL connection you can't do other things that you want to do. People realize that they come out on the negative side and don't want to share files."
The most obvious drawback of free riding is that if only a tiny percentage of users are sharing, it will be harder to find the files you want. Kan shrugs: "The providers of goods are always fewer than the consumers of goods; even if the study were true it's not such a terrible thing. Even if all the Starbucks closed, you'd still find a morning cup of coffee. Even if those people who share files are shut down, others would take their place. Gnutella didn't get here because nobody shared."
But free riding is more problematic than Kan will admit. Free riding also means that Gnutella won't scale: Each search query plugged into Gnutella has a certain "time to live," and will expire after it has queried a certain number of hosts. If the network continues to grow, and no one is providing any files, your query will hit its expiration date before it arrives at a useful host. The legal implications are also worrisome; although it's widely believed that the decentralized nature of Gnutella will make prosecuting users impossible, it's much easier to target scapegoats if only a few hundred hosts are actively dispensing files.
Adar hasn't written off Gnutella. He believes a more successful and scalable system is possible, but would require trade-offs such as decreased privacy and a few centralized network machines. Future versions of Gnutella could also include a system "default" that forces all users to share, much like Napster. Otherwise, Adar believes, Gnutella won't be able to hold up under the strain if it is flooded by Napster-like traffic.
Earlier this month, researchers from Clip2 Distributed Search Services published a report documenting another major flaw. According to the report, the Gnutella network is only as strong as the bandwidth of its weakest users -- too many people using 56K modems are being required to channel traffic, causing the entire system to slow to a crawl. The report pegs the "scalability barrier" at 10 queries per second; any more, and the network will break down.
It's not surprising that Gnutella has deep problems in its infrastructure -- after all, it was "barely in alpha" when released by Frankel. In fact, Frankel's own release notes reveal that Gnutella was designed for use by roughly only 350 "nodes" (or hosts); in a widely distributed e-mail attachment he posited that Gnutella wouldn't be scalable past 5,000 users. It's incredible that the software has survived the wear and tear of hundreds of thousands of users to this point. But it's also looking increasingly doubtful that it will be able to handle much more traffic; the system is already slowing to a crawl, and Gnutella's more devoted developers are worried about what is to come.
Eighteen-year-old Gnutella developer Sebastian Lambla of Paris is concerned. "Gnutella is not really adapted to a large number of hosts, and is not at all adapted [to] a context with spammers and flooders on the network," worries Lambla. "Free riders and slow links are two of the biggest problems of the current protocol."
Lambla has some suggestions for improving Gnutella, and he and other developers are attempting to act upon them. But because Gnutella is not owned by any single entity, the process is moving slowly -- especially compared to the increasing speed with which the recording industry is finally beginning to respond to threats. Gnutella's developers are not a cohesive group. They are scattered all across the globe, and range from teenage hackers to experienced programmers. But some are trying to form virtual networks to work on next-generation protocols and software development. Still, many programmers are prickly about letting anyone "take over" Gnutella; and when enterprising developers have attempted to spearhead the development of the client and protocol the community's reaction has been decidedly mixed.
"The general agreement is that the future designs should be by committee; there should be agreement. It doesn't help to have the babble that's initiated by independent development; it doesn't help anyone to have a whole bunch of separate systems that don't communicate with each other," Kan says. "It's a challenge, however, to coordinate the efforts of everybody; particularly because there are very, very few commercial enterprises that are devoted to Gnutella, and so it's impossible to get everyone into the same room so that they can have interaction."
As a result, there are initiatives such as gPulp -- spearheaded by Lambla -- which hopes to draw up next-generation protocols for Gnutella that will expand the system's distributed search system beyond simple file-sharing capabilities. But the gPulp group, in turn, is competing with Intel's P2P Working Group, which is also trying to develop standard protocols that will work for all P2P software products. And then there's the Gnutelladev Working Group, set up to provide services for developers who want to create their own clients.
"It's difficult to make any arrangement to get together," says Shaun Sidwall, the Canadian programmer who assembled the Gnotella client. "There's definitely some competitiveness, though; everyone would like to have the best clone out there, or the No. 1 client. Everyone likes to be No. 1, of course."
This is not unfamiliar territory -- the open-source community has been successfully tackling the issue of cooperation for years, and Linux and the Apache Web server are both excellent examples of how virtual programmers can work together toward a common end. Gnutella may well fix its technical problems in the coming months; but if Napster is shut down next week, such help won't have come fast enough.
Get Salon in your mailbox!