Gnutella is a peer-to-peer network protocol. Founded in 2000, it was the first decentralized peer-to-peer network of its kind, leading to other, later networks adopting the model.[1]
In June 2005, Gnutella's population was 1.81 million computers[2] increasing to over three million nodes by January 2006.[3] In late 2007, it was the most popular file-sharing network on the Internet with an estimated market share of more than 40%.[4]
History
The first client (also called Gnutella) from which the network got its name[5][6] was developed by Justin Frankel and Tom Pepper of Nullsoft in early 2000, soon after the company's acquisition by AOL. On March 14, the program was made available for download on Nullsoft's servers. The event was prematurely announced on Slashdot, and thousands downloaded the program that day.[7][8] The source code was to be released later, under the GNU General Public License (GPL); however, the original developers never got the chance to accomplish this purpose.[9]
The next day, AOL stopped the availability of the program over legal concerns and restrained Nullsoft from doing any further work on the project. This did not stop Gnutella; after a few days, the protocol had been reverse engineered, and compatible free and open source clones began to appear.[10] This parallel development of different clients by different groups remains the modus operandi of Gnutella development today.
Among the first independent Gnutella pioneers were Gene Kan and Spencer Kimball, who launched the first portal aimed to assemble the open-source community to work on Gnutella[11][12] and also developed "GNUbile", one of the first open-source (GNU-GPL) programs to implement the Gnutella protocol.[13][14][15]
The Gnutella network is a fully distributed alternative to such semi-centralized systems as FastTrack (KaZaA) and the original Napster. The initial popularity of the network was spurred on by Napster's threatened legal demise in early 2001. This growing surge in popularity revealed the limits of the initial protocol's scalability. In early 2001, variations on the protocol (first implemented in proprietary and closed source clients) allowed an improvement in scalability. Instead of treating every user as client and server, some users were now treated as ultrapeers, routing search requests and responses for users connected to them.
This allowed the network to grow in popularity. In late 2001, the Gnutella client LimeWire Basic became free and open source. In February 2002, Morpheus, a commercial file sharing group, abandoned its FastTrack-based peer-to-peer software and released a new client based on the free and open source Gnutella client Gnucleus.
The word Gnutella today refers not to any one project or piece of software, but to the open protocol used by the various clients.
The name is a portmanteau of GNU and Nutella, the brand name of an Italian hazelnut flavored spread: supposedly, Frankel and Pepper ate a lot of Nutella working on the original project, and intended to license their finished program under the GNU General Public License. Gnutella is not associated with the GNU project[16] or GNU's own peer-to-peer network, GNUnet.
On October 26, 2010, the popular Gnutella clientLimeWire was ordered shut down by Judge Kimba Wood of the United States District Court for the Southern District of New York when she signed a Consent Decree to which recording industry plaintiffs and LimeWire had agreed. This event was the likely cause of a notable drop[17] in the size of the network, because, while negotiating the injunction, LimeWire staff had inserted remote-disabling code into the software. As the injunction came into force, users who had installed affected versions (newer than 5.5.10) were cut off from the P2P network. Since LimeWire was free software, nothing had prevented the creation of forks that omitted the disabling code, as long as LimeWire trademarks were not used. The shutdown did not affect, for example, FrostWire, a fork of LimeWire created in 2004 that carries neither the remote-disabling code nor adware.
On November 9, 2010, LimeWire was resurrected by a secret team of developers and named LimeWire Pirate Edition.[18] It was based on LimeWire 5.6 BETA. This version had its server dependencies removed and all the PRO features enabled for free.
Design
The gnutella search and retrieval protocol
To envision how Gnutella originally worked, imagine a large circle of users (called nodes), each of whom has Gnutella client software. On initial startup, the client software must bootstrap and find at least one other node. Various methods have been used for this, including a pre-existing address list of possibly working nodes shipped with the software, using updated web caches of known nodes (called Gnutella Web Caches), UDP host caches and, rarely, even IRC. Once connected, the client requests a list of working addresses. The client tries to connect to the nodes it was shipped with, as well as nodes it receives from other clients until it reaches a certain quota. It connects to only that many nodes, locally caching the addresses which it has not yet tried and discarding the addresses which it tried and found to be invalid.[19]
When the user wants to do a search, the client sends the request to each actively connected node. In version 0.4 of the protocol, the number of actively connected nodes for a client was quite small (around 5). In that version of the protocol, each node forwards the request to all its actively connected nodes, who, in turn, forward the request. This continues until the packet has reached a predetermined number of hops from the sender (maximum 7).[20]
Since version 0.6 (2002[21]), Gnutella is a composite network made of leaf nodes and ultra nodes (also called ultrapeers). The leaf nodes are connected to a small number of ultrapeers (typically 3) while each ultrapeer is connected to more than 32 other ultrapeers. With this higher outdegree, the maximum number of hops a query can travel was lowered to 4.
Leaves and ultrapeers use the Query Routing Protocol to exchange a Query Routing Table (QRT), a table of 64 Ki-slots and up to 2 Mi-slots consisting of hashed keywords. A leaf node sends its QRT to each of the ultrapeers to which it is connected, and ultrapeers merge the QRT of all their leaves (downsized to 128 Ki-slots) plus their own QRT (if they share files) and exchange that with their own neighbors. Query routing is then done by hashing the words of the query and seeing whether all of them match in the QRT. Ultrapeers do that check before forwarding a query to a leaf node, and also before forwarding the query to a peer ultra node provided this is the last hop the query can travel.
If a search request turns up a result, the node that has the result contacts the searcher. In the classic Gnutella protocol, response messages were sent back along the route taken by the query, as the query itself did not contain identifying information for the node. This scheme was later revised, to deliver search results over UDP, directly to the node that initiated the search, usually an ultrapeer of the node. Thus, in the current protocol, the queries carry the IP address and port number of either node. This lowers the amount of traffic routed through the Gnutella network, making it significantly more scalable.[22]
If the user decides to download the file, they negotiate the file transfer. If the node which has the requested file is not firewalled, the querying node can connect to it directly. However, if the node is firewalled, stopping the source node from receiving incoming connections, the client wanting to download a file sends it a so-called push request to the server for the remote client to initiate the connection instead (to push the file). At first, these push requests were routed along the original chain it used to send the query. This was rather unreliable because routes would often break and routed packets are always subject to flow control. push proxies were introduced to address this problem. These are usually the ultrapeers of a leaf node and they are announced in search results. The client connects to one of these push proxies using an HTTP request and the proxy sends a push request to a leaf on behalf of the client. Normally, it is also possible to send a push request over UDP to the push proxy, which is more efficient than using TCP. Push proxies have two advantages: First, ultrapeer-leaf connections are more stable than routes. This makes push requests much more reliable. Second, it reduces the amount of traffic routed through the Gnutella network.[23]
Finally, when a user disconnects, the client software saves a list of known nodes. This contains the nodes to which the client was connected and the nodes learned from pong packets. The client uses that as its seed list, when it next starts, thus becoming independent of bootstrap services.[24]
In practice, this method of searching on the Gnutella network was often unreliable. Each node is a regular computer user; as such, they are constantly connecting and disconnecting, so the network is never completely stable. Also, the bandwidth cost of searching on Gnutella grew exponentially to the number of connected users,[25] often saturating connections and rendering slower nodes useless. Therefore, search requests would often be dropped, and most queries reached only a very small part of the network. This observation identified the Gnutella network as an unscalable distributed system, and inspired the development of distributed hash tables, which are much more scalable but support only exact-match, rather than keyword, search.[26]
To address the problems of bottlenecks, Gnutella developers implemented a tiered system of ultrapeers and leaves. Instead of all nodes being considered equal, nodes entering the network were kept at the 'edge' of the network, as a leaf. Leaves don't provide routing. Nodes which are capable of routing messages are promoted to ultrapeers. Ultrapeers accept leaf connections and route searches and network maintenance messages. This allows searches to propagate further through the network and allows for numerous alterations in topology. This greatly improved efficiency and scalability.[27]
Additionally, gnutella adopted a number of other techniques to reduce traffic overhead and make searches more efficient. Most notable are Query Routing Protocol (QRP) and Dynamic Querying (DQ). With QRP, a search reaches only those clients which are likely to have the files, so searches for rare files become far more efficient. With DQ, the search stops as soon as the program has acquired enough search results. This vastly reduces the amount of traffic caused by popular searches.[28]
One of the benefits of having Gnutella so decentralized is to make it very difficult to shut the network down and to make it a network in which the users are the only ones who can decide which content will be available. Unlike Napster, where the entire network relied on the central server, Gnutella cannot be shut down by shutting down any one node. A decentralized network prevents bad actors from taking control of the contents of the network and/or manipulating data by controlling the central server.[29]
Protocol features and extensions
Gnutella once operated on a purely query flooding-based protocol. The outdated Gnutella version 0.4 network protocol employs five different packet types, namely:[30]
ping: discover hosts on network
pong: reply to ping
query: search for a file
query hit: reply to query
push: download request for firewalled servants
These packets facilitate searches. File transfers are instead handled by HTTP.[31]
The development of the Gnutella protocol is currently led by the Gnutella Developers Forum (The GDF). Many protocol extensions have been and are being developed by the software vendors and by the free Gnutella developers of the GDF. These extensions include intelligent query routing, SHA-1 checksums, query hit transmission via UDP, querying via UDP, dynamic queries via TCP, file transfers via UDP, XML metadata, source exchange (also termed the download mesh) and parallel downloading in slices (swarming).[31]
There are efforts to finalize these protocol extensions in the Gnutella 0.6 specification, at the Gnutella protocol development website. The Gnutella 0.4 standard is outdated but it remains the latest protocol specification because all extensions, so far, exist as proposals. In fact, it is hard or impossible to connect today with 0.4 handshakes. According to developers in the GDF, version 0.6 is what new developers should pursue using the work-in-progress specifications.[32]
The Gnutella protocol remains under development. Despite attempts to make a clean break with the complexity inherited from the old Gnutella 0.4 and to design a clean new message architecture, it remains one of the most successful file-sharing protocols to date.[33]
Software
The following tables compare general and technical information for a number of applications supporting the Gnutella network. The tables do not attempt to give a complete list of Gnutella clients. The tables are limited to clients that can participate in the current Gnutella network.
↑ Refers to direct client-to-client chat; not IRC chat, which is often also available in the same application through an embedded HTTP browser window.
↑ Automatically configures port forwarding in routers or combination modem/gateways which support UPnP control.
↑ The Reliable UDP protocol provides NAT-to-NAT transfers, sometimes called Firewall-to-Firewall or "hole-punching", in cases where port-forwarding is not or cannot be done by the user.
↑ As GWCs had a history of problems with traffic overload and long-term reliability, UDP host caches became the preferred bootstrap method; though some GWCs remain available for the sake of older software.
↑ Client mode only, as a dependent leaf on ultrapeers.
1 2 Not high outdegree, so is unusable in its current form.
1 2 3 Via the Kademlia-based Mojito DHT network supported only by LimeWire and gtk-gnutella (starting with version r15750); this is completely different from SHA-1 searches supported by most gnutella clients.
Morpheus differs significantly and may have completely independent code from the GnucDNA engine. Morpheus can function as a modern ultrapeer whereas other GnucDNA clients cannot.
BearFlix, a functionally limited version of the BearShare 5.2 series, can search only for images or videos and shared videos are limited to a relatively short length.
MP3 Rocket, 360Share, LemonWire, MP3Torpedo, and DexterWire are variants of LimeWire.
FrostWire (up to version 4.21.8) is nearly identical to LimeWire 4.18 but versions greater than 5.00 no longer use gnutella.
Acquisition and Cabos are custom front-ends overlaying the LimeWire engine.
LimeWire Pirate Edition (5.6.2) is a resurrected version of the unreleased LimeWire 5.6.1 alpha, thus has similar features minus automatic updates (with nags) and centralized remote controls to disable core functions like searches and downloads were removed.
The Gnutella2 protocol (often referred to as G2), despite its name, is not a successor protocol of Gnutella nor related to the original Gnutella project,[36] but rather is a completely different protocol that forked from the original project and piggybacked on the Gnutella name. A sore point with many Gnutella developers is that the Gnutella2 name conveys an upgrade or superiority, which led to a flame war. Other criticism included the use of the Gnutella network to bootstrap G2 peers and poor documentation of the G2 protocol. Additionally, the more frequent search retries of the Shareaza client, one of the initial G2 clients, could unnecessarily burden the Gnutella network.[37]
Both protocols have undergone significant changes since the fork in 2002. G2 has advantages and disadvantages compared to Gnutella. An advantage often cited is that Gnutella2's hybrid search is more efficient than the original Gnutella's query flooding. However, Gnutella replaced query flooding with more efficient search methods, starting with Query Routing in 2002. This was proposed in 2001 by Limewire developers.[38][39][40] An advantage of Gnutella is its large user base, which numbers in the millions,[3]. The G2 network is approximately an order of magnitude smaller.[41] It is difficult to compare the protocols in their current form. The choice of client, on either network, probably affects the end user just as much.
LimeWire was a free peer-to-peer file sharing client for Windows, macOS, Linux and Solaris. Created by Mark Gorton in 2000, it was most prominently a tool used for the download and distribution of pirated materials, particularly pirated music. In 2007, LimeWire was estimated to be installed on over one-third of all computers globally.
Shareaza is a peer-to-peer file sharing client running under Microsoft Windows which supports the gnutella, Gnutella2 (G2), eDonkey, BitTorrent, FTP, HTTP and HTTPS network protocols and handles magnet links, ed2k links, and the now deprecated gnutella and Piolet links. It is available in 30 languages.
giFT Internet File Transfer (giFT) is a computer software daemon that allows several file sharing protocols to be used with a simple client having a graphical user interface (GUI). The client dynamically loads plugins implementing the protocols, as they are required.
A distributed hash table (DHT) is a distributed system that provides a lookup service similar to a hash table. Key–value pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key. The main advantage of a DHT is that nodes can be added or removed with minimum work around re-distributing keys. Keys are unique identifiers which map to particular values, which in turn can be anything from addresses, to documents, to arbitrary data. Responsibility for maintaining the mapping from keys to values is distributed among the nodes, in such a way that a change in the set of participants causes a minimal amount of disruption. This allows a DHT to scale to extremely large numbers of nodes and to handle continual node arrivals, departures, and failures.
MLDonkey is an open-source, multi-protocol, peer-to-peer file sharing application that runs as a back-end server application on many platforms. It can be controlled through a user interface provided by one of many separate front-ends, including a Web interface, telnet interface and over a dozen native client programs.
gtk-gnutella is a peer-to-peer file sharing application which runs on the gnutella network. gtk-gnutella uses the GTK+ toolkit for its graphical user interface. Released under the GNU General Public License, gtk-gnutella is free software.
GNUnet is a software framework for decentralized, peer-to-peer networking and an official GNU package. The framework offers link encryption, peer discovery, resource allocation, communication over many transports and various basic peer-to-peer algorithms for routing, multicast and network size estimation.
Gnutella2, often referred to as G2, is a peer-to-peer protocol developed mainly by Michael Stokes and released in 2002.
Query flooding is a method to search for a resource on a peer-to-peer network. It is simple and scales very poorly and thus is rarely used. Early versions of the Gnutella protocol operated by query flooding; newer versions use more efficient search algorithms.
An anonymous P2P communication system is a peer-to-peer distributed application in which the nodes, which are used to share resources, or participants are anonymous or pseudonymous. Anonymity of participants is usually achieved by special routing overlay networks that hide the physical location of each node from other participants.
Kademlia is a distributed hash table for decentralized peer-to-peer computer networks designed by Petar Maymounkov and David Mazières in 2002. It specifies the structure of the network and the exchange of information through node lookups. Kademlia nodes communicate among themselves using UDP. A virtual or overlay network is formed by the participant nodes. Each node is identified by a number or node ID. The node ID serves not only as identification, but the Kademlia algorithm uses the node ID to locate values.
Magnet is a URI scheme that defines the format of magnet links, a de facto standard for identifying files (URN) by their content, via cryptographic hash value rather than by their location.
The Kad network is a peer-to-peer (P2P) network which implements the Kademlia P2P overlay protocol. The majority of users on the Kad Network are also connected to servers on the eDonkey network, and Kad Network clients typically query known nodes on the eDonkey network in order to find an initial node on the Kad network.
GnucDNA was a software library for building peer-to-peer applications. It provides developers with a common layer to create their own Gnutella or Gnutella2 client or network. As a separate component, GnucDNA can be updated independently of the client, passing down improvements to the applications already using it.
This is a timeline of events in the history of networked file sharing.
FrostWire is a free and open-source BitTorrent client first released in September 2004, as a fork of LimeWire. It was initially very similar to LimeWire in appearance and functionality, but over time developers added more features, including support for the BitTorrent protocol. In version 5, support for the Gnutella network was dropped entirely, and FrostWire became a BitTorrent-only client.
File sharing is a method of distributing electronically stored information such as computer programs and digital media. Below is a list of file sharing applications, most of them make use of peer-to-peer file sharing technologies.
Peer-to-peer file sharing (P2P) systems like Gnutella, KaZaA, and eDonkey/eMule, have become extremely popular in recent years, with the estimated user population in the millions. An academic research paper analyzed Gnutella and eMule protocols and found weaknesses in the protocol; many of the issues found in these networks are fundamental and probably common on other P2P networks. Users of file sharing networks, such as eMule and Gnutella, are subject to monitoring of their activity. Clients may be tracked by IP address, DNS name, software version they use, files they share, queries they initiate, and queries they answer to. Clients may also share their private files to the network without notice due to inappropriate settings.
Phex is a peer-to-peer file sharing client for the gnutella network, released under the terms of the GNU General Public License, so Phex is free software. Phex is based on Java SE 5.0 or later.
OpenConnect is a free and open-source cross-platform multi-protocol virtual private network (VPN) client software which implement secure point-to-point connections.
↑ Chloe Albanesius (18 July 2011). "Indie Labels Sue LimeWire Over Failed Copyright Deal". PC Magazine. Ziff-Davis. Archived from the original on 2 February 2016. Retrieved 19 March 2012. In the wake of the LimeWire shutdown, the percentage of U.S. Internet users who access P2P file-sharing services dropped about 7 percent from [its] all-time high in 2007...
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
