Coral Content Distribution Network

Last updated
Coral Content Distribution Network
Coral CDN logo.png
Developer(s) Michael J. Freedman
Initial release2004
Repository OOjs UI icon edit-ltr-progressive.svg
Operating system Cross-platform (web-based application)
Type P2P Web cache
Website www.coralcdn.org

The Coral Content Distribution Network, sometimes called Coral Cache or Coral, was a free peer-to-peer content distribution network that ran from 2004 until 2015. It was designed and operated by Michael J. Freedman. Coral used the bandwidth of a worldwide network of web proxies and name servers to mirror World Wide Web content, often to avoid the Slashdot effect or to reduce the load on web servers.

Contents

Operation

Coral avoided high loads on individual nodes through an indexing abstraction called a distributed sloppy hash table (DSHT); DSHTs create self-organizing clusters of nodes which fetch information from each other to avoid communicating with more distant or heavily-loaded servers. DSHT is described in the paper Sloppy hashing and self-organizing clusters. [1]

The sloppy hash table refers to the fact that Coral was made up of concentric rings of distributed hash tables (DHTs), each ring representing a wider and wider geographic range (or rather, ping range). The DHTs are composed of nodes all within some latency of each other (for example, a ring of nodes within 20 milliseconds of each other). It avoids hot spots (the 'sloppy' part) by only continuing to query progressively larger sized rings if they are not overburdened. In other words, if the two top-most rings are experiencing too much traffic, a node will just ping closer ones: when a node that is overloaded is reached, upward progression stops. This minimises the occurrence of hot spots, with the disadvantage that knowledge of the system as a whole is reduced.

Usage

Coral users could access content through the Coral Cache by adding .nyud.net to the hostname in the site's URL, resulting in what is known as a 'coralized link'. So, for example,

http://example.com

becomes

http://example.com.nyud.net

Any additional address component after the hostname remains after .nyud.net; hence

http://example.com/folder/page.html

becomes

http://example.com.nyud.net/folder/page.html

For websites that use a non-standard port, for example,

http://example.com:8080

becomes

http://example.com.nyud.net:8080

History

The project was deployed in March 2004, during which it has been hosted on PlanetLab, a large-scale distributed research network of several hundred servers deployed at universities worldwide. It has not, as originally intended, been deployed by third-party volunteer systems. About 300 to 400 PlanetLab servers were running CoralCDN. The source code is freely available under the terms of the GNU GPL.

In August 2012 it was stated that active development had been "stopped for a while", although the CDN itself would continue to operate. [2]

As of April 2015*.nyud.net had stopped resolving, rendering the CDN non-functional. [3] [4] The project website is still up, but has not been updated since 2012.

See also

Related Research Articles

The Domain Name System (DNS) is a hierarchical and decentralized naming system for computers, services, or other resources connected to the Internet or a private network. It associates various information with domain names assigned to each of the participating entities. Most prominently, it translates more readily memorized domain names to the numerical IP addresses needed for locating and identifying computer services and devices with the underlying network protocols. By providing a worldwide, distributed directory service, the Domain Name System has been an essential component of the functionality of the Internet since 1985.

Freenet Peer-to-peer Internet platform for censorship-resistant communication

Freenet is a peer-to-peer platform for censorship-resistant communication. It uses a decentralized distributed data store to keep and deliver information, and has a suite of free software for publishing and communicating on the Web without fear of censorship. Both Freenet and some of its associated tools were originally designed by Ian Clarke, who defined Freenet's goal as providing freedom of speech on the Internet with strong anonymity protection.

Peer-to-peer Type of decentralized and distributed network architecture

Peer-to-peer (P2P) computing or networking is a distributed application architecture that partitions tasks or workloads between peers. Peers are equally privileged, equipotent participants in the application. They are said to form a peer-to-peer network of nodes.

Distributed hash table Decentralized distributed system with lookup service

A distributed hash table (DHT) is a distributed system that provides a lookup service similar to a hash table: key-value pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key. The main advantage of a DHT is that nodes can be added or removed with minimum work around re-distributing keys. Keys are unique identifiers which map to particular values, which in turn can be anything from addresses, to documents, to arbitrary data. Responsibility for maintaining the mapping from keys to values is distributed among the nodes, in such a way that a change in the set of participants causes a minimal amount of disruption. This allows a DHT to scale to extremely large numbers of nodes and to handle continual node arrivals, departures, and failures.

BitTorrent Peer-to-peer file sharing protocol

BitTorrent is a communication protocol for peer-to-peer file sharing (P2P), which enables users to distribute data and electronic files over the Internet in a decentralized manner.

Content delivery network A layer in the internet ecosystem addressing bottlenecks

A content delivery network, or content distribution network (CDN), is a geographically distributed network of proxy servers and their data centers. The goal is to provide high availability and performance by distributing the service spatially relative to end users. CDNs came into existence in the late 1990s as a means for alleviating the performance bottlenecks of the Internet, even as the Internet was starting to become a mission-critical medium for people and enterprises. Since then, CDNs have grown to serve a large portion of the Internet content today, including web objects, downloadable objects, applications, live streaming media, on-demand streaming media, and social media sites.

An overlay network is a computer network that is layered on top of another network.

In computer science, consistent hashing is a special kind of hashing such that when a hash table is resized, only keys need to be remapped on average where is the number of keys and is the number of slots. In contrast, in most traditional hash tables, a change in the number of array slots causes nearly all keys to be remapped because the mapping between the keys and the slots is defined by a modular operation.

Peer Name Resolution Protocol (PNRP) is a peer-to-peer protocol designed by Microsoft. PNRP enables dynamic name publication and resolution, and requires IPv6.

Pastry is an overlay network and routing network for the implementation of a distributed hash table (DHT) similar to Chord. The key-value pairs are stored in a redundant peer-to-peer network of connected Internet hosts. The protocol is bootstrapped by supplying it with the IP address of a peer already in the network and from then on via the routing table which is dynamically built and repaired. It is claimed that because of its redundant and decentralized nature there is no single point of failure and any single node can leave the network at any time without warning and with little or no chance of data loss. The protocol is also capable of using a routing metric supplied by an outside program, such as ping or traceroute, to determine the best routes to store in its routing table.

Perfect Dark (P2P) Peer to peer software

Perfect Dark (パーフェクトダーク) is a peer-to-peer file-sharing (P2P) application from Japan designed for use with Microsoft Windows. It was launched in 2006. Its author is known by the pseudonym Kaichō. Perfect Dark was developed with the intention for it to be the successor to both Winny and Share software. While Japan's Association for Copyright of Computer Software reported that in January 2014, the number of nodes connected on Perfect Dark was less than on Share, but more than on Winny, Netagent in 2018 reported Winny being the largest with 50 000 nodes followed by Perfect Dark with 30 000 nodes followed by Share with 10 000. Netagent asserts that the number of nodes on Perfect Dark have fallen since 2015 while the numbers of Winny hold steady. Netagent reports that users of Perfect Dark are most likely to share books/manga.

MagmaFS

Magma is a distributed file system based on a distributed hash table, written in C, compatible with Linux and BSD kernels using FUSE.

Pharos is a hierarchical and decentralized network coordinate system. With the help of a simple two-level architecture, it achieves much better prediction accuracy then the representative Vivaldi coordinates, and it is incrementally deployable.

In the BitTorrent file distribution system, a torrent file or meta-info file is a computer file that contains metadata about files and folders to be distributed, and usually also a list of the network locations of trackers, which are computers that help participants in the system find each other and form efficient distribution groups called swarms. A torrent file does not contain the content to be distributed; it only contains information about those files, such as their names, folder structure, and sizes obtained via cryptographic hash values for verifying file integrity. The term torrent may refer either to the metadata file or to the files downloaded, depending on the context.

Mainline DHT is the name given to the Kademlia-based Distributed Hash Table (DHT) used by BitTorrent clients to find peers via the BitTorrent protocol. The idea of using a DHT for distributed tracking was first implemented in Azureus 2.3.0.0 in May 2005, from which it gained significant popularity. Unrelated but similarly timed BitTorrent, Inc. released their own similar DHT into their client, called Mainline DHT and thus popularized the use of distributed tracking in the BitTorrent Protocol. Measurement shows by 2013 users of Mainline DHT is from 10 million to 25 million, with a daily churn of at least 10 million.

Content delivery network interconnection (CDNI) is a set of interfaces and mechanisms required for interconnecting two independent content delivery networks (CDNs) that enables one to deliver content on behalf of the other. Interconnected CDNs offer many benefits, such as footprint extension, reduced infrastructure costs, higher availability, etc., for content service providers (CSPs), CDNs, and end users. Among its many use cases, it allows small CDNs to interconnect and provides services for CSPs that allows them to compete against the CDNs of global CSPs.

Elliptics is a distributed key-value data storage with open source code. By default it is a classic distributed hash table (DHT) with multiple replicas put in different groups. Elliptics was created to meet requirements of multi-datacenter and physically distributed storage locations when storing huge amount of medium and large files.

InterPlanetary File System Content-addressable, peer-to-peer hypermedia distribution protocol

The InterPlanetary File System (IPFS) is a protocol and peer-to-peer network for storing and sharing data in a distributed file system. IPFS uses content-addressing to uniquely identify each file in a global namespace connecting all computing devices.

Apache Ignite

Apache Ignite is a distributed database for high-performance computing with in-memory speed.

Michael J. Freedman American computer scientist

Michael J. Freedman is an American computer scientist who is a Professor of Computer Science at Princeton University. He is notable for designing systems such as the Coral Content Distribution Network and TimescaleDB. His research interests include distributed systems, networking, and security.

References

  1. Freedman, Michael J.; Mazières, David (2003). "Sloppy Hashing and Self-Organizing Clusters" (PDF). Retrieved July 11, 2018.Cite journal requires |journal= (help)
  2. "News and Overview". The Coral Content Distribution Network. 20 August 2012. Retrieved 24 September 2016. News last updated 20 August 2012 (as of September 2016)
  3. Freedman, Mike (6 April 2016). "[coral-users] Coral CDN Down?". Archived from the original on 2018-08-29. Retrieved 10 June 2018.
  4. Freedman, Mike (4 December 2016). "[coral-users] So, no more nyud.net?". Archived from the original on 2017-05-06. Retrieved 10 June 2018.