Peering

In computer networking, peering is a voluntary interconnection of administratively separate Internet networks for the purpose of exchanging traffic between the "down-stream" users of each network. Peering is settlement-free, also known as "bill-and-keep" or "sender keeps all", meaning that neither party pays the other in association with the exchange of traffic; instead, each derives and retains revenue from its own customers.

An agreement by two or more networks to peer is instantiated by a physical interconnection of the networks, an exchange of routing information through the Border Gateway Protocol (BGP) routing protocol, tacit agreement to norms of conduct and, in some extraordinarily rare cases (0.07%), a formalized contractual document. ^{[1] [2]}

In 0.02% of cases the word "peering" is used to describe situations where there is some settlement involved. Because these outliers can be viewed as creating ambiguity, the phrase "settlement-free peering" is sometimes used to explicitly denote normal cost-free peering. ^[3]

History

The first Internet exchange point was the Commercial Internet eXchange (CIX), formed by Alternet/UUNET (now Verizon Business), PSI, and CERFNET to exchange traffic without regard for whether the traffic complied with the acceptable use policy (AUP) of the NSFNet or ANS' interconnection policy. ^[4] The CIX infrastructure consisted of a single router, managed by PSI, and was initially located in Santa Clara, California. Paying CIX members were allowed to attach to the router directly or via leased lines. After some time, the router was also attached to the Pacific Bell SMDS cloud. The router was later moved to the Palo Alto Internet Exchange, or PAIX, which was developed and operated by Digital Equipment Corporation (DEC). Because the CIX operated at OSI layer 3, rather than OSI layer 2, and because it was not neutral, in the sense that it was operated by one of its participants rather than by all of them collectively, and it conducted lobbying activities supported by some of its participants and not by others, it would not today be considered an Internet exchange point. Nonetheless, it was the first thing to bear that name.

The first exchange point to resemble modern, neutral, Ethernet-based exchanges was the Metropolitan Area Ethernet, or MAE, in Tysons Corner, Virginia. When the United States government de-funded the NSFNET backbone, Internet exchange points were needed to replace its function, and initial governmental funding was used to aid the preexisting MAE and bootstrap three other exchanges, which they dubbed NAPs, or "Network Access Points," in accordance with the terminology of the National Information Infrastructure document. ^[5] All four are now defunct or no longer functioning as Internet exchange points:

• MAE-East – Located in Tysons Corner, Virginia, and later relocated to Ashburn, Virginia
• Chicago NAP – Operated by Ameritech and located in Chicago, Illinois
• New York NAP – Operated by Sprint and located in Pennsauken, New Jersey
• San Francisco NAP – Operated by PacBell and located in the Bay Area

As the Internet grew, and traffic levels increased, these NAPs became a network bottleneck. Most of the early NAPs utilized FDDI technology, which provided only 100 Mbit/s of capacity to each participant. Some of these exchanges upgraded to ATM technology, which provided OC-3 (155 Mbit/s) and OC-12 (622 Mbit/s) of capacity.

Other prospective exchange point operators moved directly into offering Ethernet technology, such as gigabit Ethernet (1,000 Mbit/s), which quickly became the predominant choice for Internet exchange points due to the reduced cost and increased capacity offered. Today, almost all significant exchange points operate solely over Ethernet, and most of the largest exchange points offer 10, 40, and even 100 gigabit service.

During the dot-com boom, many exchange point and carrier-neutral colocation providers had plans to build as many as 50 locations to promote carrier interconnection in the United States alone. Essentially all of these plans were abandoned following the dot-com bust, and today it is considered both economically and technically infeasible to support this level of interconnection among even the largest of networks.

How peering works

The Internet is a collection of separate and distinct networks referred to as autonomous systems, each one consisting of a set of globally unique IP addresses and a unique global BGP routing policy.

The interconnection relationships between Autonomous Systems are of exactly two types:

• Peering - Two networks exchange traffic between their users freely, and for mutual benefit.
• Transit – One network pays another network for access to the Internet.

Therefore, in order for a network to reach any specific other network on the Internet, it must either:

• Sell transit service to that network or a chain of resellers ending at that network (making them a 'customer'),
• Peer with that network or with a network which sells transit service to that network, or
• Buy transit service from any other network (which is then responsible for providing interconnection to the rest of the Internet).

The Internet is based on the principle of global or end-to-end reachability, which means that any Internet user can transparently exchange traffic with any other Internet user. Therefore, a network is connected to the Internet if and only if it buys transit, or peers with every other network which also does not purchase transit (which together constitute a "default free zone" or "DFZ").

Public peering is done at Internet exchange points (IXPs), while private peering can be done with direct links between networks. ^{[6] [7]}

Motivations for peering

Peering involves two networks coming together to exchange traffic with each other freely, and for mutual benefit. ^{[8] [9]} This 'mutual benefit' is most often the motivation behind peering, which is often described solely by "reduced costs for transit services". Other less tangible motivations can include:

• Increased redundancy (by reducing dependence on one or more transit providers).
• Increased capacity for extremely large amounts of traffic (distributing traffic across many networks).
• Increased routing control over one's traffic.
• Improved performance (attempting to bypass potential bottlenecks with a "direct" path).
• Improved perception of one's network (being able to claim a "higher tier").
• Ease of requesting for emergency aid (from friendly peers).

Physical interconnections for peering

Diagram of the Layer 1 (physical) and Layer 2 (Data Link) topology of an Internet Exchange Point (IXP).

Diagram of the Layer 3 (network) topology of an Internet Exchange Point (IXP).

The physical interconnections used for peering are categorized into two types:

• Public peering – Interconnection utilizing a multi-party shared switch fabric such as an Ethernet switch.
• Private peering – Interconnection utilizing a point-to-point link between two parties.

Public peering

Public peering is accomplished across a Layer 2 access technology, generally called a shared fabric. At these locations, multiple carriers interconnect with one or more other carriers across a single physical port. Historically, public peering locations were known as network access points (NAPs). Today they are most often called exchange points or Internet exchanges ("IXP"). Many of the largest exchange points in the world can have hundreds of participants, and some span multiple buildings and colocation facilities across a city. ^[10]

Since public peering allows networks interested in peering to interconnect with many other networks through a single port, it is often considered to offer "less capacity" than private peering, but to a larger number of networks. Many smaller networks, or networks which are just beginning to peer, find that public peering exchange points provide an excellent way to meet and interconnect with other networks which may be open to peering with them. Some larger networks utilize public peering as a way to aggregate a large number of "smaller peers", or as a location for conducting low-cost "trial peering" without the expense of provisioning private peering on a temporary basis, while other larger networks are not willing to participate at public exchanges at all.

A few exchange points, particularly in the United States, are operated by commercial carrier-neutral third parties, which are critical for achieving cost-effective data center connectivity. ^[11]

Private peering

Private peering is the direct interconnection between only two networks, across a Layer 1 or 2 medium that offers dedicated capacity that is not shared by any other parties. Early in the history of the Internet, many private peers occurred across "telco" provisioned SONET circuits between individual carrier-owned facilities. Today, most private interconnections occur at carrier hotels or carrier neutral colocation facilities, where a direct crossconnect can be provisioned between participants within the same building, usually for a much lower cost than telco circuits.

Most of the traffic on the Internet, especially traffic between the largest networks, occurs via private peering. However, because of the resources required to provision each private peer, many networks are unwilling to provide private peering to "small" networks, or to "new" networks which have not yet proven that they will provide a mutual benefit.

Peering agreement

Throughout the history of the Internet, there have been a spectrum of kinds of agreements between peers, ranging from handshake agreements to written contracts as required by one or more parties. Such agreements set forth the details of how traffic is to be exchanged, along with a list of expected activities which may be necessary to maintain the peering relationship, a list of activities which may be considered abusive and result in termination of the relationship, and details concerning how the relationship can be terminated. Detailed contracts of this type are typically used between the largest ISPs, as well as the ones operating in the most heavily regulated economies. As of 2011, such contracts account for less than 0.5% of all peering agreements. ^[1]

Depeering

By definition, peering is the voluntary and free exchange of traffic between two networks, for mutual benefit. If one or both networks believes that there is no longer a mutual benefit, they may decide to cease peering: this is known as depeering. Some of the reasons why one network may wish to depeer another include:

• A desire that the other network pay settlement, either in exchange for continued peering or for transit services.
• A belief that the other network is "profiting unduly" from the no-settlement interconnection.
• Concern over traffic ratios, which is related to the fair sharing of cost for the interconnection.
• A desire to peer with the upstream transit provider of the peered network.
• Abuse of the interconnection by the other party, such as pointing default or utilizing the peer for transit.
• Instability of the peered network, repeated routing leaks, lack of response to network abuse issues, etc.
• The inability or unwillingness of the peered network to provision additional capacity for peering.
• The belief that the peered network is unduly peering with one's customers.
• Various external political factors (including personal conflicts between individuals at each network).

In some situations, networks which are being depeered have been known to attempt to fight to keep the peering by intentionally breaking the connectivity between the two networks when the peer is removed, either through a deliberate act or an act of omission. The goal is to force the depeering network to have so many customer complaints that they are willing to restore peering. Examples of this include forcing traffic via a path that does not have enough capacity to handle the load, or intentionally blocking alternate routes to or from the other network. Some notable examples of these situations have included:

• BBN Planet vs Exodus Communications ^[12]
• PSINet vs Cable & Wireless ^[13]
• AOL Transit Data Network (ATDN) vs Cogent Communications ^[14]
• France Telecom vs Cogent Communications ^[15]
• France Telecom (Wanadoo) vs Proxad (Free) ^[16]
• Level 3 Communications vs XO Communications ^[17]
• Level 3 Communications vs Cogent Communications ^[18]
• Telecom/Telefónica/Impsat/Prima vs CABASE (Argentina) ^[19]
• Cogent Communications vs TeliaSonera ^[20]
• Sprint-Nextel vs Cogent Communications ^[21]
• SFR vs OVH ^[22]
• The French ISP 'Free' vs YouTube ^[23]

Modern peering

Donut peering model

The "donut peering" model ^[24] describes the intensive interconnection of small and medium-sized regional networks that make up much of the Internet. ^[1] Traffic between these regional networks can be modeled as a toroid, with a core "donut hole" that is poorly interconnected to the networks around it. ^[25]

As detailed above, some carriers attempted to form a cartel of self-described Tier 1 networks, nominally refusing to peer with any networks outside the oligopoly. ^[1] Seeking to reduce transit costs, connections between regional networks bypass those "core" networks. Data takes a more direct path, reducing latency and packet loss. This also improves resiliency between consumers and content providers via multiple connections in many locations around the world, in particular during business disputes between the core transit providers. ^{[26] [27]}

Multilateral peering

The majority of BGP AS-AS adjacencies are the product of multilateral peering agreements, or MLPAs. ^[1] In multilateral peering, an unlimited number of parties agree to exchange traffic on common terms, using a single agreement to which they each accede. The multilateral peering is typically technically instantiated in a route server or route reflector (which differ from looking glasses in that they serve routes back out to participants, rather than just listening to inbound routes) to redistribute routes via a BGP hub-and-spoke topology, rather than a partial-mesh topology. The two primary criticisms of multilateral peering are that it breaks the shared fate of the forwarding and routing planes, since the layer-2 connection between two participants could hypothetically fail while their layer-2 connections with the route server remained up, and that they force all participants to treat each other with the same, undifferentiated, routing policy. The primary benefit of multilateral peering is that it minimizes configuration for each peer, while maximizing the efficiency with which new peers can begin contributing routes to the exchange. While optional multilateral peering agreements and route servers are now widely acknowledged to be a good practice, mandatory multilateral peering agreements (MMLPAs) have long been agreed to not be a good practice. ^[28]

Peering locations

The modern Internet operates with significantly more peering locations than at any time in the past, resulting in improved performance and better routing for the majority of the traffic on the Internet. ^[1] However, in the interests of reducing costs and improving efficiency, most networks have attempted to standardize on relatively few locations within these individual regions where they will be able to quickly and efficiently interconnect with their peering partners.

Exchange points

As of 2021, the largest exchange points in the world are Ponto de Troca de Tráfego Metro São Paulo, in São Paulo, with 2,289 peering networks; OpenIXP in Jakarta, with 1,097 peering networks; and DE-CIX in Frankfurt, with 1,050 peering networks. ^[29] The United States, with a historically larger focus on private peering and commercial public peering, has much less traffic visible on public peering switch-fabrics compared to other regions that are dominated by non-profit membership exchange points. Collectively, the many exchange points operated by Equinix are generally considered to be the largest, though traffic figures are not generally published. Other important but smaller exchange points include AMS-IX in Amsterdam, LINX and LONAP in London, and NYIIX in New York.

URLs to some public traffic statistics of exchange points include:

AMS-IX DE-CIX

LINX MSK-IX

TORIX NYIIX

LAIIX TOP-IX

Netnod Mix Milano

ix.br SP SFMIX

Peering and BGP

A great deal of the complexity in the BGP routing protocol exists to aid the enforcement and fine-tuning of peering and transit agreements. BGP allows operators to define a policy that determines where traffic is routed. Three things are commonly used to determine routing: local-preference, multi exit discriminators (MEDs) and AS-Path. Local-preference is used internally within a network to differentiate classes of networks. For example, a particular network will have a higher preference set on internal and customer advertisements. Settlement free peering is then configured to be preferred over paid IP transit.

Networks that speak BGP to each other can engage in multi exit discriminator exchange with each other, although most do not. When networks interconnect in several locations, MEDs can be used to reference that network's interior gateway protocol cost. This results in both networks sharing the burden of transporting each other's traffic on their own network (or cold potato ). Hot-potato or nearest-exit routing, which is typically the normal behavior on the Internet, is where traffic destined to another network is delivered to the closest interconnection point.

Law and policy

Internet interconnection is not regulated in the same way that public telephone network interconnection is regulated. ^[30] Nevertheless, Internet interconnection has been the subject of several areas of federal policy in the United States. Perhaps the most dramatic example of this is the attempted MCI Worldcom/Sprint merger. In this case, the Department of Justice blocked the merger specifically because of the impact of the merger on the Internet backbone market (thereby requiring MCI to divest itself of its successful "internetMCI" business to gain approval). ^[31] In 2001, the Federal Communications Commission's advisory committee, the Network Reliability and Interoperability Council recommended that Internet backbones publish their peering policies, something that they had been hesitant to do beforehand^{[ citation needed ]}. The FCC has also reviewed competition in the backbone market in its Section 706 proceedings which review whether advanced telecommunications are being provided to all Americans in a reasonable and timely manner.

Finally, Internet interconnection has become an issue in the international arena under something known as the International Charging Arrangements for Internet Services (ICAIS). ^[32] In the ICAIS debate, countries underserved by Internet backbones have complained that it is unfair that they must pay the full cost of connecting to an Internet exchange point in a different country, frequently the United States. These advocates argue that Internet interconnection should work like international telephone interconnection, with each party paying half of the cost. ^[33] Those who argue against ICAIS point out that much of the problem would be solved by building local exchange points. A significant amount of the traffic, it is argued, that is brought to the US and exchanged then leaves the US, using US exchange points as switching offices but not terminating in the US. ^[34] In some worst-case scenarios, traffic from one side of a street is brought all the way to a distant exchange point in a foreign country, exchanged, and then returned to another side of the street. ^[35] Countries with liberalized telecommunications and open markets, where competition between backbone providers occurs, tend to oppose ICAIS.^{[ citation needed ]}

See also

• Autonomous system
• Default-free zone
• Interconnect agreement
• Internet traffic engineering
• Net neutrality
• North American Network Operators' Group (NANOG)

References

1. Woodcock, Bill; Adhikari, Vijay (2 May 2011). "Survey of Characteristics of Internet Carrier Interconnection Agreements" (PDF). Packet Clearing House . Retrieved 5 May 2011.
2. Woodcock, Bill; Frigino, Marco (21 November 2016). "Survey of Characteristics of Internet Carrier Interconnection Agreements" (PDF). Packet Clearing House . Retrieved 28 May 2021. Of the total analyzed agreements, 1,347 (0.07%) were formalized in written contracts. This is down from 0.49% in 2011. The remaining 1,934,166 (99.93%) were "handshake" agreements in which the parties agreed to informal or commonly understood terms without creating a written document.
3. Woodcock, Bill; Frigino, Marco (21 November 2016). "Survey of Characteristics of Internet Carrier Interconnection Agreements" (PDF). Packet Clearing House . Retrieved 28 May 2021. Of the agreements we analyzed, 1,935,111 (99.98%) had symmetric terms, in which each party gave and received the same conditions as the other. Only 403 (0.02%) had asymmetric terms, in which the parties gave and received conditions with specifically defined differences, and these exceptions were down from 0.27% in 2011. Typical examples of asymmetric agreements are ones in which one of the parties compensates the other for routes that it would not otherwise receive (sometimes called "paid peering" or "on-net routes"), or in which one party is required to meet terms or requirements imposed by the other ("minimum peering requirements"), often concerning volume of traffic or number or geographic distribution of interconnection locations.
4. "Internet History :: Era of Disruption & Competition: CIX". Cybertelecom, Federal Internet Law & Policy. Archived from the original on 12 June 2021. Retrieved 30 March 2022.
5. Ford, Peter; Aiken, B.; Braun, H.W. (February 2004). "NSF implementation plan for interim NREN". Journal on High Speed Networking, 1993.
6. Information Network Engineering. 株式会社 オーム社. 20 July 2015. ISBN   978-4-274-99991-8.
7. Sunyaev, Ali (12 February 2020). Internet Computing: Principles of Distributed Systems and Emerging Internet-Based Technologies. Springer. ISBN   978-3-030-34957-8.
8. nowaybackbot. "What is peering & why networks peer". peer.org.uk. Retrieved 11 February 2022.
9. "DrPeering International - Top 4 Motivations to Peer".
10. "Internet Exchange Directory". Packet Clearing House.
11. Cosmano, Joe (2009), Choosing a Data Center (PDF), Disaster Recovery Journal, retrieved 21 July 2012
12. John Curran (30 November 2010). "Ratios and Peering" . Retrieved 9 July 2011.
13. Burton, Graeme (7 June 2001). "PSINet-C&W dispute causes Internet blackout". Information Age. Archived from the original on 27 September 2007. Retrieved 28 September 2006.
14. Noguchi, Yuki (27 December 2002). "'Peering' Dispute With AOL Slows Cogent Customer Access". Washington Post . Retrieved 28 September 2006.
15. Kuri, Jürgen; Smith, Robert W. (21 April 2005). "France Telecom severs all network links to competitor Cogent". Heise online . Retrieved 28 September 2006.
16. Le Bouder, Gonéri (11 January 2003). "Problème de peering entre Free et France Télécom" (in French). LinuxFr. Retrieved 28 September 2006.
17. "Level 3 and XO Communications Sign Settlement-Free Peering Agreement". PR Newswire. 7 January 2013. Retrieved 17 January 2024.
18. Cowley, Stacey (6 October 2005). "ISP spat blacks out Net connections". InfoWorld . Archived from the original on 8 January 2007. Retrieved 28 September 2006.
19. "CABASE sale aireada del conflicto NAP".
20. "The Telia-Cogent Spat Could Ruin the Web For Many – GigaOM".
21. Ricknäs, Mikael (31 October 2008). "Sprint-Cogent Dispute Puts Small Rip in Fabric of Internet". PC World . Retrieved 31 October 2008.
22. Guillaume, Nicolas (12 February 2011). "INTERCONNEXION RÉSEAUX : OVH ET SFR CALMENT LE JEU" (in French). ITespresso . Retrieved 12 February 2011.
23. Fradin, Andréa (15 January 2013). "Pourquoi ça rame quand je veux regarder une vidéo YouTube avec Free". Slate (in French). Retrieved 15 January 2013.
24. Woodcock, Bill (13 January 2003). "Internet Topology and Economics: How Supply and Demand Influence the Changing Shape of the Global Network" (ppt). lecture at the University of Minnesota Digital Technology Center. Packet Clearing House . Retrieved 28 April 2011.
25. "Changing Role of Peering & Transit in IP Network Interconnection Economics" (PDF). Cook Report on Internet. XI (8–9). Cook Network Consultants. November–December 2002. ISSN   1071-6327. Archived from the original (PDF) on 19 July 2011. Retrieved 28 April 2011.
26. Kirkwood, Grant (September 2009). "The 'Donut Peering' Model: Optimizing IP Transit for Online Video" (PDF). Archived from the original (PDF) on 16 November 2009. Retrieved 2 October 2009.
27. Mohney, Doug (4 September 2009). "A Deep Dive Into IP Voice Peering". IP Communications. Technology Marketing Corporation . Retrieved 4 September 2009.
28. "Internet Exchange Point Policy Documents: Layer 3 participant technical requirements: Mandatory multi-lateral peering". Packet Clearing House. Archived from the original on 9 August 2014. Retrieved 4 October 2013.
29. "Packet Clearing House - Internet Exchange Point Directory". pch.net. Packet Clearing House. 28 May 2021. Retrieved 28 May 2021.
30. Woodcock, Bill; Weller, Dennis (29 January 2013). "Internet Traffic Exchange: Market Developments and Policy Challenges" (PDF). OECD Digital Economy Papers. 207. OECD. doi:10.1787/5k918gpt130q-en. Archived (PDF) from the original on 8 August 2021. The performance of the Internet market model contrasts sharply with that of traditional regulated forms of voice traffic exchange. If the price of Internet transit were stated in the form of an equivalent voice minute rate, it would be about USD 0.0000008 per minute—five orders of magnitude lower than typical voice rates. This is a remarkable and under-recognised endorsement of the multi-stakeholder, market driven nature of the Internet.
31. "Justice Departments Clears WorldCom/MCI Merger after MCI Agrees to Sell its Internet Business". Archived from the original on 1 June 2009.
32. Internet Traffic Exchange and the Development of End to End International Telecommunication Competition, OECD 3/25/02
33. "ITU-T Recommendation D.50".
34. CAIDA: Internet Measurement: Myths about Internet data (5 December 2001)
35. Woodcock, Bill; Edelman, Benjamin (12 September 2012). "Toward Efficiencies in Canadian Internet Traffic Exchange" (PDF). Canadian Internet Registration Authority and Packet Clearing House. Archived from the original (PDF) on 25 August 2013. Retrieved 20 October 2013.

External links

• PeeringDB: A free database of peering locations and participants
• The peering Playbook (PDF): Strategies of peering networks
• Example Tier 1 Peering Requirements: AT&T (AS7018)
• Example Tier 1 Peering Requirements: AOL Transit Data Network (AS1668)
• Example Tier 2 Peering Requirements: Entanet (AS8468)
• Cybertelecom :: Backbones – Federal Internet Law and Policy
• How the 'Net works: an introduction into Peering and Transit, Ars Technica