IP Flow Information Export

Last updated

Internet Protocol Flow Information Export (IPFIX) is an IETF protocol, as well as the name of the IETF working group defining the protocol. It was created based on the need for a common, universal standard of export for Internet Protocol flow information from routers, probes and other devices that are used by mediation systems, accounting/billing systems and network management systems to facilitate services such as measurement, accounting and billing. The IPFIX standard defines how IP flow information is to be formatted and transferred from an exporter to a collector. [1] Previously many data network operators were relying on Cisco Systems' proprietary NetFlow technology for traffic flow information export.

Contents

The IPFIX standards requirements were outlined in the original RFC 3917. Cisco NetFlow Version 9 was the basis for IPFIX. The basic specifications for IPFIX are documented in RFC 7011 through RFC 7015, and RFC 5103.

Architecture

The following figure shows a typical architecture of information flow in an IPFIX architecture:

                       Exporter      IPFIX         Collector                           O--------------------------->O                           |                           | Observation                           | Domain                           |        Metering #1        | Metering #2          O----------------O----------------O Metering #3          |                |                |          | Observation    | Observation    | Observation          | Point #1       | Point #2       | Point #3          v                |                | ---- IP Traffic --->      |                |                           v                | --------------- More IP Traffic --->       |                                            v ---------------------------------- More IP Traffic --->

A pool of Metering Processes collects data packets at one or more Observation Points, optionally filters them and aggregates information about these packets. An Exporter then gathers each of the Observation Points together into an Observation Domain and sends this information via the IPFIX protocol to a Collector. Exporters and Collectors are in a many-to-many relationship: One Exporter can send data to many Collectors and one Collector can receive data from many Exporters. [2]

Protocol

Similar to the NetFlow Protocol, IPFIX considers a flow to be any number of packets observed in a specific timeslot and sharing a number of properties, e.g. "same source, same destination, same protocol". Using IPFIX, devices like routers can inform a central monitoring station about their view of a potentially larger network.

IPFIX is a push protocol, i.e. each sender will periodically send IPFIX messages to configured receivers without any interaction by the receiver.

The actual makeup of data in IPFIX messages is to a great extent up to the sender. IPFIX introduces the makeup of these messages to the receiver with the help of special Templates. The sender is also free to use user-defined data types in its messages, so the protocol is freely extensible and can adapt to different scenarios.

IPFIX prefers the Stream Control Transmission Protocol (SCTP) as its transport layer protocol, but also allows the use of the Transmission Control Protocol (TCP) or User Datagram Protocol (UDP).

Example

A simple information set sent via IPFIX might look like this:

Source         Destination    Packets ------------------------------------------ 192.168.0.201  192.168.0.1    235 192.168.0.202  192.168.0.1    42

This information set would be sent in the following IPFIX message:

Bits 0..15Bits 16..31
Version = 0x000aMessage Length = 64 Bytes
Export Timestamp = 2005-12-31 23:59:60
Sequence Number = 0
Observation Domain ID = 12345678
Set ID = 2 (Template)Set Length = 20 Bytes
Template ID = 256Number of Fields = 3
Typ = sourceIPv4AddressField Length = 4 Bytes
Typ = destinationIPv4AddressField Length = 4 Bytes
Typ = packetDeltaCountField Length = 4 Bytes
Set ID = 256 (Data Set
using Template 256)
Set Length = 28 Bytes
Record 1, Field 1 = 192.168.0.201
Record 1, Field 2 = 192.168.0.1
Record 1, Field 3 = 235 Packets
Record 2, Field 1 = 192.168.0.202
Record 2, Field 2 = 192.168.0.1
Record 2, Field 3 = 42 Packets

As can be seen, the message contains the IPFIX header and two IPFIX Sets: One Template Set that introduces the build-up of the Data Set used, as well as one Data Set, which contains the actual data. When IPFIX is sent over a protocol which keeps a session state (TCP or SCTP), the Template Set need not be retransmitted since it is buffered in Collectors. Since the Template Set can change over time, it must be retransmitted if a new session state is established or if IPFIX is sent over UDP which is a session-less protocol.

See also

Related Research Articles

The Internet Control Message Protocol (ICMP) is a supporting protocol in the Internet protocol suite. It is used by network devices, including routers, to send error messages and operational information indicating success or failure when communicating with another IP address. For example, an error is indicated when a requested service is not available or that a host or router could not be reached. ICMP differs from transport protocols such as TCP and UDP in that it is not typically used to exchange data between systems, nor is it regularly employed by end-user network applications.

<span class="mw-page-title-main">IPv4</span> Fourth version of the Internet Protocol

Internet Protocol version 4 (IPv4) is the first version of the Internet Protocol (IP) as a standalone specification. It is one of the core protocols of standards-based internetworking methods in the Internet and other packet-switched networks. IPv4 was the first version deployed for production on SATNET in 1982 and on the ARPANET in January 1983. It is still used to route most Internet traffic today, even with the ongoing deployment of Internet Protocol version 6 (IPv6), its successor.

Multiprotocol Label Switching (MPLS) is a routing technique in telecommunications networks that directs data from one node to the next based on labels rather than network addresses. Whereas network addresses identify endpoints, the labels identify established paths between endpoints. MPLS can encapsulate packets of various network protocols, hence the multiprotocol component of the name. MPLS supports a range of access technologies, including T1/E1, ATM, Frame Relay, and DSL.

Quality of service (QoS) is the description or measurement of the overall performance of a service, such as a telephony or computer network, or a cloud computing service, particularly the performance seen by the users of the network. To quantitatively measure quality of service, several related aspects of the network service are often considered, such as packet loss, bit rate, throughput, transmission delay, availability, jitter, etc.

The Real-time Transport Protocol (RTP) is a network protocol for delivering audio and video over IP networks. RTP is used in communication and entertainment systems that involve streaming media, such as telephony, video teleconference applications including WebRTC, television services and web-based push-to-talk features.

The Transmission Control Protocol (TCP) is one of the main protocols of the Internet protocol suite. It originated in the initial network implementation in which it complemented the Internet Protocol (IP). Therefore, the entire suite is commonly referred to as TCP/IP. TCP provides reliable, ordered, and error-checked delivery of a stream of octets (bytes) between applications running on hosts communicating via an IP network. Major internet applications such as the World Wide Web, email, remote administration, and file transfer rely on TCP, which is part of the Transport layer of the TCP/IP suite. SSL/TLS often runs on top of TCP.

The Address Resolution Protocol (ARP) is a communication protocol used for discovering the link layer address, such as a MAC address, associated with a given internet layer address, typically an IPv4 address. This mapping is a critical function in the Internet protocol suite. ARP was defined in 1982 by RFC 826, which is Internet Standard STD 37.

<span class="mw-page-title-main">Transport layer</span> Layer in the OSI and TCP/IP models providing host-to-host communication services for applications

In computer networking, the transport layer is a conceptual division of methods in the layered architecture of protocols in the network stack in the Internet protocol suite and the OSI model. The protocols of this layer provide end-to-end communication services for applications. It provides services such as connection-oriented communication, reliability, flow control, and multiplexing.

The Resource Reservation Protocol (RSVP) is a transport layer protocol designed to reserve resources across a network using the integrated services model. RSVP operates over an IPv4 or IPv6 and provides receiver-initiated setup of resource reservations for multicast or unicast data flows. It does not transport application data but is similar to a control protocol, like Internet Control Message Protocol (ICMP) or Internet Group Management Protocol (IGMP). RSVP is described in RFC 2205.

<span class="mw-page-title-main">Optimized Link State Routing Protocol</span> IP routing protocol optimized for mobile ad hoc networks

The Optimized Link State Routing Protocol (OLSR) is an IP routing protocol optimized for mobile ad hoc networks, which can also be used on other wireless ad hoc networks. OLSR is a proactive link-state routing protocol, which uses hello and topology control (TC) messages to discover and then disseminate link state information throughout the mobile ad hoc network. Individual nodes use this topology information to compute next hop destinations for all nodes in the network using shortest hop forwarding paths.

In computer networking, the Datagram Congestion Control Protocol (DCCP) is a message-oriented transport layer protocol. DCCP implements reliable connection setup, teardown, Explicit Congestion Notification (ECN), congestion control, and feature negotiation. The IETF published DCCP as RFC 4340, a proposed standard, in March 2006. RFC 4336 provides an introduction.

NetFlow is a feature that was introduced on Cisco routers around 1996 that provides the ability to collect IP network traffic as it enters or exits an interface. By analyzing the data provided by NetFlow, a network administrator can determine things such as the source and destination traffic, class of service, and the causes of congestion. A typical flow monitoring setup consists of three main components:

sFlow, short for "sampled flow", is an industry standard for packet export at Layer 2 of the OSI model. sFlow was originally developed by InMon Corp. It provides a means for exporting truncated packets, together with interface counters for the purpose of network monitoring. Maintenance of the protocol is performed by the sFlow.org consortium, the authoritative source of the sFlow protocol specifications. The current version of sFlow is v5.

IP multicast is a method of sending Internet Protocol (IP) datagrams to a group of interested receivers in a single transmission. It is the IP-specific form of multicast and is used for streaming media and other network applications. It uses specially reserved multicast address blocks in IPv4 and IPv6.

The Stream Control Transmission Protocol (SCTP) has a simpler basic packet structure than TCP. Each consists of two basic sections:

  1. The common header, which occupies the first 12 bytes. In the adjacent diagram, this header is highlighted in blue.
  2. The data chunks, which form the remaining portion of the packet. In the diagram, the first chunk is highlighted in green and the last of N chunks (Chunk N) is highlighted in red. There are several types, including payload data and different control messages.

In packet switching networks, traffic flow, packet flow or network flow is a sequence of packets from a source computer to a destination, which may be another host, a multicast group, or a broadcast domain. RFC 2722 defines traffic flow as "an artificial logical equivalent to a call or connection." RFC 3697 defines traffic flow as "a sequence of packets sent from a particular source to a particular unicast, anycast, or multicast destination that the source desires to label as a flow. A flow could consist of all packets in a specific transport connection or a media stream. However, a flow is not necessarily 1:1 mapped to a transport connection." Flow is also defined in RFC 3917 as "a set of IP packets passing an observation point in the network during a certain time interval." Packet flow temporal efficiency can be affected by one-way delay (OWD) that is described as a combination of the following components:

An IPv6 packet is the smallest message entity exchanged using Internet Protocol version 6 (IPv6). Packets consist of control information for addressing and routing and a payload of user data. The control information in IPv6 packets is subdivided into a mandatory fixed header and optional extension headers. The payload of an IPv6 packet is typically a datagram or segment of the higher-level transport layer protocol, but may be data for an internet layer or link layer instead.

A communication protocol is a system of rules that allows two or more entities of a communications system to transmit information via any variation of a physical quantity. The protocol defines the rules, syntax, semantics, and synchronization of communication and possible error recovery methods. Protocols may be implemented by hardware, software, or a combination of both.

The Stream Control Transmission Protocol (SCTP) is a computer networking communications protocol in the transport layer of the Internet protocol suite. Originally intended for Signaling System 7 (SS7) message transport in telecommunication, the protocol provides the message-oriented feature of the User Datagram Protocol (UDP), while ensuring reliable, in-sequence transport of messages with congestion control like the Transmission Control Protocol (TCP). Unlike UDP and TCP, the protocol supports multihoming and redundant paths to increase resilience and reliability.

NACK-Oriented Reliable Multicast (NORM) is a transport layer Internet protocol designed to provide reliable transport in multicast groups in data networks. It is formally defined by the Internet Engineering Task Force (IETF) in Request for Comments (RFC) 5740, which was published in November 2009.

References

  1. Hofstede, Rick; Celeda, Pavel; Trammell, Brian; Drago, Idilio; Sadre, Ramin; Sperotto, Anna; Pras, Aiko (2014). "Flow Monitoring Explained: From Packet Capture to Data Analysis with NetFlow and IPFIX". IEEE Communications Surveys & Tutorials. 16 (4): 2037–2064. doi:10.1109/COMST.2014.2321898. S2CID   14042725.
  2. IEEE communications magazine,[ dead link ]