Data Plane Development Kit

Last updated

DPDK
Stable release
24.03 / 28 March 2024 (2024-03-28) [1]
Repository git.dpdk.org
Written in C
Operating system Linux, FreeBSD, Windows
Type Packet processing
License BSD
Website www.dpdk.org

The Data Plane Development Kit (DPDK) is an open source software project managed by the Linux Foundation. It provides a set of data plane libraries and network interface controller polling-mode drivers for offloading TCP packet processing from the operating system kernel to processes running in user space. This offloading achieves higher computing efficiency and higher packet throughput than is possible using the interrupt-driven processing provided in the kernel.

Contents

DPDK provides a programming framework for x86, ARM, and PowerPC processors and enables faster development of high speed data packet networking applications. [2] [3] It scales from mobile processors, such as Intel Atom, to server-grade processors, such as Intel Xeon. It supports instruction set architectures such as Intel, IBM POWER8, EZchip, and ARM. [4] It is provided and supported under the open-source [5] BSD license.

DPDK was created by Intel engineer Venky Venkatesan, who is affectionately known as "The Father of DPDK." He died in 2018 after a long battle with cancer. [6]

Overview

The DPDK framework creates a set of libraries for specific hardware/software environments through the creation of an Environment Abstraction Layer (EAL). [7] [8] The EAL hides the environment specifics and provides a standard programming interface to libraries, available hardware accelerators and other hardware and operating system (Linux, FreeBSD) elements. Once the EAL is created for a specific environment, developers link to the library to create their applications. For instance, EAL provides the frameworks to support Linux, FreeBSD, Intel IA-32 or 64-bit, IBM POWER9 and ARM 32- or 64-bit.

The EAL also provides additional services including time references, generic bus access, trace and debug functions and alarm operations.

Using DPDK libraries one can implement a low overhead run-to-completion, pipeline or staged, event driven, or hybrid model completely in userspace eliminating kernel and kernel to user copy. Hardware assists from NIC/Regex/Accelerators, libraries enhanced to make use of Intelligence Storage Acceleration (ISA) [9] for bulk performance and accessing devices via polling helps to eliminate the performance overhead of interrupt too. Hugepages are used for large memory pool allocation, to decrease the amount of lookups and page management. [10]

The DPDK also includes software examples that highlight best practices for software architecture, tips for data structure design and storage, application profiling and performance tuning utilities and tips that address common network performance deficits.

Libraries

The DPDK includes data plane libraries and optimized network interface controller (NIC) drivers for the following: [11]

All libraries are stored in the dpdk/lib/librte_* directories

Plugins

The DPDK includes drivers for many hardware types. [12] There have been some additional out-of-tree plugin drivers in the past, which are now considered deprecated.

Environment

The DPDK was originally designed to run using a bare-metal mode which is currently deprecated. DPDK's EAL provides support for Linux or FreeBSD userland application.[ citation needed ]

EAL can be extended in order to support any processors.[ citation needed ]

Ecosystem

DPDK is now an open-source project under the Linux Foundation, supported by many companies. DPDK is governed by a Governing Board. The technical activities are overseen by a Technical Board. [13] Beside Intel, which is a contributor to the DPDK, several other vendors also support the DPDK within their products and some offer additional training, support and professional services. The list of vendors who have announced DPDK support includes: 6WIND, [14] ALTEN Calsoft Labs, [15] [16] Advantech, [17] Brocade, [18] Big Switch Networks, Mellanox Technologies, [19] Radisys, [20] Tieto, [21] Wind River, [22] Lanner Inc. [23] and NXP. [24]

Projects

The pfSense project published a road map on 25 February 2015, in which developer Jim Thompson announced the rewriting of the pfSense core—including pf, network packet forwarding and shaping, link bonding, IPsec—using DPDK: "We have a goal of being able to forward, with packet filtering at rates of at least 14.88 Mpps. This is 'line rate' on a 10 Gbps interface. There is simply no way to use today's FreeBSD (or Linux) in-kernel stacks for this type of load." [25]

Open vSwitch (OVS) has a limited set of features running userland that can be leveraged to bypass the Linux kernel OVS processing. This use case of OVS with DPDK userland is usually named OVS-DPDK. It is mostly deployed with OpenStack Neutron but it assumes that many features and software-defined networking (SDN) capabilities of Openstack are disabled. For instance, when OVS-DPDK is used, Neutron provides a lower level of security than when OVS kernel is used (no stateful firewalling, less security group).

The FD.IO VPP platform is an extensible framework that provides out-of-the-box production quality switch/router functionality. It is the open source version of Cisco's Vector Packet Processing (VPP) technology: a high performance, packet-processing stack that can run on commodity CPUs, and can leverage the Poll Mode Drivers for both NICs and cryptographic acceleration hardware and libraries. [26] VPP supports and uses the DPDK library. [27]

TRex is an open source traffic generator using DPDK. It generates L4–7 traffic based on pre-processing and smart replay of real traffic templates. TRex amplifies both client and server side traffic and can scale to 200 Gbit/s with one UCS using Intel XL710. TRex also supports multiple streams, ability to change any packet field and provides per stream statistics, latency and jitter. [28]

DTS (DPDK Test Suite) is a Python-based framework for functional tests and benchmarks. It is an open-source project, started in 2014, and is hosted on dpdk.org. It supports both software traffic generators like Scapy and dpdk-pktgen, and a hardware traffic generator like Ixia. [29]

DPDK has support for several SRIOV network drivers, enabling creating a PF (Physical Function) and VFs, and also to launch VMs (like QEMU VMs) and assign VFs to them using PCI Passthrough [30]

DDP (Dynamic Device Personalization) is one of the new advanced features implemented with DPDK. It allows you to load firmware for a device dynamically, without resetting the host. [31]

See also

Related Research Articles

<span class="mw-page-title-main">Operating system</span> Software that manages computer hardware resources

An operating system (OS) is system software that manages computer hardware and software resources, and provides common services for computer programs.

<span class="mw-page-title-main">Network interface controller</span> Hardware component that connects a computer to a network

A network interface controller is a computer hardware component that connects a computer to a computer network.

In computing, Physical Address Extension (PAE), sometimes referred to as Page Address Extension, is a memory management feature for the x86 architecture. PAE was first introduced by Intel in the Pentium Pro, and later by AMD in the Athlon processor. It defines a page table hierarchy of three levels (instead of two), with table entries of 64 bits each instead of 32, allowing these CPUs to directly access a physical address space larger than 4 gigabytes (232 bytes).

<span class="mw-page-title-main">Xen</span> Type-1 hypervisor

Xen is a free and open-source type-1 hypervisor, providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently. It was originally developed by the University of Cambridge Computer Laboratory and is now being developed by the Linux Foundation with support from Intel, Citrix, Arm Ltd, Huawei, AWS, Alibaba Cloud, AMD, Bitdefender and epam.

TCP offload engine (TOE) is a technology used in some network interface cards (NIC) to offload processing of the entire TCP/IP stack to the network controller. It is primarily used with high-speed network interfaces, such as gigabit Ethernet and 10 Gigabit Ethernet, where processing overhead of the network stack becomes significant. TOEs are often used as a way to reduce the overhead associated with Internet Protocol (IP) storage protocols such as iSCSI and Network File System (NFS).

<span class="mw-page-title-main">UEFI</span> Operating system and firmware specification

Unified Extensible Firmware Interface is a specification that defines the architecture of the platform firmware used for booting the computer hardware and its interface for interaction with the operating system. Examples of firmware that implement the specification are AMI Aptio, Phoenix SecureCore, TianoCore EDK II, InsydeH2O. UEFI replaces the BIOS which was present in the boot ROM of all personal computers that are IBM PC compatible, although it can provide backwards compatibility with the BIOS using CSM booting. Intel developed the original Extensible Firmware Interface (EFI) specification. Some of the EFI's practices and data formats mirror those of Microsoft Windows. In 2005, UEFI deprecated EFI 1.10.

<span class="mw-page-title-main">Free and open-source graphics device driver</span> Software that controls computer-graphics hardware

A free and open-source graphics device driver is a software stack which controls computer-graphics hardware and supports graphics-rendering application programming interfaces (APIs) and is released under a free and open-source software license. Graphics device drivers are written for specific hardware to work within a specific operating system kernel and to support a range of APIs used by applications to access the graphics hardware. They may also control output to the display if the display driver is part of the graphics hardware. Most free and open-source graphics device drivers are developed by the Mesa project. The driver is made up of a compiler, a rendering API, and software which manages access to the graphics hardware.

Advanced Configuration and Power Interface (ACPI) is an open standard that operating systems can use to discover and configure computer hardware components, to perform power management, auto configuration, and status monitoring. It was first released in December 1996. ACPI aims to replace Advanced Power Management (APM), the MultiProcessor Specification, and the Plug and Play BIOS (PnP) Specification. ACPI brings power management under the control of the operating system, as opposed to the previous BIOS-centric system that relied on platform-specific firmware to determine power management and configuration policies. The specification is central to the Operating System-directed configuration and Power Management (OSPM) system. ACPI defines hardware abstraction interfaces between the device's firmware, the computer hardware components, and the operating systems.

In computer security, executable-space protection marks memory regions as non-executable, such that an attempt to execute machine code in these regions will cause an exception. It makes use of hardware features such as the NX bit, or in some cases software emulation of those features. However, technologies that emulate or supply an NX bit will usually impose a measurable overhead while using a hardware-supplied NX bit imposes no measurable overhead.

Netlink is a socket family used for inter-process communication (IPC) between both the kernel and userspace processes, and between different userspace processes, in a way similar to the Unix domain sockets available on certain Unix-like operating systems, including its original incarnation as a Linux kernel interface, as well as in the form of a later implementation on FreeBSD. Similarly to the Unix domain sockets, and unlike INET sockets, Netlink communication cannot traverse host boundaries. However, while the Unix domain sockets use the file system namespace, Netlink sockets are usually addressed by process identifiers (PIDs).

In the context of free and open-source software, proprietary software only available as a binary executable is referred to as a blob or binary blob. The term usually refers to a device driver module loaded into the kernel of an open-source operating system, and is sometimes also applied to code running outside the kernel, such as system firmware images, microcode updates, or userland programs. The term blob was first used in database management systems to describe a collection of binary data stored as a single entity.

The Berkeley Packet Filter is a network tap and packet filter which permits computer network packets to be captured and filtered at the operating system level. It provides a raw interface to data link layers, permitting raw link-layer packets to be sent and received, and allows a userspace process to supply a filter program that specifies which packets it wants to receive. For example, a tcpdump process may want to receive only packets that initiate a TCP connection. BPF returns only packets that pass the filter that the process supplies. This avoids copying unwanted packets from the operating system kernel to the process, greatly improving performance. The filter program is in the form of instructions for a virtual machine, which are interpreted, or compiled into machine code by a just-in-time (JIT) mechanism and executed, in the kernel.

<span class="mw-page-title-main">Junos OS</span> Real-time operating system (RTOS) software

Junos OS is a FreeBSD-based network operating system used in Juniper Networks routing, switching and security devices.

<span class="mw-page-title-main">Kernel (operating system)</span> Core of a computer operating system

The kernel is a computer program at the core of a computer's operating system and generally has complete control over everything in the system. The kernel is also responsible for preventing and mitigating conflicts between different processes. It is the portion of the operating system code that is always resident in memory and facilitates interactions between hardware and software components. A full kernel controls all hardware resources via device drivers, arbitrates conflicts between processes concerning such resources, and optimizes the utilization of common resources e.g. CPU & cache usage, file systems, and network sockets. On most systems, the kernel is one of the first programs loaded on startup. It handles the rest of startup as well as memory, peripherals, and input/output (I/O) requests from software, translating them into data-processing instructions for the central processing unit.

<span class="mw-page-title-main">NetBSD</span> Free and open-source Unix-like operating system

NetBSD is a free and open-source Unix operating system based on the Berkeley Software Distribution (BSD). It was the first open-source BSD descendant officially released after 386BSD was forked. It continues to be actively developed and is available for many platforms, including servers, desktops, handheld devices, and embedded systems.

NVM Express (NVMe) or Non-Volatile Memory Host Controller Interface Specification (NVMHCIS) is an open, logical-device interface specification for accessing a computer's non-volatile storage media usually attached via the PCI Express bus. The initial NVM stands for non-volatile memory, which is often NAND flash memory that comes in several physical form factors, including solid-state drives (SSDs), PCIe add-in cards, and M.2 cards, the successor to mSATA cards. NVM Express, as a logical-device interface, has been designed to capitalize on the low latency and internal parallelism of solid-state storage devices.

<span class="mw-page-title-main">Open vSwitch</span> Virtual network switch

Open vSwitch, sometimes abbreviated as OVS, is an open-source implementation of a distributed virtual multilayer switch. The main purpose of Open vSwitch is to provide a switching stack for hardware virtualization environments, while supporting multiple protocols and standards used in computer networks.

WireGuard is a communication protocol and free and open-source software that implements encrypted virtual private networks (VPNs), and was designed with the goals of ease of use, high speed performance, and low attack surface. It aims to be smaller and better performing than IPsec and OpenVPN, two common tunneling protocols. The WireGuard protocol passes traffic over UDP.

XDP is an eBPF-based high-performance data path used to send and receive network packets at high rates by bypassing most of the operating system networking stack. It is merged in the Linux kernel since version 4.8. This implementation is licensed under GPL. Large technology firms including Amazon, Google and Intel support its development. Microsoft released their free and open source implementation XDP for Windows in May 2022. It is licensed under MIT License.

References

  1. ↑ "DPDK". core.dpdk.org.
  2. ↑ Simon Stanley,All Change for Packet Processing Archived 21 August 2016 at the Wayback Machine , Heavy Reading, 2013
  3. ↑ Shamus McGillicudy, DPDK, switch and server ref designs push SDN ecosystem forward Archived 18 September 2015 at the Wayback Machine , SearchSDN, April 2013
  4. ↑ "DPDK: Data Plane Development Kit – What it is". dpdk.org. Retrieved 29 March 2016.
  5. ↑ Simon Stanley,DPDK Goes Open-Source, Intel Embedded Community, May 2013
  6. ↑ "In Loving Memory: Venky Venkatesan, "The Father of DPDK"". DPDK. Retrieved 12 May 2022.
  7. ↑ DPDK Docs, Data Plane Development Kit: Programmers Guide, September 2017
  8. ↑ "How can DPDK access devices from user space?". CodiLime. 22 August 2019. Retrieved 28 August 2019.
  9. ↑ "How DPDK can Deflate your Data! | SNIA". www.snia.org. Retrieved 21 June 2021.
  10. ↑ PANTHEON.tech (9 January 2020). "VPP 105: Memory Management & DPDK APIs". PANTHEON.tech. Retrieved 21 June 2021.
  11. ↑ Intel Communications Infrastructure Division, Data Plane Development Kit Overview, December 2012
  12. ↑ "Supported NICS"
  13. ↑ DPDK Technical Board
  14. ↑ PRWeb, 6WIND Extends Portable Packet Processing Software to Support Data Plane Development Kit Archived 26 March 2016 at the Wayback Machine , September 2011
  15. ↑ Calsoft Labs to offer professional services and support for Data Plane Development Kit, ALTEN Calsoft Labs, 18 February 2014, archived from the original on 4 March 2016, retrieved 28 October 2014
  16. ↑ https://builders.intel.com/docs/ALTEN-Calsoft-Labs-and-Intel-Improving-Network-Agility-with-Virtual-CPE.pdf [ bare URL PDF ]
  17. ↑ COTS Journal, ATCA Blade Serves Up Xeon E5-2600 Processor Archived 4 March 2016 at the Wayback Machine , June 2012
  18. ↑ Brocade vRouter
  19. ↑ Technologies, Mellanox. "Data Plane Development Kit (DPDK) | Poll Mode Driver (PMD)". Mellanox. Retrieved 10 February 2021.
  20. ↑ MarketWatch, Radisys Delivers Industry's First 40G Solution for Data Plane Development Kit, September 2012
  21. ↑ Tieto, Tieto provides professional software services and support for the Data Plane Development Kit, February 2012
  22. ↑ Reuters, Wind River Delivers Support and Services for Data Plane Development Kit for High-Performance Packet Processing, May 2012
  23. ↑ Get Flying with the Data Plane Development Kit, Lanner Electronics Inc., 20 February 2013, archived from the original on 6 December 2015, retrieved 11 July 2013
  24. ↑ NXP Platform support in Data Plane Development Kit
  25. ↑ Thompson, Jim (25 February 2015). "Further (a roadmap for pfSense)". blog.pfsense.org. Electric Sheep Fencing LLC. Retrieved 21 April 2015.
  26. ↑ "VPP fd.io". wiki.fd.io. Linux Foundation . Retrieved 29 December 2016.
  27. ↑ PANTHEON.tech (9 January 2020). "VPP 105: Memory Management & DPDK APIs". PANTHEON.tech. Retrieved 21 June 2021.
  28. ↑ "TRex". Cisco Systems . Retrieved 29 December 2016.
  29. ↑ Rami Rosen, Network acceleration with DPDK, lwn.net, July 2017
  30. ↑ Rami Rosen, Userspace Networking with DPDK, Linux Journal, April 2018.
  31. ↑ Rami Rosen, DPDK in depth, Kernel TLV, August 2018
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.