Original author(s) | Thomas Graf, Daniel Borkmann, André Martins, Madhusudan Challa [1] |
---|---|
Developer(s) | Open source community, Isovalent, Google, Datadog, Red Hat, Cloud Native Computing Foundation [2] |
Initial release | December 16, 2015 [1] |
Stable release | |
Repository | github |
Written in | Go, eBPF, C, C++ |
Operating system | Linux, Windows [4] |
Platform | x86-64, ARM [5] |
Available in | English |
Type | Cloud-native Networking, Security, Observability |
License | Apache License 2.0, Dual GPL-2.0-only or BSD-2-clause for eBPF [6] |
Website | cilium.io |
Cilium is a cloud native technology for networking, observability, and security. [1] It is based on the kernel technology eBPF, originally for better networking performance, and now leverages many additional features for different use cases. The core networking component has evolved from only providing a flat Layer 3 network for containers to including advanced networking features, like BGP and Service mesh, within a Kubernetes cluster, across multiple clusters, and connecting with the world outside Kubernetes. [1] Hubble was created as the network observability component and Tetragon was later added for security observability and runtime enforcement. [1] Cilium runs on Linux and is one of the first eBPF applications being ported to Microsoft Windows through the eBPF on Windows project. [7]
Evolution from Networking CNI (Container Network Interface)
Cilium began as a networking CNI [8] for container workloads. It was originally IPv6 only and supported multiple container orchestrators, like Kubernetes. The original vision for Cilium was to build an intent and identity-based high-performance container networking platform. [9] As the cloud native ecosystem expanded, Cilium added new projects and features to address new problems in the space.
The table below summarises some of the most significant milestones of this evolution:
Cilium was accepted into the Cloud Native Computing Foundation on October 13th, 2021 as an incubation-level project. It applied to become a graduated project on October 27th 2022. [19] It became a Graduated project one year later. Cilium is one of the fastest-moving projects in the CNCF ecosystem. [30]
Cilium has been adopted by many large-scale production users, including over 100 that have stated it publicly, [31] for example:
Cilium is the CNI for many cloud providers including Alibaba, [45] APPUiO, [46] Azure, [47] AWS, [16] DigitalOcean, [48] Exoscale, [49] Google Cloud, [15] Hetzner, [50] and Tencent Cloud. [51]
Cilium began as a container networking project. With the growth of Kubernetes and container orchestration, Cilium became a CNI, [8] providing basic things like configuring container network interfaces and Pod to Pod connectivity. From the beginning, Cilium based its networking on eBPF rather than iptables or IPVS, betting that eBPF would become the future of cloud native networking. [52]
Cilium’s eBPF based dataplane provides a simple flat Layer 3 network with the ability to span multiple clusters in either a native routing or overlay mode with Cilium Cluster Mesh. It is Layer 7-protocol aware and can enforce network policies on Layer 3 to Layer 7 and with FQDN using an identity-based security model that is decoupled from network addressing.
Cilium implements distributed load balancing for traffic between Pods and to external services, and is able to fully replace kube-proxy, [53] using XDP, socket-based load-balancing and efficient hash tables in eBPF. It also supports advanced functionality like integrated ingress and egress gateways, [54] bandwidth management, a stand-alone load balancer, and service mesh. [55]
Cilium is the first CNI to support advanced kernel features such as BBR TCP congestion control [56] and BIG TCP [57] for Kubernetes Pods. [58]
Hubble is the observability, service map, and UI of Cilium which is shipped with the CNI. [59] [60] It can be used to observe individual network packet flows, view network policy decisions to allow or block traffic, and build up service maps showing how Kubernetes services are communicating. [61] Hubble can export this data to Prometheus, OpenTelemetry, Grafana, and Fluentd for further analysis of Layer 3/4 and Layer 7 metrics. [62]
Tetragon is the security observability and runtime enforcement project of Cilium. [63] Tetragon is a flexible Kubernetes-aware security observability and runtime enforcement tool that applies policy and filtering directly with eBPF. It allows users to monitor and observe the complete lifecycle of every process execution on their machine, translate policies for file monitoring, network observability, container security, and more into eBPF programs, and do synchronous monitoring, filtering, and enforcement completely in the kernel.
ebpf-go is a pure-Go library to interact with the eBPF subsystem in the Linux kernel. [64] It has minimal external dependencies, emphasises reliability and compatibility, and is widely deployed in production.
pwru ("Packet, where are you?") is an eBPF-based tool for tracing network packets in the Linux kernel with advanced filtering capabilities. It allows fine-grained introspection of kernel state to facilitate debugging network connectivity issues. Under the hood, pwru attaches eBPF debugging programs to all Linux kernel functions which are responsible for processing network packets.
This gives a user finer-grained view into a packet processing in the kernel than with tcpdump, Wireshark, or more traditional tools. Also, it can show packet metadata such as network namespace, processing timestamp, internal kernel packet representation fields, and more.
Cilium began as a networking project and has many features that allow it to provide a consistent connectivity experience from Kubernetes workloads to virtual machines and physical servers running in the cloud, on-premises, or at the edge. Some of these include:
Being in the kernel, eBPF has complete visibility of everything that is happening on a machine. Cilium leverages this with the following features:
eBPF can stop events in the kernel for security. Cilium projects leverage this through the following features:
The chart below visualises the period for which each Cilium community maintained release is/was supported:
Cilium's official website lists online forums, messaging platforms, and in-person meetups for the Cilium user and developer community.
Conferences dedicated to Cilium development in the past have included:
The Cilium community releases an annual report to cover how the community developed over the course of the year:
seccomp is a computer security facility in the Linux kernel. seccomp allows a process to make a one-way transition into a "secure" state where it cannot make any system calls except exit
, sigreturn
, read
and write
to already-open file descriptors. Should it attempt any other system calls, the kernel will either just log the event or terminate the process with SIGKILL or SIGSYS. In this sense, it does not virtualize the system's resources but isolates the process from them entirely.
Oracle Linux is a Linux distribution packaged and freely distributed by Oracle, available partially under the GNU General Public License since late 2006. It is compiled from Red Hat Enterprise Linux (RHEL) source code, replacing Red Hat branding with Oracle's. It is also used by Oracle Cloud and Oracle Engineered Systems such as Oracle Exadata and others.
The Berkeley Packet Filter is a network tap and packet filter which permits computer network packets to be captured and filtered at the operating system level. It provides a raw interface to data link layers, permitting raw link-layer packets to be sent and received, and allows a userspace process to supply a filter program that specifies which packets it wants to receive. For example, a tcpdump process may want to receive only packets that initiate a TCP connection. BPF returns only packets that pass the filter that the process supplies. This avoids copying unwanted packets from the operating system kernel to the process, greatly improving performance. The filter program is in the form of instructions for a virtual machine, which are interpreted, or compiled into machine code by a just-in-time (JIT) mechanism and executed, in the kernel.
OpenShift is a family of containerization software products developed by Red Hat. Its flagship product is the OpenShift Container Platform — a hybrid cloud platform as a service built around Linux containers orchestrated and managed by Kubernetes on a foundation of Red Hat Enterprise Linux. The family's other products provide this platform through different environments: OKD serves as the community-driven upstream, Several deployment methods are available including self-managed, cloud native under ROSA, ARO and RHOIC on AWS, Azure, and IBM Cloud respectively, OpenShift Online as software as a service, and OpenShift Dedicated as a managed service.
Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. The service has both free and premium tiers. The software that hosts the containers is called Docker Engine. It was first released in 2013 and is developed by Docker, Inc.
Kubernetes is an open-source container orchestration system for automating software deployment, scaling, and management. Originally designed by Google, the project is now maintained by a worldwide community of contributors, and the trademark is held by the Cloud Native Computing Foundation.
Mirantis Inc. is a Campbell, California, based B2B open source cloud computing software and services company. Its primary container and cloud management products, part of the Mirantis Cloud Native Platform suite of products, are Mirantis Container Cloud and Mirantis Kubernetes Engine. The company focuses on the development and support of container and cloud infrastructure management platforms based on Kubernetes and OpenStack. The company was founded in 1999 by Alex Freedland and Boris Renski. It was one of the founding members of the OpenStack Foundation, a non-profit corporate entity established in September, 2012 to promote OpenStack software and its community. Mirantis has been an active member of the Cloud Native Computing Foundation since 2016.
Dan Kohn was an American serial entrepreneur and nonprofit executive who led the Linux Foundation's Public Health initiative. He was the executive director at Cloud Native Computing Foundation (CNCF), which sustains and integrates open source cloud software including Kubernetes and Fluentd, through 2020. The first company he founded, NetMarket, conducted the first secure commercial transaction on the web in 1994.
Container Linux is a discontinued open-source lightweight operating system based on the Linux kernel and designed for providing infrastructure for clustered deployments. One of its focuses was scalability. As an operating system, Container Linux provided only the minimal functionality required for deploying applications inside software containers, together with built-in mechanisms for service discovery and configuration sharing.
TiDB is an open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads. Designed to be MySQL compatible, it is developed and supported primarily by PingCAP and licensed under Apache 2.0. It is also available as a paid product. TiDB drew its initial design inspiration from Google's Spanner and F1 papers.
XDP is an eBPF-based high-performance data path used to send and receive network packets at high rates by bypassing most of the operating system networking stack. It is merged in the Linux kernel since version 4.8. This implementation is licensed under GPL. Large technology firms including Amazon, Google and Intel support its development. Microsoft released their free and open source implementation XDP for Windows in May 2022. It is licensed under MIT License.
Kubeflow is an open-source platform for machine learning and MLOps on Kubernetes introduced by Google. The different stages in a typical machine learning lifecycle are represented with different software components in Kubeflow, including model development (Kubeflow Notebooks), model training (Kubeflow Pipelines,Kubeflow Training Operator), model serving (KServe), and automated machine learning (Katib).
In software architecture, a service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between services or microservices using a proxy.
Cloud native computing is an approach in software development that utilizes cloud computing to "build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds". These technologies, such as containers, microservices, serverless functions, cloud native processors and immutable infrastructure, deployed via declarative code are common elements of this architectural style. Cloud native technologies focus on minimizing users' operational burden.
The Cloud Native Computing Foundation (CNCF) is a Linux Foundation project that was started in 2015 to help advance container technology and align the tech industry around its evolution.
A cloud-native network function (CNF) is a software-implementation of a function, or application, traditionally performed on a physical device, but which runs inside Linux containers. The features that differ CNFs from VNFs, one of the components of network function virtualization, is the approach in their orchestration.
Open Service Mesh (OSM) was a free and open source cloud native service mesh developed by Microsoft that ran on Kubernetes.
io_uring is a Linux kernel system call interface for storage device asynchronous I/O operations addressing performance issues with similar interfaces provided by functions like read
/write
or aio_read
/aio_write
etc. for operations on data accessed by file descriptors.
Azure Linux, previously known as CBL-Mariner, is a free and open-source Linux distribution that Microsoft has developed. It is the base container OS for Microsoft Azure services and the graphical component of WSL 2.
eBPF is a technology that can run programs in a privileged context such as the operating system kernel. It is the successor to the Berkeley Packet Filter filtering mechanism in Linux and is also used in non-networking parts of the Linux kernel as well.
{{cite web}}
: CS1 maint: multiple names: authors list (link)