 | This article only references primary sources.(July 2024) |
 | |
| Original author(s) | Ask Solem Hoel |
|---|---|
| Stable release | 5.4.0 / April 18, 2024 |
| Repository | |
| Written in | Python |
| Platform | Cross-platform |
| Available in | Python |
| Type | Message-oriented middleware |
| License | BSD License |
| Website | docs |
Celery is an open source asynchronous task queue or job queue which is based on distributed message passing. While it supports scheduling, its focus is on operations in real time. [1]
The execution units, called tasks, are executed concurrently on one or more worker nodes using multiprocessing, eventlet [2] or gevent. [3] Tasks can execute asynchronously (in the background) or synchronously (wait until ready). Celery is used in production systems, for services such as Instagram, to process millions of tasks every day. [1]
Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks. [4] There is also a Ruby-Client called RCelery, [5] a PHP client, [6] a Go client, [7] a Rust client, [8] and a Node.js client. [9]
Celery requires a message broker to run. As of October 2024, Redis and RabbitMQ are supported and actively maintained and monitored. Amazon SQS is supported and maintained but does not support worker inspection and management at runtime, while Zookeeper and Kafka are currently in experimental development. [10]
Programming languages can be grouped by the number and types of paradigms supported.
Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. Message-oriented middleware is in contrast to streaming-oriented middleware where data is communicated as a sequence of bytes with no explicit message boundaries. Note that streaming protocols are almost always built above protocols using discrete messages such as frames (Ethernet), datagrams (UDP), packets (IP), cells (ATM), et al.
Markdown is a lightweight markup language for creating formatted text using a plain-text editor. John Gruber created Markdown in 2004 as an easy-to-read markup language. Markdown is widely used for blogging and instant messaging, and also used elsewhere in online forums, collaborative software, documentation pages, and readme files.
The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing, reliability and security.
This is a comparison of notable free and open-source configuration management software, suitable for tasks like server configuration, orchestration and infrastructure as code typically performed by a system administrator.
Jami is a SIP-compatible distributed peer-to-peer softphone and SIP-based instant messenger for Linux, Microsoft Windows, macOS, iOS, and Android. Jami was developed and maintained by the Canadian company Savoir-faire Linux, and with the help of a global community of users and contributors, Jami positions itself as a potential free Skype replacement.
HornetQ is an open-source asynchronous messaging project from JBoss. It is an example of Message-oriented middleware. HornetQ is an open source project to build a multi-protocol, embeddable, very high performance, clustered, asynchronous messaging system. During much of its development, the HornetQ code base was developed under the name JBoss Messaging 2.0.
Node.js is a cross-platform, open-source JavaScript runtime environment that can run on Windows, Linux, Unix, macOS, and more. Node.js runs on the V8 JavaScript engine, and executes JavaScript code outside a web browser.
ZeroMQ is an asynchronous messaging library, aimed at use in distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can run without a dedicated message broker; the zero in the name is for zero broker. The library's API is designed to resemble Berkeley sockets.
Apache ActiveMQ is an open source message broker written in Java together with a full Java Message Service (JMS) client. It provides "Enterprise Features" which in this case means fostering the communication from more than one client or server. Supported clients include Java via JMS 1.1 as well as several other "cross language" clients. The communication is managed with features such as computer clustering and ability to use any database as a JMS persistence provider besides virtual memory, cache, and journal persistency.
MQTT is a lightweight, publish–subscribe, machine-to-machine network protocol for message queue/message queuing service. It is designed for connections with remote locations that have devices with resource constraints or limited network bandwidth, such as in the Internet of things (IoT). It must run over a transport protocol that provides ordered, lossless, bi-directional connections—typically, TCP/IP. It is an open OASIS standard and an ISO recommendation.
FuelPHP is an open-source web application framework written in PHP which implements the HMVC pattern.
Apache Kafka is a distributed event store and stream-processing platform. It is an open-source system developed by the Apache Software Foundation written in Java and Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect to external systems via Kafka Connect, and provides the Kafka Streams libraries for stream processing applications. Kafka uses a binary TCP-based protocol that is optimized for efficiency and relies on a "message set" abstraction that naturally groups messages together to reduce the overhead of the network roundtrip. This "leads to larger network packets, larger sequential disk operations, contiguous memory blocks [...] which allows Kafka to turn a bursty stream of random message writes into linear writes."
Tox is a peer-to-peer instant-messaging and video-calling protocol that offers end-to-end encryption. The stated goal of the project is to provide secure yet easily accessible communication for everyone. A reference implementation of the protocol is published as free and open-source software under the terms of the GNU GPL-3.0-or-later.
MQTT is an ISO standard publish–subscribe-based messaging protocol. It works on top of the Internet protocol suite TCP/IP. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker.
Deno is a runtime for JavaScript, TypeScript, and WebAssembly that is based on the V8 JavaScript engine and the Rust programming language. Deno was co-created by Ryan Dahl, the creator of Node.js and Bert Belder.
Tokio is a software library for the Rust programming language. It provides a runtime and functions that enable the use of asynchronous I/O, allowing for concurrency in regards to task completion.
 | This free and open-source software article is a stub. You can help Wikipedia by expanding it. |
