Dead letter queue

Last updated

In message queueing a dead letter queue (DLQ) is a service implementation to store messages that the messaging system cannot or should not deliver. [1] Although implementation-specific, messages can be routed to the DLQ for the following reasons:

  1. The message is sent to a queue that does not exist. [2] [3]
  2. The maximum queue length is exceeded.
  3. The message exceeds the size limit.
  4. The message expires because it reached the TTL (time to live) [4]
  5. The message is rejected by another queue exchange. [5]
  6. The message has been read and rejected too many times. [6]

Routing these messages to a dead letter queue enables analysis of common fault patterns and potential software problems. [7] If a message consumer receives a message that it considers invalid, it can instead forward it an Invalid Message Channel, [8] allowing a separation between application-level faults and delivery failures.

Queueing systems that incorporate dead letter queues include Amazon EventBridge, [9] Amazon Simple Queue Service, [7] Apache ActiveMQ, Google Cloud Pub/Sub, [10] HornetQ, Microsoft Message Queuing, [2] Microsoft Azure Event Grid and Azure Service Bus, [11] WebSphere MQ, [12] Solace PubSub+, [13] Rabbit MQ, [5] Confluent Cloud [14] and Apache Pulsar. [15] [16]

See also

Related Research Articles

The Jakarta Messaging API is a Java application programming interface (API) for message-oriented middleware. It provides generic messaging models, able to handle the producer–consumer problem, that can be used to facilitate the sending and receiving of messages between software systems. Jakarta Messaging is a part of Jakarta EE and was originally defined by a specification developed at Sun Microsystems before being guided by the Java Community Process.

In computer science, message queues and mailboxes are software-engineering components typically used for inter-process communication (IPC), or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content. Group communication systems provide similar kinds of functionality.

Microsoft Message Queuing (MSMQ) is a message queue implementation developed by Microsoft and deployed in its Windows Server operating systems since Windows NT 4 and Windows 95. Windows Server 2016 and Windows 10 also includes this component. In addition to its mainstream server platform support, MSMQ has been incorporated into Microsoft Embedded platforms since 1999 and the release of Windows CE 3.0.

IBM MQ is a family of message-oriented middleware products that IBM launched in December 1993. It was originally called MQSeries, and was renamed WebSphere MQ in 2002 to join the suite of WebSphere products. In April 2014, it was renamed IBM MQ. The products that are included in the MQ family are IBM MQ, IBM MQ Advanced, IBM MQ Appliance, IBM MQ for z/OS, and IBM MQ on IBM Cloud. IBM MQ also has containerised deployment options.

The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing, reliability and security.

<span class="mw-page-title-main">Message broker</span> Computer program module

A message broker is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally-defined messages. Message brokers are a building block of message-oriented middleware (MOM) but are typically not a replacement for traditional middleware like MOM and remote procedure call (RPC).

<span class="mw-page-title-main">Amazon Simple Queue Service</span> Cloud-based message queuing service

Amazon Simple Queue Service is a distributed message queuing service introduced by Amazon.com as a beta in late 2004, and generally available in mid 2006. It supports programmatic sending of messages via web service applications as a way to communicate over the Internet. SQS is intended to provide a highly scalable hosted message queue that resolves issues arising from the common producer–consumer problem or connectivity between producer and consumer.

<span class="mw-page-title-main">Microsoft Azure</span> Cloud computing platform by Microsoft

Microsoft Azure, or just Azure, is the cloud computing platform developed by Microsoft. It offers management, access and development of applications and services to individuals, companies, and governments through its global infrastructure. It also provides a range of capabilities, including software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). Microsoft Azure supports many programming languages, tools, and frameworks, including Microsoft-specific and third-party software and systems.

A message queueing service is a message-oriented middleware or MOM deployed in a compute cloud using software as a service model. Service subscribers access queues and or topics to exchange data using point-to-point or publish and subscribe patterns.

<span class="mw-page-title-main">AppScale</span> American cloud infrastructure software company

AppScale is a software company that offers cloud infrastructure software and services to enterprises, government agencies, contractors, and third-party service providers. The company commercially supports one software product, AppScale ATS, a managed hybrid cloud infrastructure software platform that emulates the core AWS APIs. In 2019, the company ended commercial support for its open-source serverless computing platform AppScale GTS, but AppScale GTS source code remains freely available to the open-source community.

RabbitMQ is an open-source message-broker software that originally implemented the Advanced Message Queuing Protocol (AMQP) and has since been extended with a plug-in architecture to support Streaming Text Oriented Messaging Protocol (STOMP), MQ Telemetry Transport (MQTT), and other protocols.

MQTT is a lightweight, publish-subscribe, machine to machine network protocol for message queue/message queuing service. It is designed for connections with remote locations that have devices with resource constraints or limited network bandwidth, such as in the Internet of Things (IoT). It must run over a transport protocol that provides ordered, lossless, bi-directional connections—typically, TCP/IP. It is an open OASIS standard and an ISO recommendation.

<span class="mw-page-title-main">Celery (software)</span> Distributed task queue

Celery is an open source asynchronous task queue or job queue which is based on distributed message passing. While it supports scheduling, its focus is on operations in real time.

DataStax, Inc. is a real-time data for AI company based in Santa Clara, California. Its product Astra DB is a cloud database-as-a-service based on Apache Cassandra. DataStax also offers DataStax Enterprise (DSE), an on-premises database built on Apache Cassandra, and Astra Streaming, a messaging and event streaming cloud service based on Apache Pulsar. As of June 2022, the company has roughly 800 customers distributed in over 50 countries.

Cloud analytics is a marketing term for businesses to carry out analysis using cloud computing. It uses a range of analytical tools and techniques to help companies extract information from massive data and present it in a way that is easily categorised and readily available via a web browser.

Apache Kafka is a distributed event store and stream-processing platform. It is an open-source system developed by the Apache Software Foundation written in Java and Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect to external systems via Kafka Connect, and provides the Kafka Streams libraries for stream processing applications. Kafka uses a binary TCP-based protocol that is optimized for efficiency and relies on a "message set" abstraction that naturally groups messages together to reduce the overhead of the network roundtrip. This "leads to larger network packets, larger sequential disk operations, contiguous memory blocks [...] which allows Kafka to turn a bursty stream of random message writes into linear writes."

MassTransit is free software/open-source .NET-based Enterprise Service Bus (ESB) software that helps .NET developers route messages over RabbitMQ, Azure Service Bus, SQS, and ActiveMQ service busses. It supports multicast, versioning, encryption, sagas, retries, transactions, distributed systems and other features. It uses a "Control Bus" design to coordinate and the Rete algorithm to route. Since it does not include "business monitoring" or a "business rules engine", MassTransit is typically considered to be in the category of "lightweight ESB" software.

<span class="mw-page-title-main">Boomi, LP</span> American software company

Boomi is a software company that specializes in integration platform as a service (iPaaS), API management, master data management and data preparation. Boomi was founded in Berwyn, Pennsylvania, and first launched its services in 2007.

MQTT is an ISO standard publish–subscribe-based messaging protocol. It works on top of the Internet protocol suite TCP/IP. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker.

<span class="mw-page-title-main">Apache RocketMQ</span> Open-source stream processing platform

RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability. It is the third generation distributed messaging middleware open sourced by Alibaba in 2012. On November 21, 2016, Alibaba donated RocketMQ to the Apache Software Foundation. Next year, on February 20, the Apache Software Foundation announced Apache RocketMQ as a Top-Level Project.

References

  1. ↑ "Dead Letter Channel". www.enterpriseintegrationpatterns.com.
  2. 1 2 Redkar, Arohi (2004). Pro MSMQ: Microsoft Message Queue Programming. Apress. p. 148. ISBN   1430207329.
  3. ↑ "Dead-letter queues". IBM. Retrieved 23 February 2014.
  4. ↑ "Dead Letter Exchanges — RabbitMQ". www.rabbitmq.com. Retrieved 2020-12-06.
  5. 1 2 RabbitMQ dead letter queue "Dead Letter Exchanges".
  6. ↑ "Amazon SQS dead-letter queues". AWS.
  7. 1 2 "Using Amazon SQS Dead Letter Queues". Amazon. Retrieved 23 February 2014.
  8. ↑ "Invalid Message Channel". www.enterpriseintegrationpatterns.com.
  9. ↑ "Amazon EventBridge announces support for Dead Letter Queues". Amazon.
  10. ↑ "Forwarding to dead-letter topics | Cloud Pub/Sub". Google Cloud. Retrieved 2020-12-26.
  11. ↑ spelluru. "Compare Azure messaging services". docs.microsoft.com. Retrieved 2020-01-17.
  12. ↑ Böhm-Mäder, Johannes (14 December 2011). WebSphere MQ Security: Tales of Scowling Wolves Among Unglamorous Sheep. BoD. p. 68. ISBN   978-3842381506.
  13. ↑ "Solace Dead Message Queues".
  14. ↑ "Confluent Cloud documentation".
  15. ↑ "Apache Pulsar documentation".
  16. ↑ "Apache Pulsar PIP-22:Dead Letter Topic". GitHub .
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.