Andy Stanford-Clark (IBM) and Arlen Nipper (then working for Eurotech, Inc.) authored the first version of the protocol in 1999.[5] It was used to monitor oil pipelines within the SCADA industrial control system.[6] The goal was to have a protocol that is bandwidth-efficient, lightweight and uses little battery power, because the devices were connected via satellite link which, at that time, was extremely expensive.[7]
Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') product line, where it stands for "Message Queue". However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name).[8] In the specification opened by IBM as version 3.1 the protocol was referred to as "MQ Telemetry Transport".[9][10] Subsequent versions released by OASIS strictly refers to the protocol as just "MQTT", although the technical committee itself is named "OASIS Message Queuing Telemetry Transport Technical Committee".[2] Since 2013, "MQTT" does not stand for anything.[11][8]
In 2013, IBM submitted MQTT v3.1 to the OASIS specification body with a charter that ensured only minor changes to the specification could be accepted.[2] After taking over maintenance of the standard from IBM, OASIS released version 3.1.1 on October 29, 2014.[12][13] A more substantial upgrade to MQTT version 5, adding several new features,[14] was released on March 7, 2019.[1]
MQTT-SN (MQTT for Sensor Networks) is a variation of the main protocol aimed at battery-powered embedded devices on non-TCP/IP networks,[15] such as Zigbee.[16]
Overview
The MQTT protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients.[17] An MQTT client is any device (from a micro controller up to a fully-fledged server) that runs an MQTT library and connects to an MQTT broker over a network.[18]
Information is organized in a hierarchy of topics. When a publisher has a new item of data to distribute, it sends a control message with the data to the connected broker. The broker then distributes the information to any clients that have subscribed to that topic. The publisher does not need to have any data on the number or locations of subscribers, and subscribers, in turn, do not have to be configured with any data about the publishers.
If a broker receives a message on a topic for which there are no current subscribers, the broker discards the message unless the publisher of the message designated the message as a retained message. A retained message is a normal MQTT message with the retained flag set to true. The broker stores the last retained message and the corresponding quality of service (QoS) for the selected topic. Each client that subscribes to a topic pattern that matches the topic of the retained message receives the retained message immediately after they subscribe. The broker stores only one retained message per topic.[19] This allows new subscribers to a topic to receive the most current value rather than waiting for the next update from a publisher.
When a publishing client first connects to the broker, it can set up a default message to be sent to subscribers if the broker detects that the publishing client has unexpectedly disconnected from the broker.
Clients only interact with a broker, but a system may contain several broker servers that exchange data based on their current subscribers' topics.
A minimal MQTT control message can be as little as two bytes of data. A control message can carry nearly 256 megabytes of data if needed. There are fourteen defined message types used to connect and disconnect a client from a broker, to publish data, to acknowledge receipt of data, and to supervise the connection between client and server.
MQTT relies on the TCP protocol for data transmission. A variant, MQTT-SN, is used over other transports such as UDP or Bluetooth.
MQTT sends connection credentials in plain text format and does not include any measures for security or authentication. This can be provided by using TLS to encrypt and protect the transferred information against interception, modification or forgery.
The default unencrypted MQTT port is 1883. The encrypted port is 8883.[20]
MQTT broker
The MQTT broker is a piece of software running on a computer (running on-premises or in the cloud), and could be self-built or hosted by a third party. It is available in both open source and proprietary implementations.
The broker acts as a post office. MQTT clients don't use a direct connection address of the intended recipient, but use the subject line called "Topic". Anyone who subscribes receives a copy of all messages for that topic. Multiple clients can subscribe to a topic from a single broker (one to many capability), and a single client can register subscriptions to topics with multiple brokers (many to one).
Each client can both produce and receive data by both publishing and subscribing, i.e. the devices can publish sensor data and still be able to receive the configuration information or control commands (MQTT is a bi-directional communication protocol). This helps in both sharing data, managing and controlling devices. A client cannot broadcast the same data to a range of topics, and must publish multiple messages to the broker, each with a single topic given.
With the MQTT broker architecture, the client devices and server application become decoupled. In this way, the clients are kept unaware of each other's information. MQTT, if configured to, can use TLS encryption with certificate, username and password protected connections. Optionally, the connection may require certification, in the form of a certificate file that a client provides and must match with the server's copy.
In case of failure, the broker software and clients can automatically hand over to a redundant/automatic backup broker. Backup brokers can also be set up to share the load of clients across multiple servers on site, in the cloud, or a combination of these.
The broker can support both standard MQTT and MQTT for compliant specifications such as Sparkplug.[21] This can be done with same server, at the same time and with the same levels of security.
The broker keeps track of all the session's information as the device goes on and off, in a function called "persistent sessions". In this state, a broker will store both connection info for each client, topics each client has subscribed to, and any messages for a topic with a QoS of 1 or 2.[22]
The main advantages of MQTT broker are:
Eliminates vulnerable and insecure client connections, if configured to
Can easily scale from a single device to thousands
Manages and tracks all client connection states, including security credentials and certificates, if configured to
Reduced network strain without compromising the security, if configured to (cellular or satellite network)
Message types
Connect
Example of an MQTT connection (QoS 0) with connect, publish/subscribe, and disconnect. The first message from client B is stored due to the retain flag.
Waits for a connection to be established with the server and creates a link between the nodes.
Disconnect
Waits for the MQTT client to finish any work it must do, and for the TCP/IP session to disconnect.
Publish
Returns immediately to the application thread after passing the request to the MQTT client.
Version 5.0
In 2019, OASIS released the official MQTT 5.0 standard.[1] Version 5.0 includes the following major new features:[23]
Reason codes: Acknowledgements now support return codes, which provide a reason for a failure.
Shared subscriptions: Allow the load to be balanced across clients and thus reduce the risk of load problems
Message expiry: Messages can include an expiry date and are deleted if they are not delivered within this time period.
Topic alias: The name of a topic can be replaced with a single number
MQTT 5.0 also supports MQTT connections over the QUIC transport protocol. MQTT over QUIC offers improved performance by reducing the number of exchanges during the connection process, reducing overall latency, and offering better handling of network congestion and switching.[24]
Quality of service
Each connection to the broker can specify a QoS measure.[25] These are classified in increasing order of overhead:
At most once – the message is sent only once and the client and broker take no additional steps to acknowledge delivery (fire and forget).
At least once – the message is re-tried by the sender multiple times until acknowledgement is received (acknowledged delivery).
Exactly once – the sender and receiver engage in a two-level handshake to ensure only one copy of the message is received (assured delivery).
This field does not affect handling of the underlying TCP data transmissions; it is only used between MQTT senders and receivers.
Security
Security of the MQTT protocol was compromised[26] in 2020 by Italian researchers, executing Slow DoS Attacks on such protocol (see CVE-2020-13849).
Clustering
MQTT clustering is a technique employed to ensure high availability, fault tolerance, and scalability in MQTT deployments.[27] As an efficient and lightweight messaging protocol, MQTT clustering allows for the creation of a resilient network of interconnected broker nodes, ensuring continuous and reliable message delivery even in the face of hardware failures or network disruptions.
↑ MQ stands for "message queues", as derived from the IBM MQ product name.
Related Research Articles
The Hypertext Transfer Protocol (HTTP) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, where hypertext documents include hyperlinks to other resources that the user can easily access, for example by a mouse click or by tapping the screen in a web browser.
The Jakarta Messaging API is a Java application programming interface (API) for message-oriented middleware. It provides generic messaging models, able to handle the producer–consumer problem, that can be used to facilitate the sending and receiving of messages between software systems. Jakarta Messaging is a part of Jakarta EE and was originally defined by a specification developed at Sun Microsystems before being guided by the Java Community Process.
In computer science, message queues and mailboxes are software-engineering components typically used for inter-process communication (IPC), or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content. Group communication systems provide similar kinds of functionality.
Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating systems and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.
IBM MQ is a family of message-oriented middleware products that IBM launched in December 1993. It was originally called MQSeries, and was renamed WebSphere MQ in 2002 to join the suite of WebSphere products. In April 2014, it was renamed IBM MQ. The products that are included in the MQ family are IBM MQ, IBM MQ Advanced, IBM MQ Appliance, IBM MQ for z/OS, and IBM MQ on IBM Cloud. IBM MQ also has containerised deployment options.
In software architecture, publish–subscribe is a messaging pattern where publishers categorize messages into classes that are received by subscribers. This is contrasted to the typical messaging pattern model where publishers send messages directly to subscribers.
Mule is a lightweight enterprise service bus (ESB) and integration framework provided by MuleSoft. It has a Java-based platform and can also act as broker for interactions between other platforms such as .NET using web services or sockets.
The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing, reliability and security.
IBM App Connect Enterprise is IBM's premier integration software offering, allowing business information to flow between disparate applications across multiple hardware and software platforms. Rules can be applied to the data flowing through user-authored integrations to route and transform the information. The product can be used as an Enterprise Service Bus supplying a communication channel between applications and services in a service-oriented architecture.
A message broker is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally-defined messages. Message brokers are a building block of message-oriented middleware (MOM) but are typically not a replacement for traditional middleware like MOM and remote procedure call (RPC).
Fault Tolerant Messaging in the context of computer systems and networks, refers to a design approach and set of techniques aimed at ensuring reliable and continuous communication between components or nodes even in the presence of errors or failures. This concept is especially critical in distributed systems, where components may be geographically dispersed and interconnected through networks, making them susceptible to various potential points of failure.
The broker pattern is an architectural pattern that can be used to structure distributed software systems with decoupled components that interact by remote procedure calls. A broker component is responsible for coordinating communication, such as forwarding requests, as well as for transmitting results and exceptions.
A message queueing service is a message-oriented middleware or MOM deployed in a compute cloud using software as a service model. Service subscribers access queues and or topics to exchange data using point-to-point or publish and subscribe patterns.
HornetQ is an open-source asynchronous messaging project from JBoss. It is an example of Message-oriented middleware. HornetQ is an open source project to build a multi-protocol, embeddable, very high performance, clustered, asynchronous messaging system. During much of its development, the HornetQ code base was developed under the name JBoss Messaging 2.0.
RabbitMQ is an open-source message-broker software that originally implemented the Advanced Message Queuing Protocol (AMQP) and has since been extended with a plug-in architecture to support Streaming Text Oriented Messaging Protocol (STOMP), MQ Telemetry Transport (MQTT), and other protocols.
Apache Kafka is a distributed event store and stream-processing platform. It is an open-source system developed by the Apache Software Foundation written in Java and Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect to external systems via Kafka Connect, and provides the Kafka Streams libraries for stream processing applications. Kafka uses a binary TCP-based protocol that is optimized for efficiency and relies on a "message set" abstraction that naturally groups messages together to reduce the overhead of the network roundtrip. This "leads to larger network packets, larger sequential disk operations, contiguous memory blocks [...] which allows Kafka to turn a bursty stream of random message writes into linear writes."
WAMP is a WebSocket subprotocol registered at IANA, specified to offer routed RPC and PubSub. Its design goal is to provide an open standard for soft, real-time message exchange between application components and ease the creation of loosely coupled architectures based on microservices. Because of this, it is a suitable enterprise service bus (ESB), fit for developing responsive web applications or coordinating multiple connected IoT devices.
OMA Lightweight M2M (LwM2M) is a protocol from the Open Mobile Alliance for machine to machine (M2M) or Internet of things (IoT) device management and service enablement. The LwM2M standard defines the application layer communication protocol between an LwM2M Server and an LwM2M Client which is located in an IoT device. It offers an approach for managing IoT devices and allows devices and systems from different vendors to co-exist in an IoT- ecosystem. LwM2M was originally built on Constrained Application Protocol (CoAP) but later LwM2M versions also support additional transfer protocols.
MQTT is an ISO standard publish–subscribe-based messaging protocol. It works on top of the Internet protocol suite TCP/IP. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker.
RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability. It is the third generation distributed messaging middleware open sourced by Alibaba in 2012. On November 21, 2016, Alibaba donated RocketMQ to the Apache Software Foundation. Next year, on February 20, the Apache Software Foundation announced Apache RocketMQ as a Top-Level Project.
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
