Bolt (network protocol)

Last updated

Bolt
Original author(s) Neo4j
Stable release
Version 5.8
Written inVarious languages
Operating system Any
Platform Cross-platform
Type Network protocol
License Creative Commons 3.0 Attribution-ShareAlike
Website www.neo4j.com/docs/bolt/current/

The Bolt Protocol (Bolt) is a connection oriented network protocol used for client-server communication in database applications. It operates over a TCP connection or WebSocket.

Contents

Bolt is statement-oriented, allowing a client to send messages containing a statement consisting of a single string and a set of typed parameters. The server responds to each statement with a result message and an optional stream of result records.

History

The Bolt protocol was first introduced to the public in November 2015, during an interview conducted by Duncan Brown and published on DZone. [1] The first release of software implementing the protocol occurred in December 2015, as part of a milestone release of Neo4j Server. [2] In April 2016, Neo4j Server 3.0 was released and contained the first server implementation of the protocol, accompanied by a suite of Bolt client drivers. This release received attention from several mainstream media outlets. [3] [4] [5]

Versioning

The Bolt network protocol uses version negotiation to ensure compatibility between clients and servers. Clients send up to four supported protocol versions, encoded as 32-bit integers, prioritizing earlier entries. The server responds with the highest compatible version or a zero value if no match is found, closing the connection. Starting with Bolt 4.0, the protocol supports major and minor versioning. Bolt 4.3 introduced range-based minor versioning, allowing clients to specify consecutive minor versions compactly. [6]

Protocol overview

Protocol new versions are released regularly, introducing enhancements in performance, functionality, and compatibility.

Messaging

A typical interaction in the Bolt network protocol Bolt-typical-interaction.png
A typical interaction in the Bolt network protocol

Bolt clients and servers both send data over the connection as a sequence of messages. Each message has a type (denoted by a "signature" byte) and may include additional data. The client drives the interaction, and each message sent by the client will cause one or more response messages to be sent by the server.

The following information pertains to Bolt version 5.8 and may not reflect the latest developments in the protocol.

Client messages:

TypeSignature
HELLO0x01 [7]
LOGON0x6A [8]
LOGOFF0x6B [9]
TELEMETRY0x54 [10]
GOODBYE0x02 [11]
RESET0x0F [12]
RUN0x10 [13]
DISCARD0x2F [14]
PULL0x3F [15]
BEGIN0x11 [16]
COMMIT0x12 [17]
ROLLBACK0x13 [18]
ROUTE0x66 [19]

Server messages:

TypeSignature
SUCCESS0x70 [20]
FAILURE0x7F [21]
IGNORED0x7E [22]
RECORD0x71 [23]

Message transfer encoding

Each message is encoded into a sequence of bytes. These bytes are transferred using a binary chunked encoding, where each chunk is preceded by an unsigned, big-endian 16-bit integer denoting the number of bytes that immediately follow. A length of 0 is used to denote the end of the message.

Failure handling

A client may send multiple messages to a server, without first waiting for a response. [24] The server processes each message sequentially. However, as there may be logical dependencies between messages sent by the client, the server will not evaluate requests it receives after sending FAILURE in response to a preceding message. Instead, it will send an IGNORED message in reply to every client message, until the client acknowledges the failure by sending an RESET message.

This is similar to the failure handling and recovery in the PostgreSQL wire protocol.

PackStream overview - version 1

PackStream is a binary presentation format for the exchange of richly-typed data. It provides a syntax layer for the Bolt messaging protocol.

Data encoding

Bolt supports encoding for a number of different data types.

TypeDescription
Null [25] Represents the absence of a value.
Boolean [26] Boolean true or false.
Integer [27] 64-bit signed integer.
Float [28] 64-bit floating point number.
Bytes [29] Byte array.
String [30] UTF-8 encoded string.
List [31] Ordered collection of values.
Dictionary [32] Collection of key-value entries (no order guaranteed).
Structure [33] Composite value with a type signature.

Related Research Articles

<span class="mw-page-title-main">HTTP</span> Application protocol for distributed, collaborative, hypermedia information systems

HTTP is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, where hypertext documents include hyperlinks to other resources that the user can easily access, for example by a mouse click or by tapping the screen in a web browser.

Multipurpose Internet Mail Extensions (MIME) is a standard that extends the format of email messages to support text in character sets other than ASCII, as well as attachments of audio, video, images, and application programs. Message bodies may consist of multiple parts, and header information may be specified in non-ASCII character sets. Email messages with MIME formatting are typically transmitted with standard protocols, such as the Simple Mail Transfer Protocol (SMTP), the Post Office Protocol (POP), and the Internet Message Access Protocol (IMAP).

The Real-Time Streaming Protocol (RTSP) is an application-level network protocol designed for multiplexing and packetizing multimedia transport streams over a suitable transport protocol. RTSP is used in entertainment and communications systems to control streaming media servers. The protocol is used for establishing and controlling media sessions between endpoints. Clients of media servers issue commands such as play, record and pause, to facilitate real-time control of the media streaming from the server to a client or from a client to the server.

The Simple Mail Transfer Protocol (SMTP) is an Internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typically use SMTP only for sending messages to a mail server for relaying, and typically submit outgoing email to the mail server on port 587 or 465 per RFC 8314. For retrieving messages, IMAP is standard, but proprietary servers also often implement proprietary protocols, e.g., Exchange ActiveSync.

Short Message Peer-to-Peer (SMPP) in the telecommunications industry is an open, industry standard protocol designed to provide a flexible data communication interface for the transfer of short message data between External Short Messaging Entities (ESMEs), Routing Entities (REs) and SMSC.

The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and data connections between the client and the server. FTP users may authenticate themselves with a plain-text sign-in protocol, normally in the form of a username and password, but can connect anonymously if the server is configured to allow it. For secure transmission that protects the username and password, and encrypts the content, FTP is often secured with SSL/TLS (FTPS) or replaced with SSH File Transfer Protocol (SFTP).

A Berkeley (BSD) socket is an application programming interface (API) for Internet domain sockets and Unix domain sockets, used for inter-process communication (IPC). It is commonly implemented as a library of linkable modules. It originated with the 4.2BSD Unix operating system, which was released in 1983.

Direct Client-to-Client (DCC) is an IRC-related sub-protocol enabling peers to interconnect using an IRC server for handshaking in order to exchange files or perform non-relayed chats. Once established, a typical DCC session runs independently from the IRC server. Originally designed to be used with ircII it is now supported by many IRC clients. Some peer-to-peer clients on napster-protocol servers also have DCC send/get capability, including TekNap, SunshineUN and Lopster. A variation of the DCC protocol called SDCC, also known as DCC SCHAT supports encrypted connections. An RFC specification on the use of DCC does not exist.

External Data Representation (XDR) is a standard data serialization format, for uses such as computer network protocols. It allows data to be transferred between different kinds of computer systems. Converting from the local representation to XDR is called encoding. Converting from XDR to the local representation is called decoding. XDR is implemented as a software library of functions which is portable between different operating systems and is also independent of the transport layer.

<span class="mw-page-title-main">Modbus</span> Serial communications protocol

Modbus or MODBUS is a client/server data communications protocol in the application layer. It was originally designed for use with programmable logic controllers (PLCs), but has become a de facto standard communication protocol for communication between industrial electronic devices in a wide range of buses and networks.

The Financial Information eXchange (FIX) protocol is an electronic communications protocol initiated in 1992 for international real-time exchange of information related to securities transactions and markets. With trillions of dollars traded annually on the NASDAQ alone, financial service entities are employing direct market access (DMA) to increase their speed to financial markets. Managing the delivery of trading applications and keeping latency low increasingly requires an understanding of the FIX protocol.

Client-to-client protocol (CTCP) is a special type of communication between Internet Relay Chat (IRC) clients.

Real-Time Messaging Protocol (RTMP) is a communication protocol for streaming audio, video, and data over the Internet. Originally developed as a proprietary protocol by Macromedia for streaming between Flash Player and the Flash Communication Server, Adobe has released an incomplete version of the specification of the protocol for public use.

CANopen is a communication protocol stack and device profile specification for embedded systems used in automation. In terms of the OSI model, CANopen implements the layers above and including the network layer. The CANopen standard consists of an addressing scheme, several small communication protocols and an application layer defined by a device profile. The communication protocols have support for network management, device monitoring and communication between nodes, including a simple transport layer for message segmentation/desegmentation. The lower level protocol implementing the data link and physical layers is usually Controller Area Network (CAN), although devices using some other means of communication can also implement the CANopen device profile.

Within communication protocols, TLV is an encoding scheme used for informational elements. A TLV-encoded data stream contains code related to the record type, the record value's length, and finally the value itself.

Chunked transfer encoding is a streaming data transfer mechanism available in Hypertext Transfer Protocol (HTTP) version 1.1, defined in RFC 9112 §7.1. In chunked transfer encoding, the data stream is divided into a series of non-overlapping "chunks". The chunks are sent out and received independently of one another. No knowledge of the data stream outside the currently-being-processed chunk is necessary for both the sender and the receiver at any given time.

Action Message Format (AMF) is a binary format used to serialize object graphs such as ActionScript objects and XML, or send messages between an Adobe Flash client and a remote service, usually a Flash Media Server or third party alternatives. The Actionscript 3 language provides classes for encoding and decoding from the AMF format.

<span class="mw-page-title-main">Key Management Interoperability Protocol</span> Communication protocol for the manipulation of cryptographic keys

The Key Management Interoperability Protocol (KMIP) is an extensible communication protocol that defines message formats for the manipulation of cryptographic keys on a key management server. This facilitates data encryption by simplifying encryption key management. Keys may be created on a server and then retrieved, possibly wrapped by other keys. Both symmetric and asymmetric keys are supported, including the ability to sign certificates. KMIP also allows for clients to ask a server to encrypt or decrypt data, without needing direct access to the key.

MQTT is a lightweight, publish-subscribe, machine to machine network protocol for message queue/message queuing service. It is designed for connections with remote locations that have devices with resource constraints or limited network bandwidth, such as in the Internet of Things (IoT). It must run over a transport protocol that provides ordered, lossless, bi-directional connections—typically, TCP/IP. It is an open OASIS standard and an ISO recommendation.

Universal Binary JSON (UBJSON) is a computer data interchange format. It is a binary form directly imitating JSON, but requiring fewer bytes of data. It aims to achieve the generality of JSON, combined with being much easier to process than JSON.

References

  1. "Introducing Bolt, Neo4j's Upcoming Binary Protocol – Part 1 - DZone Database". dzone.com. Retrieved June 2, 2017.
  2. "Bolting Forward: The Neo4j 3.0 Milestone 1 Release Is Here - Neo4j Graph Database". neo4j.com. December 4, 2015. Retrieved June 2, 2017.
  3. Martin, Alexander J. (April 26, 2016). "Neo4j bolts on binary protocol to up its graph database game". theregister.co.uk. Retrieved June 2, 2017.
  4. "Neo4j 3.0 Released with Binary Communication Protocol and Standardised Drivers". InfoQ. Retrieved June 2, 2017.
  5. "Neo Technology Releases Neo4j 3.0". tomsitpro.com. April 26, 2016. Archived from the original on May 7, 2017. Retrieved June 2, 2017.
  6. "Versioning". neo4j.com. Retrieved January 10, 2025.
  7. "Hello message". neo4j.com. Retrieved January 9, 2025.
  8. "Logon message". neo4j.com. Retrieved January 9, 2025.
  9. "Logoff message". neo4j.com. Retrieved January 9, 2025.
  10. "Telemetry message". neo4j.com. Retrieved January 9, 2025.
  11. "Goodbye message". neo4j.com. Retrieved January 9, 2025.
  12. "Reset message". neo4j.com. Retrieved January 9, 2025.
  13. "Run message". neo4j.com. Retrieved January 9, 2025.
  14. "Discard message". neo4j.com. Retrieved January 9, 2025.
  15. "Pull message". neo4j.com. Retrieved January 9, 2025.
  16. "Begin message". neo4j.com. Retrieved January 9, 2025.
  17. "Commit message". neo4j.com. Retrieved January 9, 2025.
  18. "Rollback message". neo4j.com. Retrieved January 9, 2025.
  19. "Route message". neo4j.com. Retrieved January 9, 2025.
  20. "Success message". neo4j.com. Retrieved January 9, 2025.
  21. "Failure message". neo4j.com. Retrieved January 9, 2025.
  22. "Ignored message". neo4j.com. Retrieved January 9, 2025.
  23. "Record message". neo4j.com. Retrieved January 9, 2025.
  24. "Pipelining". neo4j.com. Retrieved January 9, 2025.
  25. "Null data type". neo4j.com. Retrieved January 10, 2025.
  26. "Boolean data type". neo4j.com. Retrieved January 10, 2025.
  27. "Integer data type". neo4j.com. Retrieved January 10, 2025.
  28. "Float data type". neo4j.com. Retrieved January 10, 2025.
  29. "Bytes data type". neo4j.com. Retrieved January 10, 2025.
  30. "String data type". neo4j.com. Retrieved January 10, 2025.
  31. "List data type". neo4j.com. Retrieved January 10, 2025.
  32. "Dictionary data type". neo4j.com. Retrieved January 10, 2025.
  33. "Structure data type". neo4j.com. Retrieved January 10, 2025.