History
The FIX protocol specification was originally authored in 1992 by Robert "Bob" Lamoureux and Chris Morstatt to enable electronic communication of equity trading data between Fidelity Investments and Salomon Brothers. FIX initially addressed information between broker-dealers and their institutional clients. At the time, this information was communicated verbally over the telephone. Fidelity realized that information from their broker-dealers could be routed to the wrong trader, or simply lost when the parties hung up their phones. It wanted such communications to be replaced with machine-readable data which could then be shared among traders, analyzed, acted on and stored. For example, broker-dealers call with an indication of interest (IOI) to buy or sell a block of stock. The FIX initiative created new messages such as the IOI.
According to the FIX Trading Community, FIX has become the de facto messaging standard for pre-trade and trade communication in the global equity markets, and is expanding into the post-trade space to support straight-through processing, as well as continuing to expand into foreign exchange, fixed income and derivatives markets. [1]
The FIX Trading Community is a non-profit, industry-driven standards body with a mission to address the business and regulatory issues impacting multi-asset trading across the global financial markets through the increased use of standards, including the FIX Protocol messaging language, delivering operational efficiency, increased transparency, and reduced costs and risk for all market participants. [2]
Technical specifications
Originally, the FIX standard was monolithic, including application layer semantics, message encoding, and session layer in a single technical specification. It remained monolithic through FIX version 4.2. [3] Thereafter, message encodings and session layer specifications began to be split into separate documents, and ultimately, FIX evolved into a family of related technical standards. [4]
Message encodings
Message encoding, called Presentation Layer in the Open Systems Interconnection model (OSI model), is responsible for the wire format of messages.
Tagvalue encoding (classic FIX)
The original FIX message encoding is known as tagvalue encoding. Each field consists of a unique numeric tag and a value. The tag identifies the field semantically. Therefore, messages are self-describing. Tagvalue encoding is character-based, using ASCII codes.
A message is composed of a header, a body, and a trailer. The message fields are separated by the start of heading (SOH) character (ASCII 0x01).
Up to FIX.4.4, the header contains three fields: 8 (BeginString), 9 (BodyLength), and 35 (MsgType).
From FIXT.1.1 / FIX.5.0, the header contains five or six fields: 8 (BeginString), 9 (BodyLength), 35 (MsgType), 49 (SenderCompID), 56 (TargetCompID) and the optional 1128 (ApplVerID).
The content of the message body is defined by the message type (35 MsgType) in the header.
The trailer contains the last field of the message, 10 ( Checksum ), always expressed as a three-digit number (e.g. 10=002).
Example of a FIX message, Execution Report (35=8), with the pipe character (|) representing the SOH character:
8=FIX.4.2 | 9=178 | 35=8 | 49=PHLX | 56=PERS | 52=20071123-05:30:00.000 | 11=ATOMNOCCC9990900 | 20=3 | 150=E | 39=E | 55=MSFT | 167=CS | 54=1 | 38=15 | 40=2 | 44=15 | 58=PHLX EQUITY TESTING | 59=0 | 47=C | 32=0 | 31=0 | 151=15 | 14=0 | 6=0 | 10=128 |
Body
FIX messages are formed from several fields; each field has a tag value pairing that is separated from the next field by a delimiter SOH (0x01). The tag is an integer that indicates the meaning of the field. The value is an array of bytes that hold a specific meaning for the particular tag (e.g. tag 48 is SecurityID, a string that identifies the security; tag 22 is IDSource, an integer that indicates the identifier class being used). The values may be in plain text or encoded as pure binary (in which case the value is preceded by a length field). The FIX protocol defines meanings for most tags, but leaves a range of tags reserved for private use between consenting parties.
The FIX protocol also defines sets of fields that make a particular message; within the set of fields, some will be mandatory and others optional. The ordering of fields within the message is generally unimportant, however repeating groups are preceded by a count and encrypted fields are preceded by their length. The message is broken into three distinct sections: the head, body and tail. Fields must remain within the correct section and within each section, the position may be important as fields can act as delimiters that stop one message from running into the next. The final field in any FIX message is tag 10 (checksum).
There are two main groups of messages—admin and application. The admin messages handle the basics of a FIX session. They allow for a session to be started and terminated and for recovery of missed messages. The application messages deal with the sending and receiving of trade-related information such as an order request or information on the current state and subsequent execution of that order.
Body length
The body length is the character count starting at tag 35 (included) all the way to tag 10 (excluded), including trailing SOH delimiters.
The example below (displayed with SOH delimiters as '|') has a body length of 65:
8=FIX.4.2|9=65|35=A|49=SERVER|56=CLIENT|34=177|52=20090107-18:15:16|98=0|108=30|10=062| ^ 5 + 10 + 10 + 7 + 21 + 5 + 7 ^ = 65
Checksum
The checksum of a FIX message is always the last field in the message, with tag 10 and a 3 character value. [5] It is given by summing the ASCII value of all characters in the message (except for the checksum field itself), then modulo 256. [6] For example, in the message above, the summation of all ASCII values (including the SOH characters with ASCII value 1) results in 4158. Performing the modulo operation gives the value 62. Since the checksum is composed of three characters, this results in 10=062.
FIXML
FIXML [7] is an XML schema for FIX messages. It is semantically equivalent to tagvalue encoded messages but takes advantage of XML parser technology. FIXML is commonly used for back-office and clearing applications rather than trading.
Simple Binary Encoding (SBE)
Simple Binary Encoding [8] defines a wire format using primitive data types that are native to computing systems. Message encoding and decoding is therefore much lower latency than character-based protocols since no translation is needed to put data into a format that computers can use. Aside from latency advantages, performance is more deterministic because SBE messages are constrained by templates and fixed-length data elements are preferred. Another consequence is that fields are generally at a fixed position so that message filters and routers do not need to crack an entire message to access key fields.
SBE was developed by the FIX High Performance Working Group to support high performance trading. Tagvalue encoding was deemed no longer fit-for-purpose since it is character based rather than binary and its variable-length fields and messages result in non-deterministic performance.
Unlike tagvalue and FIXML, an SBE message is not self-describing. Only data is sent on the wire with a minimal header to identify the template that controls a message. Metadata that describes a message layout is exchanged out-of-band between peers.
FIX Trading Community publishes an XML schema for SBE message schemas. A message schema may contain any number of message templates. A template describes the fields that constitute a message. Additionally, a schema provides a listing of simple and composite data types that may be reused by any number of fields.
From a practical perspective, assuming a C/C++ implementation, and adjusting for endianness: most non-composite types in the message directly map to the same type in the language. For example, 32-bit integer maps to uint32_t, fixed strings maps const char *, floating point maps to float and so on. One can generate a C/C++ struct from the schema definition. Then, given a pointer to a message buffer, accessing non-composite fields of the message amount to type-casting it to a pointer to structure and accessing structure members directly.
/* Generated struct from schema */structMessage{...uint32_tqty;...constchar*symbol;...};voidconsume_message(void*incoming_message){conststructMessage*msg=(conststructMessage*)incoming_message;printf("Traded %u of %s\n",msg->qty,msg->symbol);...}Other FIX encodings
FIX Trading Community has also developed standard mappings between FIX and other message protocols, including:
Session protocols
The session layer is responsible for message exchange including checkpoint recovery mechanisms.
FIX Transport (FIXT)
The original FIX session protocol did not have its own name since it was part of a monolithic specification covering application layer semantics and message encoding as well. However, starting with FIX version 5.0, the session layer was split off as an independent specification with the introduction of FIXT. [9] FIXT was largely the same as the original unnamed session layer in version 4.x, but it offered one significant innovation--it provided a mechanism to mix FIX application layer versions over a common session version. The current FIXT version is 1.1.
Theoretically, FIXT is transport independent. However, it is usually employed over Transmission Control Protocol (TCP).
FIXT is a point-to-point protocol. It guarantees message delivery in both directions. Messages sent in each direction carry a message sequence number in the message header. If there is a communication fault, a peer may request retransmission of missed messages. Message delivery is supported even in the event of disconnection and later reestablishment of a session.
To implement session establishment and guaranteed delivery, FIXT and classic FIX 4.x define these session message types:
- Heartbeat
- Test Request
- ResendRequest
- Reject
- SequenceReset
- Logout
- Logon
- XMLnonFIX
FIXP [10] was developed by the FIX High Performance Working Group [11] to meet the needs of high performance trading. The primary need is for low latency message encoding and decoding and control over message delivery guarantees.
To provide low latency, binary message encodings are supported both for session layer and application messages. The actual wire format is abstracted in the FIXP specification, so users may select a FIX encoding of their choice, so long as peers agree on a protocol to use. Early development has used Simple Binary Encoding.
FIXP covers both point-to-point and multicast use cases with common primitives.
When a point-to-point session is established, peers negotiate delivery guarantees from among the following choices:
- Recoverable: exactly-once message delivery. If gaps are detected, then missed messages may be recovered by retransmission.
- Idempotent: at-most-once delivery. If gaps are detected, the sender is notified, but recovery is under the control of the application, if it is done at all.
- Unsequenced: makes no delivery guarantees. This choice is appropriate if guarantees are unnecessary or if recovery is provided at the application layer or through a different communication channel.
- Note: No application messages should be sent in one direction of a session.
Delivery guarantees may be asymmetrical. For example, a trader may enter orders over an idempotent flow while executions are returned over a recoverable flow. In fast moving markets, the delay inherent in retransmission is often undesirable, resulting in missed opportunities or bad trades.
Latest developments in FIX protocol
The latest version of FIX Protocol implements "Transport Independence" by permitting multiple versions of application messages to be carried over a single version of Transport Independent FIX Session (FIXT.1.1 and higher).
Transport Independence also paves the way for transport protocols such as message queues and web services to be used instead of traditional FIX over TCP.
FIX now supports algorithmic trading by the use of FIX Algorithmic Trading Definition Language FIXatdl.
In 2005, the FIX Trading Community released FAST protocol which stands for FIX Adapted for Streaming. FAST is a binary protocol and is used mostly for sending Multicast market data via UDP connections.
Further, in 2020, the FIX Trading Community released a new FIX binary encoding, based on Simple Binary Encoding (SBE), meant to complement the existing FAST encoding. [13]
Internet Protocol version 4 (IPv4) is the fourth version of the Internet Protocol (IP). It is one of the core protocols of standards-based internetworking methods in the Internet and other packet-switched networks. IPv4 was the first version deployed for production on SATNET in 1982 and on the ARPANET in January 1983. It is still used to route most Internet traffic today, even with the ongoing deployment of Internet Protocol version 6 (IPv6), its successor.
The Internet Protocol (IP) is the network layer communications protocol in the Internet protocol suite for relaying datagrams across network boundaries. Its routing function enables internetworking, and essentially establishes the Internet.
Short Message Peer-to-Peer (SMPP) in the telecommunications industry is an open, industry standard protocol designed to provide a flexible data communication interface for the transfer of short message data between External Short Messaging Entities (ESMEs), Routing Entities (REs) and SMSC.
The Transmission Control Protocol (TCP) is one of the main protocols of the Internet protocol suite. It originated in the initial network implementation in which it complemented the Internet Protocol (IP). Therefore, the entire suite is commonly referred to as TCP/IP. TCP provides reliable, ordered, and error-checked delivery of a stream of octets (bytes) between applications running on hosts communicating via an IP network. Major internet applications such as the World Wide Web, email, remote administration, and file transfer rely on TCP, which is part of the Transport layer of the TCP/IP suite. SSL/TLS often runs on top of TCP.
In computer networking, the User Datagram Protocol (UDP) is one of the core communication protocols of the Internet protocol suite used to send messages to other hosts on an Internet Protocol (IP) network. Within an IP network, UDP does not require prior communication to set up communication channels or data paths.
Abstract Syntax Notation One (ASN.1) is a standard interface description language (IDL) for defining data structures that can be serialized and deserialized in a cross-platform way. It is broadly used in telecommunications and computer networking, and especially in cryptography.
External Machine Interface (EMI), an extension to Universal Computer Protocol (UCP), is a protocol primarily used to connect to short message service centres (SMSCs) for mobile telephones. The protocol was developed by CMG Wireless Data Solutions, now part of Mavenir.
Modbus or MODBUS is a client/server data communications protocol in the application layer. It was originally published by Modicon in 1979 for use with its programmable logic controllers (PLCs). Modbus has become a de facto standard communication protocol for communication between industrial electronic devices in a wide range of buses and network.
LIN is a network protocol used for communication between components in modern vehicles. It is a low-cost single-wire serial protocol that supports communications up to 19.2 Kbit/s with a maximum bus length of 40 metres (131.2 ft).
In computer networking, the Datagram Congestion Control Protocol (DCCP) is a message-oriented transport layer protocol. DCCP implements reliable connection setup, teardown, Explicit Congestion Notification (ECN), congestion control, and feature negotiation. The IETF published DCCP as RFC 4340, a proposed standard, in March 2006. RFC 4336 provides an introduction.
The Blocks Extensible Exchange Protocol (BEEP) is a framework for creating network application protocols. BEEP includes building blocks like framing, pipelining, multiplexing, reporting and authentication for connection and message-oriented peer-to-peer (P2P) protocols with support of asynchronous full-duplex communication.
JSON is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays. It is a commonly used data format with diverse uses in electronic data interchange, including that of web applications with servers.
Within communication protocols, TLV is an encoding scheme used for informational elements. A TLV-encoded data stream contains code related to the record type, the record value's length, and finally the value itself.
The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing, reliability and security.
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data. It is useful in developing programs that communicate with each other over a network or for storing data. The method involves an interface description language that describes the structure of some data and a program that generates source code from that description for generating or parsing a stream of bytes that represents the structured data.
This is a comparison of data serialization formats, various ways to convert complex objects to sequences of bits. It does not include markup languages used exclusively as document file formats.
FIX Algorithmic Trading Definition Language, better known as FIXatdl, is a standard for the exchange of meta-information required to enable algorithmic trading activity within the financial markets. It works in tandem with the Financial Information eXchange (FIX) protocol which is the lingua franca of electronic trading in the securities market.
Avro is a row-oriented remote procedure call and data serialization framework developed within Apache's Hadoop project. It uses JSON for defining data types and protocols, and serializes data in a compact binary format. Its primary use is in Apache Hadoop, where it can provide both a serialization format for persistent data, and a wire format for communication between Hadoop nodes, and from client programs to the Hadoop services. Avro uses a schema to structure the data that is being encoded. It has two different types of schema languages; one for human editing and another which is more machine-readable based on JSON.
A communication protocol is a system of rules that allows two or more entities of a communications system to transmit information via any variation of a physical quantity. The protocol defines the rules, syntax, semantics, and synchronization of communication and possible error recovery methods. Protocols may be implemented by hardware, software, or a combination of both.
Cap’n Proto is a data serialization format and Remote Procedure Call (RPC) framework for exchanging data between computer programs. The high-level design focuses on speed and security, making it suitable for network as well as inter-process communication. Cap'n Proto was created by the former maintainer of Google's popular Protocol Buffers framework and was designed to avoid some of its perceived shortcomings.
