YMODEM

Last updated
YMODEM
Communication protocol
Purpose file transfer protocol
Developer(s) Chuck Forsberg
Introduction1985;38 years ago (1985)
Based on XMODEM
Influenced ZMODEM
Hardware modems

YMODEM is a file transfer protocol used between microcomputers connected together using modems. It was primarily used to transfer files to and from bulletin board systems. YMODEM was developed by Chuck Forsberg as an expansion of XMODEM and was first implemented in his CP/M YAM program. Initially also known as YAM, it was formally given the name "YMODEM" in 1985 by Ward Christensen, author of the original XMODEM.

Contents

YMODEM extended XMODEM in three ways, combining features found in other extended XMODEM varieties. Like XMODEM-CRC, YMODEM replaced the 8-bit checksum with a 16-bit cyclic redundancy check (CRC), but made it the default form of correction instead of optional. From TeLink it added the "block 0" header that sent the filename and size, which allowed batch transfers (multiple files in a single session) and eliminated the need to add padding at the end of the file. Finally, YMODEM allowed the block size to be increased from the original 128 bytes of data to 1024, as in XMODEM-1k, which greatly improved throughput on faster modems.

Forsberg built the standard with all of these features as runtime options, allowing a single protocol driver to fall back to XMODEM-CRC or even XMODEM when connecting to non-YAM systems. He believed that programmers would want to implement as many of these features as possible on any given platform. He was dismayed to find that the majority of implementations were actually providing nothing more than 1k block size with CRC-16, failing to implement the "block 0" while continuing to use the YMODEM name. The result was the release of many mutually incompatible YMODEM implementations, and the use of the name YMODEM Batch to clearly indicate those versions that did support the complete standard.

Features

XMODEM

The original XMODEM was a very simple protocol and that is the reason for its success; it could be implemented on practically any machine of the era, even those with very limited processors and storage. It worked by breaking up the data to be sent into 128-byte packets, adding a 3-byte header and 1-byte checksum footer, and sending the resulting 132-byte packets out in order. The receiving computer recalculated the checksum from the 128 bytes of data, and if it matched the checksum sent in the footer it sent back an ACK, and if it did not match, a NAK. When the sender received an ACK it sent the next packet, while a NAK caused it to re-send the last one.

There were a number of problems with the protocol. The use of a simple checksum meant some common errors could go unnoticed. The small packet size and requirement to wait for the ACK or NAK led to slow performance on higher speed links or ones with significant latency. Finally, as the transfer contained no details of the file, every file had to be manually started, which could be a chore when many small files were being transferred.

Solutions to these problems were developed during the early 1980s. XMODEM-CRC replaced the checksum with a 16-bit cyclic redundancy check (CRC), which was much more resistant to common errors. XMODEM-1k expanded the packet size from 128 bytes to 1024, improving performance on higher-speed connections, while others, like WXMODEM and SEAlink instead introduced sliding window systems to combat both performance and latency, at the cost of some complexity. Still others, like TeLink and MODEM7, added file information so that a single transfer could contain multiple files, allowing batches of files to be sent with a single command.

YMODEM

Chuck Forsberg, author of the CP/M "Yet Another Modem program", or YAM, decided to write a single protocol driver that supported many features compared to XMODEM and called it YMODEM. When the users started a transfer they could indicate which options they wanted on the command line, for instance, saying they wished to use CRC. The protocol was written so that it would attempt this style, but gracefully fall-back to match whatever capabilities the remote software did implement.

Abort

One problem with the original XMODEM was that there was no defined way to abort the transfer once started. The normal solution was to send NAKs to every subsequent packet if the user requested it. Since the XMODEM protocol defined a limit of ten NAKs to abort a send, and each packet might take a second to send, this meant there was a ten second delay where the sender continually sent data that was simply ignored.

Some implementations had added the ability to send a CAN instead of ACK or NAK at the end of a received packet to indicate an abort. Unfortunately, there was the possibility that a CAN could be generated by line noise and trigger an abort. YAM thus modified this slightly to require two CANs back-to-back, which would immediately perform a "graceful abort" on the sender end.

CRC

CRC support had been introduced in XMODEM-CRC. This was a very simple change to the original protocol; if requested, the receiver would attempt to trigger the transfer by sending an initial C instead of a NAK. If the remote sender supported the CRC option, it would begin sending packets as normal, but with a 16-bit CRC in the footer rather than the 1-byte checksum. YAM supported this option with no changes.

1k

1024-byte packets had been introduced in XMODEM-1k. This version did not change the trigger character from the receiver, so there was no way for the sender to know whether the receiver supported larger packets. Instead, XMODEM-1k was presented as a separate protocol on both ends of the connection. When such a connection was started, the sender could choose to send either 1024 bytes in a packet or 128, indicating the larger with an STX character in the header rather than the normal SOH. Normally only the last few packets would use the smaller packets, to avoid sending large amounts of padding. 1k also assumed CRC for all connections. YAM supported 1k with no changes.

Zero packet

In order to support automated transfers of FidoNet mail, MODEM7 introduced the ability to send the filename as plain text before sending the first block of data. This was not reliable, and TeLink improved this by placing the filename, and optionally other data like the creation date and file length, in a complete 128-byte packet. XMODEM started transfers with packet number one, so TeLink sent this packet as number zero. This "zero packet" or "block zero" became common in other FidoNet systems like SEAlink and others.

YAM supported the zero packet format, but it was ignored by many 3rd party implementations of YMODEM. When one implementation attempted to send the zero packet to a non-aware version, the receiver would naturally NAK the packet, as packet zero is illegal. The sender would then see the NAK as a transmission error and try to send the packet again, attempting this ten times before failing.

For reasons that are not entirely clear, many implementations of YMODEM did not implement this feature. Because they were unaware of it, they sent a NAK, triggering a series of resend attempts before failing out. This meant that if the user chose to use a compliant YMODEM with a non-compliant version, the transfers would fail. Nevertheless, such non-compliant versions were common.

As a result, it was common to see both YMODEM and YMODEM Batch listed as two separate protocols. Further confusion was created by the similarity between XMODEM-1k and these non-compliant YMODEMs, which were similar to the point that they were often incorrectly listed as the same.

Streaming support

YMODEM-G is a streaming variant used for error-free connections. It does not wait for an ACK to be received before sending the next packet. The protocol is faster than YMODEM because no latency is introduced between packets, but has no capability for correcting errors. It is reliant on the underlying connection being error-free, which is the case for modems supporting MNP for instance.

Normally, a YMODEM transfer would be started by the receiver sending a C to indicate it wants to use the 128-byte format with CRC, or NAK if it wishes to use the original checksum system. When g-protocol is desired, the transfer is instead triggered by sending a G. If the sender does not support the g-protocol, it considers this to be an error and ignores it, but if it does support g it begins sending packets in a continual stream. It only expects a single ACK after the final packet is received, which is indicated by the presence of an EOT character in the data. YMODEM-g assumes 1k packets are available.

However, despite this protocol potentially being faster than ZMODEM it was still rarely used. This was partially due to the lack of other functionality, but also a more serious issue. Before the emergence of the 16550 UART, there was a substantial risk of buffer overrun on the serial port. Although this would be detected by YMODEM-g it could not be corrected since no block re-transmission is possible. The receiver would have to cancel and restart the whole transmission from the beginning. ZMODEM on the other hand has transfer resume capability which made it more appealing.


Related Research Articles

The Transmission Control Protocol (TCP) is one of the main protocols of the Internet protocol suite. It originated in the initial network implementation in which it complemented the Internet Protocol (IP). Therefore, the entire suite is commonly referred to as TCP/IP. TCP provides reliable, ordered, and error-checked delivery of a stream of octets (bytes) between applications running on hosts communicating via an IP network. Major internet applications such as the World Wide Web, email, remote administration, and file transfer rely on TCP, which is part of the Transport Layer of the TCP/IP suite. SSL/TLS often runs on top of TCP.

In computer networking, the User Datagram Protocol (UDP) is one of the core communication protocols of the Internet protocol suite used to send messages to other hosts on an Internet Protocol (IP) network. Within an IP network, UDP does not require prior communication to set up communication channels or data paths.

Trivial File Transfer Protocol (TFTP) is a simple lockstep File Transfer Protocol which allows a client to get a file from or put a file onto a remote host. One of its primary uses is in the early stages of nodes booting from a local area network. TFTP has been used for this application because it is very simple to implement.

UUCP is an acronym of Unix-to-Unix Copy. The term generally refers to a suite of computer programs and protocols allowing remote execution of commands and transfer of files, email and netnews between computers.

ZMODEM is an inline file transfer protocol developed by Chuck Forsberg in 1986, in a project funded by Telenet in order to improve file transfers on their X.25 network. In addition to dramatically improved performance compared to older protocols, ZMODEM offered restartable transfers, auto-start by the sender, an expanded 32-bit CRC, and control character quoting supporting 8-bit clean transfers, allowing it to be used on networks that would not pass control characters.

XMODEM is a simple file transfer protocol developed as a quick hack by Ward Christensen for use in his 1977 MODEM.ASM terminal program. It allowed users to transmit files between their computers when both sides used MODEM. Keith Petersen made a minor update to always turn on "quiet mode", and called the result XMODEM.

<span class="mw-page-title-main">Punter (protocol)</span>

Punter is a protocol for file transfer developed in the 1980s by Steve Punter. There are various types of Punter such as PET Transfer Protocol (PTP), C1 and C2.

<span class="mw-page-title-main">Chuck Forsberg</span> American computer programmer

Charles Alton "Chuck" Forsberg developed two data transmission protocols popular in the 1990s, for uploading and downloading files from dial-up bulletin board systems. He received a Dvorak Award for Excellence in Telecommunications in 1992 for developing ZMODEM. He was also the project engineer on the Tektronix 4010-series graphics terminals.

Protocol spoofing is used in data communications to improve performance in situations where an existing protocol is inadequate, for example due to long delays or high error rates.

SEAlink is a file transfer protocol that is backward compatible with XMODEM but features a sliding window system for improved throughput. SEAlink was written in 1986 as a part of the SEAdog FidoNet mailer written by System Enhancement Associates, creators of the famous ARC program. It was licensed with a simple "give credit" requirement, but nevertheless was not very widely used except in FidoNet mailers. SEAlink, and most other XMODEM enhancements, were quickly displaced following the introduction of ZMODEM.

Lynx is a file transfer protocol for use with modems, and the name of the program that implements the protocol. Lynx is based on a sliding window protocol with two to sixteen packets per window, and 64 bytes of data per packet. It also applies run length encoding (RLE) to the data on a per-block basis to compress suitable data.

ZMax is a file transfer protocol developed in 1990-1991 by Mike Bryeans who also developed TMODEM.

The Microcom Networking Protocols, almost always shortened to MNP, is a family of error-correcting protocols commonly used on early high-speed modems. Originally developed for use on Microcom's own family of modems, the protocol was later openly licensed and used by most of the modem industry, notably the "big three", Telebit, USRobotics and Hayes. MNP was later supplanted by v.42bis, which was used almost universally starting with the first v.32bis modems in the early 1990s.

The B protocol, or CIS B, is a file transfer protocol developed for the CompuServe Information Service, and implemented in 1981. The protocol was later expanded in the QuickB version and later the enhanced B Plus version. It was a fairly advanced protocol for its era, supporting efficient transfers of files, commands and other data as well, and could be used in both directions at the same time in certain modes. These advanced features were not widely used, but could be found in a small number of client-side packages.

<span class="mw-page-title-main">SMODEM</span>

Smodem refers to a bidirectional protocol for file transfer used between modems and the DOS program in which the protocol is implemented, both of which were developed by a Finnish company named Arisoft. It was mainly used in bulletin board systems because it could transfer files in both directions at the same time and allowed users to chat with each other with AriSoft's GroupChat software. Other popular bidirectional protocols such as BiModem, HS/Link and HydraCom also offered a chat option with the operator, but not with the system's other users.

The Stream Control Transmission Protocol (SCTP) has a simpler basic packet structure than TCP. Each consists of two basic sections:

  1. The common header, which occupies the first 12 bytes. In the adjacent diagram, this header is highlighted in blue.
  2. The data chunks, which form the remaining portion of the packet. In the diagram, the first chunk is highlighted in green and the last of N chunks (Chunk N) is highlighted in red. There are several types, including payload data and different control messages.

A sliding window protocol is a feature of packet-based data transmission protocols. Sliding window protocols are used where reliable in-order delivery of packets is required, such as in the data link layer as well as in the Transmission Control Protocol (TCP). They are also used to improve efficiency when the channel may include high latency.

In data networking, telecommunications, and computer buses, an acknowledgment (ACK) is a signal that is passed between communicating processes, computers, or devices to signify acknowledgment, or receipt of message, as part of a communications protocol. The negative-acknowledgement is a signal that is sent to reject a previously received message or to indicate some kind of error. Acknowledgments and negative acknowledgments inform a sender of the receiver's state so that it can adjust its own state accordingly.

Janus is a file transfer protocol for use on bulletin board systems (BBSs). It has the relatively rare feature that it is fully bidirectional, allowing the protocol to upload and download files at the same time. It was written by Rick Huebner in 1987; Huebner had previously written a ZMODEM module for the Opus-CBBS system.

MEGAlink is a file transfer protocol for modem-equipped microcomputers written by Paul Meiners in 1987. Like many protocols of the era, MEGAlink is an expanded version of the seminal XMODEM. While it was a relatively simple and high-performance system, it remains relatively obscure because it was overshadowed by ZMODEM, which had been released a year earlier and saw rapid uptake.

References

    This page is based on this Wikipedia article
    Text is available under the CC BY-SA 4.0 license; additional terms may apply.
    Images, videos and audio are available under their respective licenses.