TRICKLE

Last updated

TRICKLE was a file-forwarding service on the BITNET (EARN/NetNorth/GulfNet) network.

When it was created, many sites in Europe only had access to BITNET or compatible networks, and not to the Internet. Therefore, there was a great need to access files that were available only on the Internet. The TRICKLE server allowed anyone on the BITNET-compatible networks to access hundreds of thousands of files from many popular FTP servers around the world - especially to the SIMTEL20.

The TRICKLE server was a 'concentrator' of FTP sites into central servers. Each server had its own (large) cache disk to store files that have been recently ordered by users, and was in communication with all other TRICKLE servers about the contents of its cache. When a user ordered a file that is not in local cache, TRICKLE could usually forward this command to one other TRICKLE server that was known to hold the file. In some cases, "broadcasts" to all other TRICKLEs for this file is necessary, and if the file is new, or accessed very infrequently, it was requested from one of the 3 FTP slave-servers it had. These 3 were special servers who fetch the file via FTP and send it via NJE to TRICKLE.

TRICKLE servers reduced load on FTP sites substantially: Their incoming-to outgoing file rate was usually 1/5. Meaning, each cached file was usually sent out five times during its lifetime in the cache disk.

TRICKLE supported a wide range of delivery methods to the user. Besides the usual email+(UUE, XXE, BTOA, etc.), it could also FTP the files to the user, or use NJE SENDFILE where possible and desired.

It was possible to subscribe to file patterns, such as <MSDOS.VIRUS>SCAN and to have the server deliver you a new copy of the file each time it is made available. A special keyword, '*' could be used instead of the pattern to subscribe to an entire subdirectories, like: <OS2.NEW>*

In its heyday, TRICKLE processed thousands of requests daily, delivering many gigabytes of software to its users.

The server allowed users anywhere around the world to request files from an FTP server on the Internet via a gateway server which was connected to both networks. It was intended to enable the more widespread distribution of (what was then referred to as) Public Domain software from the SIMTEL20 and about a dozen other repositories (hence the command /PDGET, see below).

The software was originally written by Turgut Kalfaoglu at the Turkish national node TREARN (Ege University, Izmir) and was rapidly mirrored at many other BITNET nodes including AWIWUW11 (Austria), BANUFS11 (Belgium), DKTC11 (Denmark), DB0FUB11 or DTUZDV1 (Germany), IMIPOLI (Italy), EB0UB011 (Spain), TAUNIVM (Israel) and others.

To use the service, BITNET users would type a command to their local machine which would forward the remainder of the line as a command to the TRICKLE server. The syntax of the forwarded command was similar to those used on LISTSERV but prefixed with a slash:

TELL TRICKLE AT node /PDGET <directory>file (mode

where node was the BITNET node hosting a TRICKLE gateway; <directory>file was the directory and file required; and (mode specified the file transfer mode (e.g. UUE). On BITNET VAX systems the local syntax was SEND TRICKLE@node.

An example command would look like:

TELL TRICKLE AT TREARN /PDGET <MSDOS.GAMES>MADMAX.ARC

It also accepted commands sent via email; any number of commands could be put in the body of the message.

The target file was optionally divided into 64Kb chunks after zipping and UUencoding to alleviate congestion on the BITNET network, which used a store-and-forward transmission model where files in transit were sorted into ascending order of size at each intervening node. This enabled very small files to travel reasonably quickly, leaving larger ones to queue for later (e.g. overnight). It was the receiving user's responsibility to piece together the chunks in order for reassembly into the full UUencoded file for decoding and unzipping.

TRICKLE used a distributed cache database - that is, it only kept one copy of each requested file in Europe and each TRICKLE server was aware of each other's files. If anyone asked for any of those files, the request would be automatically forwarded to whichever TRICKLE server happened to have it. This alleviated the load on the saturated international links (esp. Europe-USA), and permitted rapid delivery of popular files. Users could also subscribe to file patterns, using a command like /SUB <MSDOS.VIRUS>SCAN* for example, and each new matching file that appeared in the repository would automatically be sent to the user.

TRICKLE also had the ability to create "newsletters" and mail them out to subscribers - it contained information about each new file that was added to the repository. Likewise, users could send the /NEW command and get a list of the newest additions to the archives.

The server also knew of which servers were closest to their users, and forced users to use their local server. However, if that server was detected to be down, then any other server accepted commands from the users of that region.

TRICKLE was written in REXX programming language (over 40k lines of code) with some code in 370 Assembler mainly written by Gustaf Neumann. It ran on IBM VM/CMS systems, For more information you may contact Turgut Kalfaoglu

For this server, Turgut Kalfaoglu was awarded European Academic and Research Network's Software of the Year award.

References: [1] [2] [3]

Related Research Articles

Freenet Peer-to-peer Internet platform for censorship-resistant communication

Freenet is a peer-to-peer platform for censorship-resistant, anonymous communication. It uses a decentralized distributed data store to keep and deliver information, and has a suite of free software for publishing and communicating on the Web without fear of censorship. Both Freenet and some of its associated tools were originally designed by Ian Clarke, who defined Freenet's goal as providing freedom of speech on the Internet with strong anonymity protection.

Gnutella is a peer-to-peer network protocol. Founded in 2000, it was the first decentralized peer-to-peer network of its kind, leading to other, later networks adopting the model.

The Secure Shell Protocol (SSH) is a cryptographic network protocol for operating network services securely over an unsecured network. Its most notable applications are remote login and command-line execution.

Web server Computer software that distributes web pages

A web server is computer software and underlying hardware that accepts requests via HTTP or its secure variant HTTPS. A user agent, commonly a web browser or web crawler, initiates communication by making a request for a web page or other resource using HTTP, and the server responds with the content of that resource or an error message. A web server can also accept and store resources sent from the user agent if configured to do so.

Uploading refers to transmitting data from one computer system to another through means of a network. Common methods of uploading include: uploading via web browsers, FTP clients], and terminals (SCP/SFTP). Uploading can be used in the context of clients that send files to a central server. While uploading can also be defined in the context of sending files between distributed clients, such as with a peer-to-peer (P2P) file-sharing protocol like BitTorrent, the term file sharing is more often used in this case. Moving files within a computer system, as opposed to over a network, is called file copying.

The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and data connections between the client and the server. FTP users may authenticate themselves with a clear-text sign-in protocol, normally in the form of a username and password, but can connect anonymously if the server is configured to allow it. For secure transmission that protects the username and password, and encrypts the content, FTP is often secured with SSL/TLS (FTPS) or replaced with SSH File Transfer Protocol (SFTP).

Proxy server Computer server that makes and receives requests on behalf of a user

In computer networking, a proxy server is a server application that acts as an intermediary between a client requesting a resource and the server providing that resource.

BITNET Relay, also known as the Inter Chat Relay Network, was a chat network setup over BITNET nodes. It predated Internet Relay Chat and other online chat systems. The program that made the network possible was called "Relay" and was developed by Jeff Kell of the University of Tennessee at Chattanooga in 1985 using the REXX programming language.

Squid (software) Caching and forwarding HTTP web proxy

Squid is a caching and forwarding HTTP web proxy. It has a wide variety of uses, including speeding up a web server by caching repeated requests, caching web, DNS and other computer network lookups for a group of people sharing network resources, and aiding security by filtering traffic. Although primarily used for HTTP and FTP, Squid includes limited support for several other protocols including Internet Gopher, SSL, TLS and HTTPS. Squid does not support the SOCKS protocol, unlike Privoxy, with which Squid can be used in order to provide SOCKS support.

Wget

GNU Wget is a computer program that retrieves content from web servers. It is part of the GNU Project. Its name derives from "World Wide Web" and "get." It supports downloading via HTTP, HTTPS, and FTP.

In software engineering, the terms frontend and backend refer to the separation of concerns between the presentation layer (frontend), and the data access layer (backend) of a piece of software, or the physical infrastructure or hardware. In the client–server model, the client is usually considered the frontend and the server is usually considered the backend, even when some presentation work is actually done on the server itself.

Secure copy protocol (SCP) is a means of securely transferring computer files between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol. "SCP" commonly refers to both the Secure Copy Protocol and the program itself.

In software architecture, publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are.

FTPmail is the term used for the practice of using an FTPmail server to gain access to various files over the Internet. An FTPmail server is a proxy server which (asynchronously) connects to remote FTP servers in response to email requests, returning the downloaded files as an email attachment. This service might be useful to users who cannot themselves initiate an FTP session—for example, because they are constrained by restrictions on their Internet access.

A spatial data infrastructure (SDI) is a data infrastructure implementing a framework of geographic data, metadata, users and tools that are interactively connected in order to use spatial data in an efficient and flexible way. Another definition is "the technology, policies, standards, human resources, and related activities necessary to acquire, process, distribute, use, maintain, and preserve spatial data".


This is a comparison of notable free and open-source configuration management software, suitable for tasks like server configuration, orchestration and infrastructure as code typically performed by a system administrator.

Toybox is a free and open-source software implementation of over 200 Unix command line utilities such as ls, cp, and mv. The Toybox project was started in 2006, and became a 0BSD licensed BusyBox alternative. Toybox is used for most of Android's command line tools in all currently supported Android versions, and is also used to build Android on Linux and macOS. All of the tools are tested on Linux, and many of them also work on BSD and macOS.

BITNET was a co-operative U.S. university computer network founded in 1981 by Ira Fuchs at the City University of New York (CUNY) and Greydon Freeman at Yale University. The first network link was between CUNY and Yale.

Remote Spooling Communications Subsystem or RSCS is a subsystem of IBM's VM/370 operating system which accepts files transmitted to it from local or remote system and users and transmits them to destination local or remote users and systems. RSCS also transmits commands and messages among users and systems.

The European Academic and Research Network (EARN) was a computer network connecting universities and research institutions across Europe, and was connected in 1983 via transatlantic circuits and a gateway funded by IBM to BITNET, its peer in the United States.

References

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.