Amavis

Last updated
Developer(s) Patrick Ben Koetter
Initial release1997 (1997)
Stable release
2.12.0 [1]   OOjs UI icon edit-ltr-progressive.svg / 25 July 2019;4 years ago (25 July 2019)
Repository
Written in Perl
Operating system Unix-like
Type Mail filtering
License GPLv2, Simplified BSD License
Website www.amavis.org OOjs UI icon edit-ltr-progressive.svg

Amavis is an open-source content filter for electronic mail, implementing mail message transfer, decoding, some processing and checking, and interfacing with external content filters to provide protection against spam and viruses and other malware. It can be considered an interface between a mailer (MTA, Mail Transfer Agent) and one or more content filters. [2]

Contents

Amavis can be used to:

Notable features:

A common mail filtering installation with Amavis consists of a Postfix as an MTA, SpamAssassin as a spam classifier, and ClamAV as an anti-virus protection, all running under a Unix-like operating system. Many other virus scanners (about 30) and some other spam scanners (CRM114, DSPAM, Bogofilter) are supported, too, as well as some other MTAs.

Interfacing topology

Three topologies for interfacing with an MTA are supported. The amavisd process can be sandwiched between two instances of an MTA, yielding a classical after-queue [3] mail filtering setup, or amavisd can be used as an SMTP proxy filter in a before-queue [4] filtering setup, or the amavisd process can be consulted to provide mail classification but not to forward a mail message by itself, in which case the consulting client remains in charge of mail forwarding. This last approach is used in a Milter setup (with some limitations), or with a historical client program amavisd-submit.

Since version 2.7.0 a before-queue setup is preferred, as it allows for a mail message transfer to be rejected during an SMTP session [4] with a sending client. In an after-queue setup filtering takes place after a mail message has already been received and enqueued by an MTA, in which case a mail filter can no longer reject a message, but can only deliver it (possibly tagged), or discard it, or generate a non-delivery notification, which can cause unwanted backscatter in case of bouncing a message with a fake sender address.

A disadvantage of a before-queue setup [4] is that it requires resources (CPU, memory) proportional to a current (peak) mail transfer rate, unlike an after-queue setup, where some delay is acceptable and resource usage corresponds to average mail transfer rate. With introduction of an option smtpd_proxy_options=speed_adjust in Postfix 2.7.0 the resource requirements for a before-queue content filter have been much reduced. [5]

In some countries [6] the legislation does not permit mail filtering to discard a mail message once it has been accepted by an MTA, so this rules out an after-queue filtering setup with discarding or quarantining of messages, but leaves a possibility of delivering (possibly tagged) messages, or rejecting them in a before-queue setup (SMTP proxy or milter).

Interfacing protocols

Amavis can receive mail messages from an MTA over one or more sockets of protocol families PF_INET (IPv4), PF_INET6 (IPv6) or PF_LOCAL (Unix domain socket), via protocols SMTP, LMTP, or a simple private protocol AM.PDP can be used with a helper program like amavisd-milter [7] to interface with milters. On the output side protocols SMTP or LMTP can be used to pass a message to a back-end MTA instance or to an LDA, or a message can be passed to a spawned process over a Unix pipe. When SMTP or LMTP are used, a session can optionally be encrypted using a TLS STARTTLS (RFC 3207) extension to the protocol. SMTP Command Pipelining (RFC 2920) is supported in client and server code.

Interfacing with SpamAssassin

When spam scanning is enabled, a daemon process amavisd is conceptually very similar to a spamd process of a SpamAssassin project. In both cases forked child processes call SpamAssassin Perl modules directly, hence their performance is similar.

The main difference is in protocols used: Amavis typically speaks a standard SMTP protocol to an MTA, while in the spamc/spamd case an MTA typically spawns a spamc program passing a message to it over a Unix pipe, then the spamc process transfers the message to a spamd daemon using a private protocol, and spamd then calls SpamAssassin Perl modules.

Design priorities

Design priorities of the amavisd-new (from here on just called Amavis) are: reliability, security, adherence to standards, performance, and functionality.

Reliability

With the intention that no mail message could be lost due to unexpected events like I/O failures, resources depletion and unexpected program terminations, the amavisd program meticulously checks a completion status of every system call and I/O operation. Unexpected events are logged if at all possible, and handled with several layers of event handling. Amavis never takes a responsibility for a mail message delivery away from an MTA: the final success status is reported to an MTA only after the message has been passed on to the back-end MTA instance and reception was confirmed. In case of any fatal failures during processing or transferring of a message, the message being processed just stays in a queue of the front-end MTA instance, to be re-tried later. This approach also covers potential unexpected host failures, crashes of the amavisd process or one of its components.

The use of program resources like memory size, file descriptors, disk usage and creation of subprocesses is controlled. Large mail messages are not kept in memory, so the available memory size does not impose a limit on the size of mail messages that can be processed, and memory resources are not wasted unnecessarily.

Security

A great deal of attention is given to security aspects, required by handling potentially malicious, nonstandard or just garbled data in mail messages coming from untrusted sources.

The process which is handling mail messages runs with reduced privileges under a dedicated user ID. Optionally it can run chroot-ed. Risks of buffer overflows and memory allocation bugs is largely avoided by implementing all protocol handling and mail processing in Perl, which handles dynamic memory management transparently. Care is taken that content of processed messages does not inadvertently propagate to the system. Perl provides an additional security safety net with its marking of tainted data originating from the wild, and Amavis is careful to put this Perl feature to good use by avoiding automatic untainting of data (use re "taint") and only untainting it explicitly at strategic points, late in a data flow.

Amavis can use several external programs to enhance its functionality. These are de-archivers, de-compressors, virus scanners and spam scanners. As these programs are often implemented in languages like C or C++, there is a potential risk that a mail message passed to one of these programs can cause its failure or even open a security hole. The risk is limited by running these programs as an unprivileged user ID, and possibly chroot-ed. Nevertheless, external programs like unmaintained de-archivers should be avoided. The use of these external programs is configurable, and they can be disabled selectively or as a group (like all decoders or all virus scanners).

Performance

Despite being implemented in an interpreted programming language Perl, Amavis itself is not slow. The good performance of the functionality implemented by Amavis itself (not speaking of external components) is achieved by dealing with data in large chunks (e.g. not line-by-line), by avoiding unnecessary data copying, by optimizing frequently traversed code paths, by using suitable data structures and algorithms, as well as by some low-level optimizations. Bottlenecks are detected during development by profiling code and by benchmarking. Detailed timing report in the log can help recognize bottlenecks in a particular installation.

Certain external modules or programs like SpamAssassin or some command-line virus scanners can be very slow, and using these would constitute a vast majority of elapsed time and processing resources, making resources used by Amavis itself proportionally quite small.

Components like external mail decoders, virus scanners and spam scanners can each be selectively disabled if they are not needed. What remains is functionality implemented by Amavis itself, like transferring mail message from and to an MTA using an SMTP or LMTP protocol, checking mail header section validity, checking for banned mail content types, verifying and generating DKIM signatures.

As a consequence, mail processing tasks like DKIM signing and verification (with other mail checking disabled) can be exceptionally fast and can rival implementations in compiled languages. [8] [9] [10] Even full checks using a fast virus scanner but with spam scanning disabled can be surprisingly fast.

Adherence to standards

Implementation of protocols and message structures closely follows a set of applicable standards such as RFC 5322, RFC 5321, RFC 2033, RFC 3207, RFC 2045, RFC 2046, RFC 2047, RFC 3461, RFC 3462, RFC 3463, RFC 3464, RFC 4155, RFC 5965, RFC 6376, RFC 5451, RFC 6008, and RFC 4291. In several cases some functionality was re-implemented in the Amavis code even though a public (CPAN) Perl module exists, but lacks attention to detail in following a standard or lacks sufficient checking and handling of errors.

License

Amavis is licensed under a GPLv2 license. This applies to the current code, as well as to historical branches. An exception to this are some of the supporting programs (like monitoring and statistics reporting), which are covered by a New BSD License.

The project

The project started in 1997 as a Unix shell script to detect and block e-mail messages containing a virus. It was intended to block viruses at the MTA (mail transfer agent) or LDA (local delivery) stage, running on a Unix-like platform, complementing other virus protection mechanisms running on end-user personal computers.

Next the tool was re-implemented as a Perl program, which later evolved into a daemonized process. A dozen of developers took turns during the first five years of the project, developing several variants while keeping a common goal, the project name and some of the development infrastructure.

Since December 2008 (until 2018-10-09) the only active branch was officially amavisd-new, which was being developed and maintained by Mark Martinec since March 2002. This was agreed between the developers at the time in a private correspondence: Christian Bricart, Lars Hecking, Hilko Bengen, Rainer Link and Mark Martinec. The project name Amavis is largely interchangeable with the name of the amavisd-new branch.

Much functionality has been added through the years, like adding protection against spam and other unwanted content, besides the original virus protection. The focus is kept on reliability, security, adherence to standards and performance.

A domain amavis.org in use by the project was registered in 1998 by Christian Bricart, one of the early developers, who is still maintaining the domain name registration. The domain is now entirely dedicated to the only active branch. The project mailing list was moved from SourceForge to amavis.org in March 2011, and is hosted by Ralf Hildebrandt and Patrick Ben Koetter. The project web page and the main distribution site was located at the Jožef Stefan Institute, Ljubljana, Slovenia (until the handover in 2018), where most of the development was taking place between years 2002 and 2018.

Change of Project Leaders Announcement

On October 9 of 2018 Mark Martinec announced [11] at the general support and discussion mailing list his retirement from the project and also that Patrick Ben Koetter will continue as new project leader.

I know Ben personally, he is one of the two authors of The Book of Postfix, and uses Amavis in his professional life too, so I think the project will be in good hands. [11]

— Mark Martinec

After that Patrick notified [12] the migration of the source code to a public GitLab repository and his plan for the next steps regarding the project development.

Branches and the project name

Through the history of the project the name of the project or its branches varied somewhat. Initially the spelling of the project name was AMaViS (A Mail Virus Scanner), introduced by Christian Bricart. With a rewrite to Perl the name of the program was Amavis-perl. Daemonized versions were initially distributed under a name amavisd-snapshot and then as amavisd. A modular rewrite by Hilko Bengen was called Amavis-ng.

In March 2002 the amavisd-new branch was introduced by Mark Martinec, initially as a patch against amavisd-snapshot-20020300. This later evolved into a self-contained project, which is now the only surviving and actively maintained branch. Nowadays a project name is preferably spelled Amavis (while the name of the program itself is amavisd). The name Amavis is now mostly interchangeable with amavisd-new.

History of the project

shell program

Perl program

Perl daemon: amavisd

Perl, modular re-design

(Hilko Bengen)

amavisd-new

(Mark Martinec)

See also

Related Research Articles

<span class="mw-page-title-main">Email</span> Mail sent using electronic means

Electronic mail is a method of transmitting and receiving messages using electronic devices. It was conceived in the late–20th century as the digital version of, or counterpart to, mail. Email is a ubiquitous and very widely used communication medium; in current use, an email address is often treated as a basic and necessary part of many processes in business, commerce, government, education, entertainment, and other spheres of daily life in most countries.

Within the Internet email system, a message transfer agent (MTA), or mail transfer agent, or mail relay is software that transfers electronic mail messages from one computer to another using the Simple Mail Transfer Protocol. In some contexts the alternative names mail server, mail exchanger, and MX host can be used to describe an MTA.

The Simple Mail Transfer Protocol (SMTP) is an Internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typically use SMTP only for sending messages to a mail server for relaying, and typically submit outgoing email to the mail server on port 587 or 465 per RFC 8314. For retrieving messages, IMAP is standard, but proprietary servers also often implement proprietary protocols, e.g., Exchange ActiveSync.

A mail exchanger record specifies the mail server responsible for accepting email messages on behalf of a domain name. It is a resource record in the Domain Name System (DNS). It is possible to configure several MX records, typically pointing to an array of mail servers for load balancing and redundancy.

Various anti-spam techniques are used to prevent email spam.

Sender Policy Framework (SPF) is an email authentication method which ensures the sending mail server is authorized to originate mail from the email sender's domain. This authentication only applies to the email sender listed in the "envelope from" field during the initial SMTP connection. If the email is bounced, a message is sent to this address, and for downstream transmission it typically appears in the "Return-Path" header. To authenticate the email address which is actually visible to recipients on the "To:" line, other technologies such as DMARC must be used. Forgery of this address is known as email spoofing, and is often used in phishing and email spam.

Greylisting is a method of defending e-mail users against spam. A mail transfer agent (MTA) using greylisting will "temporarily reject" any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again after a delay, and if sufficient time has elapsed, the email will be accepted.

A bounce message or just "bounce" is an automated message from an email system, informing the sender of a previous message that the message has not been delivered. The original message is said to have "bounced".

Milter is an extension to the widely used open source mail transfer agents (MTA) Sendmail and Postfix. It allows administrators to add mail filters for filtering spam or viruses in the mail-processing chain. In the language of the art, "milter" refers to the protocol and API implementing the service, while "a milter" has come to refer to a filter application that uses milter to provide service.

qpsmtpd is an SMTP daemon written in Perl. It was originally designed to be a drop-in replacement for qmail-smtpd, the SMTP component of qmail, and it is now also compatible with Postfix, Exim, sendmail and virtually any software that "speaks SMTP". It has a flexible plugin system, making it easy to interoperate with other pieces in a mail system.

<span class="mw-page-title-main">Postfix (software)</span> Mail transfer agent

Postfix is a free and open-source mail transfer agent (MTA) that routes and delivers electronic mail.

Email authentication, or validation, is a collection of techniques aimed at providing verifiable information about the origin of email messages by validating the domain ownership of any message transfer agents (MTA) who participated in transferring and possibly modifying a message.

<span class="mw-page-title-main">Message submission agent</span>

A message submission agent (MSA), or mail submission agent, is a computer program or software agent that receives electronic mail messages from a mail user agent (MUA) and cooperates with a mail transfer agent (MTA) for delivery of the mail. It uses ESMTP, a variant of the Simple Mail Transfer Protocol (SMTP), as specified in RFC 6409.

The Sender Rewriting Scheme (SRS) is a scheme for bypassing the Sender Policy Framework's (SPF) methods of preventing forged sender addresses. Forging a sender address is also known as email spoofing.

policyd-weight is a mail filter for the Postfix mail transfer agent (MTA) written in Perl. It allows postfix to evaluate mail envelope information and to score mail against several DNS-based Blackhole Lists (DNSBL) before the mail is queued. The final score will determine whether a mail is rejected or accepted, in which case it is usually then subject to more resource intensive checks by a virus scanner and spam filter.

DomainKeys Identified Mail (DKIM) is an email authentication method designed to detect forged sender addresses in email, a technique often used in phishing and email spam.

SMTP proxies are specialized mail transfer agents (MTAs) that, similar to other types of proxy servers, pass SMTP sessions through to other MTAs without using the store-and-forward approach of a typical MTA. When an SMTP proxy receives a connection, it initiates another SMTP session to a destination MTA. Any errors or status information from the destination MTA will be passed back to the sending MTA through the proxy.

International email arises from the combined provision of internationalized domain names (IDN) and email address internationalization (EAI). The result is email that contains international characters, encoded as UTF-8, in the email header and in supporting mail transfer protocols. The most significant aspect of this is the allowance of email addresses in most of the world's writing systems, at both interface and transport levels.

hMailServer Open-source e-mail server

hMailServer is a free email server for Windows created by Martin Knafve. It runs as a Windows service and includes administration tools for management and backup. It has support for IMAP, POP3, and SMTP email protocols. It can use external database engines such as MySQL, MS SQL or PostgreSQL, or an internal MS SQL Compact Edition engine to store configuration and index data. The actual email messages are stored on disk in a raw MIME format. It has active support and development forums.

Backscatter is incorrectly automated bounce messages sent by mail servers, typically as a side effect of incoming spam.

References

  1. ↑ Error: Unable to display the reference properly. See the documentation for details.
  2. ↑ LeBlanc, Robert (2004-11-11). "Beating Spam and Viruses with amavisd-new and Maia Mailguard | Linux Journal". Linux Journal . Archived from the original on 2022-04-10. Retrieved 2022-04-10.
  3. ↑ Venema, Wietse (2010-01-17). "Postfix After-Queue Content Filter". Archived from the original on 2012-07-29.
  4. 1 2 3 Venema, Wietse (2009-12-11). "Postfix Before-Queue Content Filter". Archived from the original on 2012-06-16.
  5. ↑ Venema, Wietse (2010-02-14). "Postfix stable release 2.7.0". Archived from the original on 2012-07-19.
  6. ↑ "StGB (Germany), § 206 Verletzung des Post- oder Fernmeldegeheimnisses". Bundesgesetzblattes. Archived from the original on 2012-10-19.
  7. ↑ Rehor, Petr (2010-05-02). "amavisd-milter". Archived from the original on 2012-06-30.
  8. ↑ Martinec, Mark (2011-06-10). "amavisd 2.7.0 and Mail::OpenDKIM". Archived from the original on 2012-07-30.
  9. ↑ Martinec, Mark (2011-06-10). "amavisd 2.7.0 and Mail::OpenDKIM". Archived from the original on 2012-07-30.
  10. ↑ Martinec, Mark (2011-06-10). "amavisd 2.7.0 and Mail::OpenDKIM". Archived from the original on 2012-07-30.
  11. 1 2 Martinec, Mark (2018-10-09). "ANNOUNCE: Amavis project handed over to new project leaders". The amavis-users-list Archives. The amavisd-new Project.
  12. ↑ Koetter, Patrick Ben (2018-10-09). "amavis: Next Steps". The amavis-users-list Archives. The amavisd-new Project.
  13. ↑ Kjaer, Mogens. "How safe is this: Automatic scanning of mails for virus (LONG)". Newsgroup:  comp.mail.sendmail.
  14. 1 2 Bricart, Christian; Link, Rainer (2000-10-31). "AMaViS - A Mail Virus Scanner - Technical Information and Installation Instructions". Archived from the original on 2012-07-19.
  15. ↑ Quade, Jürgen (1998). "Mausefalle - EMail-Viren-Check unter Linux". IX - Magazin für professionelle Computertechnik (in German). Heise online (2/1998): 136. Archived from the original on 2012-07-27.
  16. 1 2 "AMaViS - A Mail Virus Scanner". 2004-10-07. Archived from the original on 2012-08-27.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.