Greylisting (email)

Last updated

Greylisting is a method of defending e-mail users against spam. A mail transfer agent (MTA) using greylisting will "temporarily reject" any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again after a delay, and if sufficient time has elapsed, the email will be accepted.

Contents

Mechanism

A server employing greylisting temporarily rejects email from unknown or suspicious sources by sending 4xx reply codes ("please call back later"), as defined in the Simple Mail Transfer Protocol (SMTP). Fully capable SMTP implementations are expected to maintain queues for retrying message transmissions in such cases, [1] and so while legitimate mail may be delayed, it should still get through. [2]

Temporary rejection can be issued at different stages of the SMTP dialogue, allowing for an implementation to store more or less data about the incoming message. The trade-off is more work and bandwidth for more exact matching of retries with original messages. [3] Rejecting a message after its content has been received allows the server to store a choice of headers and/or a hash of the message body.

In addition to whitelisting good senders, a greylister can provide for exceptions. Greylisting can generally be overridden by a fully validated TLS connection with a matching certificate. Because large senders often have a pool of machines that can send (and resend) email, IP addresses that have the most-significant 24 bits (/24) the same are treated as equivalent, or in some cases SPF records are used to determine the sending pool. Similarly, some e-mail systems use unique per-message return-paths, for example variable envelope return path (VERP) for mailing lists, Sender Rewriting Scheme for forwarded e-mail, Bounce Address Tag Validation for backscatter protection, etc. If an exact match on the sender address is required, every e-mail from such systems will be delayed. Some greylisting systems try to avoid this delay by eliminating the variable parts of the VERP by using only the sender domain and the beginning of the local-part of the sender address.

Greylisting is effective against mass email tools used by spammers that do not queue and reattempt mail delivery as a regular mail transport agent normally does. Delaying delivery also gives real-time blackhole lists and similar lists the time to identify and flag the spam source. Thus, these subsequent attempts are more likely to be detected as spam by other mechanisms than they were before the greylisting delay.

Advantages

The main advantage from the user's point of view is that greylisting requires no additional user configuration. If the server utilizing greylisting is configured appropriately, the end user will only notice a delay on the first message from a given sender, so long as the sending email server is identified as belonging to the same whitelisted group as earlier messages. If mail from the same sender is repeatedly greylisted it may be worth contacting the mail system administrator with detailed headers of delayed mail.

From a mail administrator's point of view the benefit is twofold. Greylisting takes minimal configuration to get up and running with occasional modifications of any local whitelists. The second benefit is that rejecting email with a temporary 451 error (actual error code is implementation dependent) is very cheap in system resources. Most spam filtering tools are very intensive users of CPU and memory. By stopping spam before it hits filtering processes, far fewer system resources are used.

Disadvantages

Delayed delivery issues

The biggest disadvantage of greylisting is that for unrecognized servers, it destroys the near-instantaneous nature of email that users expect. Mail from unrecognized servers is typically delayed by about 15 minutes, and could be delayed up to a few days for poorly configured sending systems. Explaining this to users who have become accustomed to immediate email delivery will probably not convince them that a mail server that uses greylisting is behaving correctly.

This can be a particular problem with websites that require an account to be created and the email address confirmed before they can be used – or when a user of a greylisting mailserver attempts to reset their credentials on a website that uses email confirmation of password resets. If the sending MTA of the site is poorly configured, greylisting may delay the initial email link. In extreme cases, the delivery delay imposed by the greylister can exceed the expiry time of the password reset token delivered in email. In these cases, manual intervention may be required to whitelist the website's mailserver such that the email containing the reset token can be used before it expires.

When a mail server is greylisted, the duration of time between the initial delay and the retransmission is variable; the greylisting server has no control or visibility of the delay. [4] SMTP says the retry interval should be at least 30 minutes, while the give-up time needs to be at least 4–5 days; [1] but actual values vary widely between different mail server software. [5]

Modern greylisting applications (such as Postgrey for Unix-like operating systems) automatically whitelist senders that prove themselves capable of recovering from temporary errors, [6] regardless of the reputed spamminess of the sender.

Implementation also generally include the ability to manually whitelist some mailservers.

One 2007 analysis of greylisting considers it totally undesirable due to the delay to mail, and unreliable as, if greylisting becomes widespread, junkmailers can adapt their systems to get around it. The conclusion is that the purpose of greylisting is to reduce the amount of spam that the server's spam-filtering software needs to analyze, resource-intensively, and save money on servers, not to reduce the spam reaching users. The conclusion: "[Greylisting] is very, very annoying. Much more annoying than spam." [7]

Other problems

The current SMTP specification (RFC 5321) clearly states that "the SMTP client retains responsibility for delivery of that message" (section 4.2.5) and "mail that cannot be transmitted immediately MUST be queued and periodically retried by the sender." (section 4.5.4.1). Most MTAs will therefore queue and retry messages, but a small number do not. [2] [4] [8] These are typically handled by whitelisting or exception lists.

Also, legitimate mail might not get delivered if the retry comes from a different IP address than the original attempt. When the source of an email is a server farm or goes out through some other kind of relay service, it is likely that a server other than the original one will make the next attempt. For network fault tolerance, their IPs can belong to completely unrelated address blocks, thereby defying the simple technique of identifying the most significant part of the address. Since the IP addresses will be different, the recipient's server will fail to recognize that a series of attempts are related, and refuse each of them in turn. This can continue until the message ages out of the queue if the number of servers is large enough. This problem can partially be bypassed by proactively identifying as exceptions such server farms. Likewise, exception have to be configured for multihomed hosts and hosts using DHCP. [2] In the extreme case, a sender could (legitimately) use a different IPv6 address for each outbound SMTP connection.

A sender server subjected to greylisting might also reattempt delivery to another receiving mailserver if the receiving domain has more than one MX record. This may cause problems if all such hosts do not implement the same greylisting policy and share the same database. [2]

See also

Related Research Articles

<span class="mw-page-title-main">Email</span> Mail sent using electronic means

Electronic mail is a method of transmitting and receiving messages using electronic devices. It was conceived in the late–20th century as the digital version of, or counterpart to, mail. Email is a ubiquitous and very widely used communication medium; in current use, an email address is often treated as a basic and necessary part of many processes in business, commerce, government, education, entertainment, and other spheres of daily life in most countries.

The Simple Mail Transfer Protocol (SMTP) is an Internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typically use SMTP only for sending messages to a mail server for relaying, and typically submit outgoing email to the mail server on port 587 or 465 per RFC 8314. For retrieving messages, IMAP is standard, but proprietary servers also often implement proprietary protocols, e.g., Exchange ActiveSync.

<span class="mw-page-title-main">Open mail relay</span>

An open mail relay is a Simple Mail Transfer Protocol (SMTP) server configured in such a way that it allows anyone on the Internet to send e-mail through it, not just mail destined to or originating from known users. This used to be the default configuration in many mail servers; indeed, it was the way the Internet was initially set up, but open mail relays have become unpopular because of their exploitation by spammers and worms. Many relays were closed, or were placed on blacklists by other servers.

A mail exchanger record specifies the mail server responsible for accepting email messages on behalf of a domain name. It is a resource record in the Domain Name System (DNS). It is possible to configure several MX records, typically pointing to an array of mail servers for load balancing and redundancy.

A Domain Name System blocklist, Domain Name System-based blackhole list, Domain Name System blacklist (DNSBL) or real-time blackhole list (RBL) is a service for operation of mail servers to perform a check via a Domain Name System (DNS) query whether a sending host's IP address is blacklisted for email spam. Most mail server software can be configured to check such lists, typically rejecting or flagging messages from such sites.

A tarpit is a service on a computer system that purposely delays incoming connections. The technique was developed as a defense against a computer worm, and the idea is that network abuses such as spamming or broad scanning are less effective, and therefore less attractive, if they take too long. The concept is analogous with a tar pit, in which animals can get bogged down and slowly sink under the surface, like in a swamp.

An email address identifies an email box to which messages are delivered. While early messaging systems used a variety of formats for addressing, today, email addresses follow a set of specific rules originally standardized by the Internet Engineering Task Force (IETF) in the 1980s, and updated by RFC 5322 and 6854. The term email address in this article refers to just the addr-spec in Section 3.4 of RFC 5322. The RFC defines address more broadly as either a mailbox or group. A mailbox value can be either a name-addr, which contains a display-name and addr-spec, or the more common addr-spec alone.

Various anti-spam techniques are used to prevent email spam.

A bounce message or just "bounce" is an automated message from an email system, informing the sender of a previous message that the message has not been delivered. The original message is said to have "bounced".

Email filtering is the processing of email to organize it according to specified criteria. The term can apply to the intervention of human intelligence, but most often refers to the automatic processing of messages at an SMTP server, possibly applying anti-spam techniques. Filtering can be applied to incoming emails as well as to outgoing ones.

Email authentication, or validation, is a collection of techniques aimed at providing verifiable information about the origin of email messages by validating the domain ownership of any message transfer agents (MTA) who participated in transferring and possibly modifying a message.

<span class="mw-page-title-main">Message submission agent</span>

A message submission agent (MSA), or mail submission agent, is a computer program or software agent that receives electronic mail messages from a mail user agent (MUA) and cooperates with a mail transfer agent (MTA) for delivery of the mail. It uses ESMTP, a variant of the Simple Mail Transfer Protocol (SMTP), as specified in RFC 6409.

Forward-confirmed reverse DNS (FCrDNS), also known as full-circle reverse DNS, double-reverse DNS, or iprev, is a networking parameter configuration in which a given IP address has both forward (name-to-address) and reverse (address-to-name) Domain Name System (DNS) entries that match each other. This is the standard configuration expected by the Internet standards supporting many DNS-reliant protocols. David Barr published an opinion in RFC 1912 (Informational) recommending it as best practice for DNS administrators, but there are no formal requirements for it codified within the DNS standard itself.

Variable envelope return path (VERP) is a technique used by some electronic mailing list software to enable automatic detection and removal of undeliverable e-mail addresses. It works by using a different return path for each recipient of a message.

In computing, Bounce Address Tag Validation (BATV) is a method, defined in an Internet Draft, for determining whether the bounce address specified in an E-mail message is valid. It is designed to reject backscatter, that is, bounce messages to forged return addresses.

A challenge–response system is a type of that automatically sends a reply with a challenge to the (alleged) sender of an incoming e-mail. It was originally designed in 1997 by Stan Weatherby, and was called Email Verification. In this reply, the purported sender is asked to perform some action to assure delivery of the original message, which would otherwise not be delivered. The action to perform typically takes relatively little effort to do once, but great effort to perform in large numbers. This effectively filters out spammers. Challenge–response systems only need to send challenges to unknown senders. Senders that have previously performed the challenging action, or who have previously been sent e-mail(s) to, would be automatically receive a challenge.

Nolisting is a technique to defend electronic mail domain names against e-mail spam.

<span class="mw-page-title-main">Callback verification</span> Technique used with SMTP to validate e-mail addresses

Callback verification, also known as callout verification or Sender Address Verification, is a technique used by SMTP software in order to validate e-mail addresses. The most common target of verification is the sender address from the message envelope. It is mostly used as an anti-spam measure.

Email forwarding generically refers to the operation of re-sending a previously delivered email to an email address to one or more different email addresses.

Backscatter is incorrectly automated bounce messages sent by mail servers, typically as a side effect of incoming spam.

References

  1. 1 2 John Klensin (October 2008). Simple Mail Transfer Protocol. IETF. doi: 10.17487/RFC5321 . RFC 5321 . Retrieved 1 November 2012.
  2. 1 2 3 4 Murray Kucherawy; Dave Crocker (June 2012). Email Greylisting: An Applicability Statement for SMTP. IETF. doi: 10.17487/RFC6647 . RFC 6647 . Retrieved 1 November 2012.
  3. ↑ John Levine (2005). "Experiences with Greylisting" (PDF). Second Conference on Email and Anti-Spam.
  4. 1 2 "Filtering Spam: Combined techniques give best results". Shamrock Software GmbH. December 2007. Retrieved 2008-01-09.
  5. ↑ Sendmail default is 0, 15, ..., Exim default is 0, 15, ..., Postfix default is 0, 16.6, ..., Qmail default is 0, 6:40, 26:40, ..., Courier default is 0, 5, 10, 15, 30, 35, 40, 70, 75, 80, ... Microsoft Exchange defaults to 0, 1, 2, 22, 42, 62 ..., Message Systems Momentum defaults to 0, 20, 60, 100, 180, ...
  6. ↑ David Schweikert. "Postgrey - Postfix Greylisting Policy Server" . Retrieved 1 November 2012. Clients which repeatedly show to be able to pass the greylist, are entered in a "clients whitelist", for which no greylisting is done anymore.
  7. ↑ Marco Arment. (5 April 2007). "Greylisting: The worst thing to happen to email since spam". Articles.marco.org. Retrieved 17 January 2019.
  8. ↑ Evan Harris (21 August 2003). "The Next Step in the Spam Control War: Greylisting". PureMagic Software. Retrieved 2008-01-09.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.