Callback verification

Last updated

Callback verification, also known as callout verification or Sender Address Verification, is a technique used by SMTP software in order to validate e-mail addresses. The most common target of verification is the sender address from the message envelope (the address specified during the SMTP dialogue as "MAIL FROM"). It is mostly used as an anti-spam measure.

Contents

The three hosts involved in an SMTP callout verification. If the address is not forged, the sender and the MX server may coincide. Sav.png
The three hosts involved in an SMTP callout verification. If the address is not forged, the sender and the MX server may coincide.

Purpose

Since a large percentage of e-mail spam is generated from forged sender ("mfrom") addresses[ citation needed ], some spam can be detected by checking whether forging resulted in an invalid address, using this method.

A related technique is "call forwards", in which a secondary or firewall mail exchanger can verify recipients at the primary mail exchanger for the domain in order to decide whether the address is deliverable.

Process

The receiving mail server verifies the sender address, by verifying both parts of the sender address - the domain name (part after the @ character), and the local part (part before the @ character). The first step is to establish a successful SMTP connection to the mail exchanger for the sender address. The mail exchanger is found by looking up the MX records in the domain's DNS zone. The second step is to query the exchanger, and make sure that it accepts the address as a valid one. This is done in the same way as sending an email to the address, however the process is stopped after the mail exchanger accepts or rejects the recipient address. These are the same steps the receiving mail server would take to bounce mail back to the sender, however in this case no mail is sent. The SMTP commands sent out are:

HELO verifier host name MAIL FROM:<> RCPT TO:<the address to be tested> QUIT

Equivalently, the MAIL FROM and RCPT TO commands can be replaced by the VRFY command, however the VRFY command is not required to be supported and is usually disabled in modern MTAs.

Both of these techniques are technically compliant with the relevant SMTP RFCs (RFC 5321), however RFC 2505 (a Best Current Practice) recommends, by default, disabling the VRFY command to prevent directory harvest attacks. (One widespread interpretation implies that the MAIL FROM/RCPT TO pair of commands should also respond the same way, but this is not stated by the RFCs.)

Disadvantages

Callback verification is abusive [1] and can lead to blocklisting of the server, the domain or the IP block.

If the admin of the mail server has disabled VRFY, then using RCPT TO as a way to circumvent that is an unauthorized access and an attempt to evade access controls. As such, the operator is vulnerable to civil or criminal prosecution, depending on the jurisdiction.

Limitations

The documentation for both postfix and exim caution against the use [2] [3] of this technique and mention many limitations to SMTP callbacks. In particular, there are many situations where it is either ineffective or causes problems to the systems that receive the callbacks.

Some of these problems are caused by originating systems violating or stretching the limits of RFCs; verification problems are only reflecting these problems back to the senders, like unintentionally used invalid addresses, rejection of the null sender, or greylisting (where, for example, the delay caused by the verifying recipient is closely related to the delay caused by the originator). In many cases this in turn helps originator system to detect the problems, and fix them (like unintentionally not being able to receive valid bounces).

Several of the above problems are reduced by caching of verification results. In particular, systems that give no useful information (not rejecting at the RCPT TO time, have catch-all e-email, etc.) can be remembered and no future call backs to those systems need to be made. Also, results (positive or negative) for specific e-mail addresses can be remembered. MTAs like Exim have caching built in. [3]

Related Research Articles

The Simple Mail Transfer Protocol (SMTP) is an internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typically use SMTP only for sending messages to a mail server for relaying, and typically submit outgoing email to the mail server on port 587 or 465 per RFC 8314. For retrieving messages, IMAP is standard, but proprietary servers also often implement proprietary protocols, e.g., Exchange ActiveSync.

A mail exchanger record specifies the mail server responsible for accepting email messages on behalf of a domain name. It is a resource record in the Domain Name System (DNS). It is possible to configure several MX records, typically pointing to an array of mail servers for load balancing and redundancy.

An email address identifies an email box to which messages are delivered. While early messaging systems used a variety of formats for addressing, today, email addresses follow a set of specific rules originally standardized by the Internet Engineering Task Force (IETF) in the 1980s, and updated by RFC 5322 and 6854. The term email address in this article refers to addr-spec in RFC 5322, not to address or mailbox; i.e., a raw address without a display-name.

Various anti-spam techniques are used to prevent email spam.

Sender Policy Framework (SPF) is an email authentication method designed to detect forging sender addresses during the delivery of the email. SPF alone, though, is limited to detecting a forged sender claim in the envelope of the email, which is used when the mail gets bounced. Only in combination with DMARC can it be used to detect the forging of the visible sender in emails, a technique often used in phishing and email spam.

Greylisting is a method of defending e-mail users against spam. A mail transfer agent (MTA) using greylisting will "temporarily reject" any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again after a delay, and if sufficient time has elapsed, the email will be accepted.

A bounce message or just "bounce" is an automated message from an email system, informing the sender of a previous message that the message has not been delivered. The original message is said to have "bounced".

In computers and technology, a postmaster is the administrator of a mail server. Nearly every domain should have the e-mail address postmaster@example.com where errors in e-mail processing are directed. Error e-mails automatically generated by mail servers' MTAs usually appear to have been sent to the postmaster address.

Email authentication, or validation, is a collection of techniques aimed at providing verifiable information about the origin of email messages by validating the domain ownership of any message transfer agents (MTA) who participated in transferring and possibly modifying a message.

Forward-confirmed reverse DNS (FCrDNS), also known as full-circle reverse DNS, double-reverse DNS, or iprev, is a networking parameter configuration in which a given IP address has both forward (name-to-address) and reverse (address-to-name) Domain Name System (DNS) entries that match each other. This is the standard configuration expected by the Internet standards supporting many DNS-reliant protocols. David Barr published an opinion in RFC 1912 (Informational) recommending it as best practice for DNS administrators, but there are no formal requirements for it codified within the DNS standard itself.

For a RFC 5321 mail transfer agent (MTA), the Sender Rewriting Scheme (SRS) is a scheme for rewriting the envelope sender address of an email message, in view of remailing it. In this context, remailing is a kind of email forwarding. SRS was devised in order to forward email without breaking the Sender Policy Framework (SPF), in 2003.

Variable envelope return path (VERP) is a technique used by some electronic mailing list software to enable automatic detection and removal of undeliverable e-mail addresses. It works by using a different return path for each recipient of a message.

Email spoofing Creating email spam or phishing messages with a forged sender identity or address

Email spoofing is the creation of email messages with a forged sender address.

In computing, Bounce Address Tag Validation (BATV) is a method, defined in an Internet Draft, for determining whether the bounce address specified in an E-mail message is valid. It is designed to reject backscatter, that is, bounce messages to forged return addresses.

A challenge–response system is a type of spam filter that automatically sends a reply with a challenge to the (alleged) sender of an incoming e-mail. It was originally designed in 1997 by Stan Weatherby, and was called Email Verification. In this reply, the purported sender is asked to perform some action to assure delivery of the original message, which would otherwise not be delivered. The action to perform typically takes relatively little effort to do once, but great effort to perform in large numbers. This effectively filters out spammers. Challenge–response systems only need to send challenges to unknown senders. Senders that have previously performed the challenging action, or who have previously been sent e-mail(s) to, would be automatically whitelisted.

Backscatter is incorrectly automated bounce messages sent by mail servers, typically as a side effect of incoming spam.

A bounce address is an email address to which bounce messages are delivered. There are many variants of the name, none of them used universally, including return path, reverse path, envelope from, envelope sender, MAIL FROM, 5321-FROM, return address, From_, Errors-to, etc. It is not uncommon for a single document to use several of these names.

People tend to be much less bothered by spam slipping through filters into their mail box, than having desired e-mail ("ham") blocked. Trying to balance false negatives vs false positives is critical for a successful anti-spam system. As servers are not able to block all spam there are some tools for individual users to help control over this balance.

Amavis is an open-source content filter for electronic mail, implementing mail message transfer, decoding, some processing and checking, and interfacing with external content filters to provide protection against spam and viruses and other malware. It can be considered an interface between a mailer and one or more content filters.

References