Sender Policy Framework (SPF) is an email authentication method which ensures the sending mail server is authorized to originate mail from the email sender's domain.[1][2] This authentication only applies to the email sender listed in the "envelope from" field during the initial SMTP connection. If the email is bounced, a message is sent to this address,[2] and for downstream transmission it typically appears in the "Return-Path" header. To authenticate the email address which is actually visible to recipients on the "From:" line, other technologies such as DMARC must be used. Forgery of this address is known as email spoofing,[3] and is often used in phishing and email spam.
The list of authorized sending hosts and IP addresses for a domain is published in the DNS records for that domain. Sender Policy Framework is defined in RFC 7208 dated April 2014 as a "proposed standard".[4]
History
The first public mention of the concept was in 2000 but went mostly unnoticed.[5] No mention was made of the concept again until a first attempt at an SPF-like specification was published in 2002 on the IETF "namedroppers" mailing list by Dana Valerie Reese,[6][3][5] who was unaware of the 2000 mention of the idea. The very next day, Paul Vixie posted his own SPF-like specification on the same list.[7][5] These posts ignited a lot of interest, led to the forming of the IETF Anti-Spam Research Group (ASRG) and their mailing list, where the SPF idea was further developed. Among the proposals submitted to the ASRG were "Reverse MX" (RMX) by Hadmut Danisch, and "Designated Mailer Protocol" (DMP) by Gordon Fecyk.[8]
In June 2003, Meng Weng Wong merged the RMX and DMP specifications[9] and solicited suggestions from others. Over the next six months, a large number of changes were made and a large community had started working on SPF.[10] Originally SPF stood for Sender Permitted From and was sometimes also called SMTP+SPF; but its name was changed to Sender Policy Framework in February 2004.
In early 2004, the IETF created the MARID working group and tried to use SPF and Microsoft's CallerID proposal as the basis for what is now known as Sender ID; but this collapsed due to technical and licensing conflicts.[11]
The SPF community returned to the original "classic" version of SPF. In July 2005, this version of the specification was approved by the IESG as an IETF experiment, inviting the community to observe SPF during the two years following publication. On April 28, 2006, the SPF RFC was published as experimental RFC 4408.
In April 2014 IETF published SPF in RFC 7208 as a "proposed standard".
The Simple Mail Transfer Protocol permits any computer to send email claiming to be from any source address. This is exploited by spammers and scammers who often use forged email addresses,[12] making it more difficult to trace a message back to its source, and easy for spammers to hide their identity in order to avoid responsibility. It is also used in phishing techniques, where users can be duped into disclosing private information in response to an email purportedly sent by an organization such as a bank.
SPF allows the owner of an Internet domain to specify which computers are authorized to send mail with envelope-from addresses in that domain, using Domain Name System (DNS) records. Receivers verifying the SPF information in TXT records may reject messages from unauthorized sources before receiving the body of the message. Thus, the principles of operation are similar to those of DNS-based blackhole lists (DNSBL), except that SPF uses the authority delegation scheme of the Domain Name System. Current practice requires the use of TXT records,[13] just as early implementations did. For a while a new record type (SPF, type 99) was registered and made available in common DNS software. Use of TXT records for SPF was intended as a transitional mechanism at the time. The experimental RFC, RFC 4408, section 3.1.1, suggested "an SPF-compliant domain name SHOULD have SPF records of both RR types".[14] The proposed standard, RFC 7208, says "use of alternative DNS RR types was supported in SPF's experimental phase but has been discontinued".[13]
The envelope-from address is transmitted at the beginning of the SMTP dialog. If the server rejects the domain, the unauthorized client should receive a rejection message, and if that client was a relaying message transfer agent (MTA), a bounce message to the original envelope-from address may be generated. If the server accepts the domain, and subsequently also accepts the recipients and the body of the message, it should insert a Return-Path field in the message header in order to save the envelope-from address. While the address in the Return-Path often matches other originator addresses in the mail header such as the header-from, this is not necessarily the case, and SPF does not prevent forgery of these other addresses such as sender header.
Spammers can send email with an SPF PASS result if they have an account in a domain with a sender policy, or abuse a compromised system in this domain. However, doing so makes the spammer easier to trace.
The main benefit of SPF is to the owners of email addresses that are forged in the Return-Path. They receive large numbers of unsolicited error messages and other auto-replies. If such receivers use SPF to specify their legitimate source IP addresses and indicate FAIL result for all other addresses, receivers checking SPF can reject forgeries, thus reducing or eliminating the amount of backscatter.
SPF has potential advantages beyond helping identify unwanted mail. In particular, if a sender provides SPF information, then receivers can use SPF PASS results in combination with an allow list to identify known reliable senders. Scenarios like compromised systems and shared sending mailers limit this use.
Reasons to implement
If a domain publishes an SPF record, spammers and phishers are less likely to forge emails pretending to be from that domain, because the forged emails are more likely to be caught in spam filters which check the SPF record. Therefore, an SPF-protected domain is less attractive to spammers and phishers. Because an SPF-protected domain is less attractive as a spoofed address, it is less likely to be denylisted by spam filters and so ultimately the legitimate email from the domain is more likely to get through.[15]
FAIL and forwarding
SPF breaks plain message forwarding. When a domain publishes an SPF FAIL policy, legitimate messages sent to receivers forwarding their mail to third parties may be rejected and/or bounced if all of the following occur:
The forwarder does not rewrite the Return-Path, unlike mailing lists.
The next hop does not allowlist the forwarder.
This hop checks SPF.
This is a necessary and obvious feature of SPF – checks behind the "border" MTA (MX) of the receiver cannot work directly.
Publishers of SPF FAIL policies must accept the risk of their legitimate emails being rejected or bounced. They should test (e.g., with a SOFTFAIL policy) until they are satisfied with the results. See below for a list of alternatives to plain message forwarding.
HELO tests
For an empty Return-Path as used in error messages and other auto-replies, an SPF check of the HELO identity is mandatory.
With a bogus HELO identity the result NONE would not help, but for valid host names SPF also protects the HELO identity. This SPF feature was always supported as an option for receivers, and later SPF drafts including the final specification recommend to check the HELO always.
This allows receivers to allowlist sending mailers based on a HELO PASS, or to reject all mails after a HELO FAIL. It can also be used in reputation systems (any allow or deny list is a simple case of a reputation system).
Implementation
Compliance with SPF consists of three loosely related tasks:
Publishing a policy: Domains and hosts identify the machines authorized to send email on their behalf. They do this by adding additional records to their existing DNS information: every domain name or host that has an A record or MX record should have an SPF record specifying the policy if it is used either in an email address or as HELO/EHLO argument. Hosts which do not send mail should have an SPF record published which indicate such ("v=spf1 -all").
Checking and using SPF information: Receivers use ordinary DNS queries, which are typically cached to enhance performance. Receivers then interpret the SPF information as specified and act upon the result.
Revising mail forwarding: Plain mail forwarding is not allowed by SPF. The alternatives are:
Remailing (i.e., replacing the original sender with one belonging to the local domain)
Refusing (e.g., answering 551 User not local; please try <user@example.com>)
Allowlisting on the target server, so that it will not refuse a forwarded message
Sender Rewriting Scheme, a more complicated mechanism that handles routing non-delivery notifications to the original sender
Thus, the key issue in SPF is the specification for the new DNS information that domains set and receivers use. The records laid out below are in typical DNS syntax, for example:
"v=spf1 ip4:192.0.2.0/24 ip4:198.51.100.123 a -all"
"v=" defines the version of SPF used. The following words provide mechanisms to use to determine if a domain is eligible to send mail. The "ip4" and "a" specify the systems permitted to send messages for the given domain. The "-all" at the end specifies that, if the previous mechanisms did not match, the message should be rejected.
Mechanisms
Eight mechanisms are defined:
ALL
Matches always; used for a default result like -all for all IPs not matched by prior mechanisms.
A
If the domain name has an address record (A or AAAA) that can be resolved to the sender's address, it will match.
IP4
If the sender is in a given IPv4 address range, match.
IP6
If the sender is in a given IPv6 address range, match.
MX
If the domain name has an MX record resolving to the sender's address, it will match (i.e. the mail comes from one of the domain's incoming mail servers).
PTR
If the domain name (PTR record) for the client's address is in the given domain and that domain name resolves to the client's address (forward-confirmed reverse DNS), match. This mechanism is discouraged and should be avoided, if possible.[13]
EXISTS
If the given domain name resolves to any address, match (no matter the address it resolves to). This is rarely used. Along with the SPF macro language it offers more complex matches like DNSBL-queries.
INCLUDE
References the policy of another domain. If that domain's policy passes, this mechanism passes. However, if the included policy fails, processing continues. To fully delegate to another domain's policy, the redirect extension must be used.
Qualifiers
Each mechanism can be combined with one of four qualifiers:
+ for a PASS result. This can be omitted; e.g., +mx is the same as mx.
? for a NEUTRAL result interpreted like NONE (no policy).
~ (tilde) for SOFTFAIL, a debugging aid between NEUTRAL and FAIL. Typically, messages that return a SOFTFAIL are accepted but tagged.
- (minus) for FAIL, the mail should be rejected (see below).
Modifiers
The modifiers allow for future extensions to the framework. To date only the two modifiers defined in the RFC 4408 have been widely deployed:
exp=some.example.com gives the name of a domain with a DNS TXT record (interpreted using SPF's macro language) to get an explanation for FAIL results—typically a URL which is added to the SMTP error code. This feature is rarely used.
redirect=some.example.com can be used instead of the ALL-mechanism to link to the policy record of another domain. This modifier is easier to understand than the somewhat similar INCLUDE-mechanism.
Error handling
As soon as SPF implementations detect syntax errors in a sender policy they must abort the evaluation with result PERMERROR. Skipping erroneous mechanisms cannot work as expected, therefore include:bad.example and redirect=bad.example also cause a PERMERROR.
Another safeguard is the maximum of ten mechanisms querying DNS, i.e. any mechanism except from IP4, IP6, and ALL. Implementations can abort the evaluation with result TEMPERROR when it takes too long or a DNS query times out or they can continue pretending that the query returned no data —which is called a "void lookup". However, they must return PERMERROR if the policy directly or indirectly needs more than ten queries for mechanisms. In addition, they should return PERMERROR as soon as more than two "void lookups" have been encountered. Any redirect= also counts towards this processing limits.[16]
A typical SPF HELO policy v=spf1 a mx ip4:192.0.2.0 -all may execute four or more DNS queries: (1) TXT record (SPF type was obsoleted by RFC 7208), (2) A or AAAA for mechanism a, (3) MX record and (4+) A or AAAA for each MX name, for mechanism mx. Except the first one, all those queries count towards the limit of 10. In addition if, for example, the sender has an IPv6 address, while its name and its two MX names have only IPv4 addresses, then the evaluation of the first two mechanisms already results in more than two void lookups and hence PERMERROR. Mechanisms ip4, ip6 and all need no DNS lookup.
Issues
DNS SPF Records
To enable rapid testing and deployment, initial versions of SPF checked for its setting in the DNS TXT record of the sending domain - even though this record was traditionally supposed to be free-form text with no semantics attached.[17] Although in July 2005, IANA assigned a specific Resource Record type 99 to SPF the uptake of was never high, and having two mechanisms was confusing for users. In 2014 the use of this record was discontinued after the SPFbis working group concluded that " ...significant migration to the SPF RR type in the foreseeable future was very unlikely and that the best solution for resolving this interoperability issue was to drop support for the SPF RR type."[13]
Header limitations
As SPF increasingly prevents spammers from spoofing the envelope-from address, many have moved to only spoof the address in the From field of the mail header, which is actually displayed to the recipient rather than only processed by the recipient's message transfer agent (MTA). SPF (or DKIM) can be used together with DMARC though, to also check the From field of the mail header. This is called 'identifier alignment'.
Custom proprietary implementations are required to protect against such display name spoofing and cannot utilize SPF.[18][19][20]
Deployment
Anti-spam software such as SpamAssassin version 3.0.0 and ASSP implement SPF. Many mail transfer agents (MTAs) support SPF directly such as Courier, CommuniGate Pro, Wildcat, MDaemon, and Microsoft Exchange, or have patches or plug-ins available that support SPF, including Postfix, Sendmail, Exim, qmail, and Qpsmtpd.[21] As of 2017, more than eight million domains publish SPF FAIL -all policies.[22] In a survey published in 2007, 5% of the .com and .net domains had some kind of SPF policy. In 2009, a continuous survey run at Nokia Research reports that 51% of the tested domains specify an SPF policy.[23] These results can include trivial policies like v=spf1 ?all.[24][25][needs update]
In April 2007, BITS, a division of the Financial Services Roundtable, published email security recommendations for its members including SPF deployment.[26] In 2008, the Messaging Anti-Abuse Working Group (MAAWG) published a paper about email authentication covering SPF, Sender ID, and DomainKeys Identified Mail (DKIM).[27] In their "Sender Best Communication Practices" the MAAWG stated: "At the very least, senders should incorporate SPF records for their mailing domains".[28] In 2015, the Messaging Anti-Abuse Working Group (MAAWG) revised a paper about email authentication covering SPF, DomainKeys Identified Mail (DKIM), and DMARC (DMARC). In their revised "Sender Best Communication Practices" the MAAWG stated: "Authentication supports transparency by further identifying the sender(s) of a message, while also contributing to the reduction or elimination of spoofed and forged addresses".[29]
The Simple Mail Transfer Protocol (SMTP) is an Internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typically use SMTP only for sending messages to a mail server for relaying, and typically submit outgoing email to the mail server on port 587 or 465 per RFC 8314. For retrieving messages, IMAP is standard, but proprietary servers also often implement proprietary protocols, e.g., Exchange ActiveSync.
A mail exchanger record specifies the mail server responsible for accepting email messages on behalf of a domain name. It is a resource record in the Domain Name System (DNS). It is possible to configure several MX records, typically pointing to an array of mail servers for load balancing and redundancy.
Various anti-spam techniques are used to prevent email spam.
Sender ID is an historic anti-spoofing proposal from the former MARID IETF working group that tried to join Sender Policy Framework (SPF) and Caller ID. Sender ID is defined primarily in Experimental RFC 4406, but there are additional parts in RFC 4405, RFC 4407 and RFC 4408.
Email authentication, or validation, is a collection of techniques aimed at providing verifiable information about the origin of email messages by validating the domain ownership of any message transfer agents (MTA) who participated in transferring and possibly modifying a message.
A message submission agent (MSA), or mail submission agent, is a computer program or software agent that receives electronic mail messages from a mail user agent (MUA) and cooperates with a mail transfer agent (MTA) for delivery of the mail. It uses ESMTP, a variant of the Simple Mail Transfer Protocol (SMTP), as specified in RFC 6409.
Forward-confirmed reverse DNS (FCrDNS), also known as full-circle reverse DNS, double-reverse DNS, or iprev, is a networking parameter configuration in which a given IP address has both forward (name-to-address) and reverse (address-to-name) Domain Name System (DNS) entries that match each other. This is the standard configuration expected by the Internet standards supporting many DNS-reliant protocols. David Barr published an opinion in RFC 1912 (Informational) recommending it as best practice for DNS administrators, but there are no formal requirements for it codified within the DNS standard itself.
MARID was an IETF working group in the applications area tasked to propose standards for email authentication in 2004. The name is an acronym of MTA Authorization Records In DNS.
The Sender Rewriting Scheme (SRS) is a scheme for bypassing the Sender Policy Framework's (SPF) methods of preventing forged sender addresses. Forging a sender address is also known as email spoofing.
Email spoofing is the creation of email messages with a forged sender address. The term applies to email purporting to be from an address which is not actually the sender's; mail sent in reply to that address may bounce or be delivered to an unrelated party whose identity has been faked. Disposable email address or "masked" email is a different topic, providing a masked email address that is not the user's normal address, which is not disclosed, but forwards mail sent to it to the user's real address.
In computing, Author Domain Signing Practices (ADSP) is an optional extension to the DKIM E-mail authentication scheme, whereby a domain can publish the signing practices it adopts when relaying mail on behalf of associated authors.
DomainKeys Identified Mail (DKIM) is an email authentication method designed to detect forged sender addresses in email, a technique often used in phishing and email spam.
Domain-based Message Authentication, Reporting and Conformance (DMARC) is an email authentication protocol. It is designed to give email domain owners the ability to protect their domain from unauthorized use, commonly known as email spoofing. The purpose and primary outcome of implementing DMARC is to protect a domain from being used in business email compromise attacks, phishing email, email scams and other cyber threat activities.
Email forwarding generically refers to the operation of re-sending a previously delivered email to an email address to one or more different email addresses.
Vouch by Reference (VBR) is a protocol used in Internet mail systems for implementing sender certification by third-party entities. Independent certification providers vouch for the reputation of senders by verifying the domain name that is associated with transmitted electronic mail. VBR information can be used by a message transfer agent, a mail delivery agent or by an email client.
SMTP Authentication, often abbreviated SMTP AUTH, is an extension of the Simple Mail Transfer Protocol (SMTP) whereby a client may log in using any authentication mechanism supported by the server. It is mainly used by submission servers, where authentication is mandatory.
Spam reporting, more properly called abuse reporting, is the action of designating electronic messages as abusive for reporting to an authority so that they can be dealt with. Reported messages can be email messages, blog comments, or any kind of spam.
A TXT record is a type of resource record in the Domain Name System (DNS) used to provide the ability to associate arbitrary text with a host or other name, such as human readable information about a server, network, data center, or other accounting information.
Authenticated Received Chain (ARC) is an email authentication system designed to allow an intermediate mail server like a mailing list or forwarding service to sign an email's original authentication results. This allows a receiving service to validate an email when the email's SPF and DKIM records are rendered invalid by an intermediate server's processing.
Murray S. Kucherawy is a computer scientist, mostly known for his work on email standardization and open source software.
1 2 Carranza, Pablo (16 July 2013). "How To use an SPF Record to Prevent Spoofing & Improve E-mail Reliability". DigitalOcean. Archived from the original on 20 April 2015. Retrieved 23 September 2019. A carefully tailored SPF record will reduce the likelihood of your domain name getting fraudulently spoofed and keep your messages from getting flagged as spam before they reach your recipients. Email spoofing is the creation of email messages with a forged sender address; something that is simple to do because many mail servers do not perform authentication. Spam and phishing emails typically use such spoofing to mislead the recipient about the origin of the message.
https://senderpolicyframework.com/ and https://www.spfwizard.net/ (tools for the generation, validation and configuration of the SPF record)
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
