Being able to prove that the email really was sent by the address claiming to have sent it is one mechanism to detect and prevent spammers, phishers, and fraudsters.
This proof is not easy.
SPF, DKIM, DMARC, and ARC are standards used to try to solve this problem.
- Why is sender authentication hard?
- SPF
- DKIM
- DMARC
- ARC
- SPF, DKIM, DMARC, and ARC at Fastmail: inbound mail
- SPF, DKIM, DMARC, and ARC at Fastmail: outbound mail
Why is sender authentication hard?
Consider email as if it were a traditional letter. The envelope has addressing information on it that informs the postal service where to deliver the envelope and its contents. The letter inside the envelope also tends to include a greeting to the recipient and information on who sent it.
These two things -- the delivery instructions on the envelope, and the information inside the letter -- do not have to match.
Email is the same. Within a message, there is information to tell delivery systems who sent the mail and where to deliver the mail to. This is usually not shown to end users and is hidden in email headers. There is also the payload of actual email content which is what mail clients show to their users. These two things do not have to match. This is by design, to support features like BCC (blind carbon copy).
In proving sender authenticity, we need to:
- Make sure that the address information in the email contents matches the addressing information in the delivery instructions.
- Check that the server sending the mail for that particular email address is allowed to do so.
- Make sure that nothing has changed the contents of the email along the way.
- Ask the authorized sender for a particular domain what to do with an email that fails authenticity checks.
Of course, it's still not even that simple.
SPF
What is SPF
SPF (Sender Policy Framework) lets a receiver check that the system that sent the email for a particular domain is authorized to do so. This ensures that the delivery information is correct.
What SPF is good for
SPF is helpful to detect backscatter emails. When spammers send messages pretending to be you, and those messages can't be delivered, backscatter is the bounce messages you receive.
SPF is not a complete solution
Unfortunately, SPF breaks when emails are forwarded. If someone sends from Outlook.com to Fastmail, and the Fastmail account is set to forward all messages to Gmail, the SPF checks fails the forwarded mail, because Fastmail is not allowed to deliver mail for @outlook.com email addresses.
It also doesn't help users detect scammers: the message content seen in the client can still claim to be from admin@paypal.com, even if the underlying delivery was actually sent from badscammer@outlook.com.
DKIM
What is DKIM
DKIM (DomainKeys Identified Mail) lets a sender crytographically sign the email, to prove that certain parts of the message haven't been tampered with.
What DKIM is good for
DKIM-signed mail can at least be forwarded (unlike SPF-signed mail), providing the process of forwarding doesn't alter any of the signed message headers. It does give an extra level of trust that the original mail did genuinely originate from the domain it belongs to.
DKIM is not a complete solution
DKIM-signed mail still breaks under some forwarding clients, and definitely breaks with mailing lists, which are prone to adding their own footer, as well as adding [list-id] tags to the message subject.
As with SPF, DKIM also doesn't help end users detect scammers: a message can be signed from @paypal-admin.com while being sent from @paypal.com in the From header. There's also no way for a receiver to know if all email being sent from @paypal.com should be DKIM-signed: perhaps there's a department that doesn't yet have DKIM set up. Are all unsigned emails from certain domains suspicious, or not?
DMARC
What is DMARC
DMARC (Domain-based Message Authentication, Reporting and Conformance) builds on DKIM and SPF. It lets the owner of a domain decide what should happen to an email should it fail both DKIM and SPF checks.
DMARC lets the receiver verify that the From header (the user-visible sender) aligns with the DKIM signed delivery header, or that the From header matches the SPF signed delivery domain and that it's sent from an address validated by SPF.
If either the DKIM or the SPF check fail, DMARC lets the domain owner specify if the mail should be quarantined or rejected (and sent to a reject address).
Fastmail users do not need to set a DMARC policy for their domain to work, but users who would like to set a DMARC policy on their domain are welcome to do so.
What DMARC is good for
DMARC lets the receiver verify that the delivery address (on the envelope) matches the user-visible address (the letter inside) and that the person who sent it is allowed to do so.
DMARC also provides more robustness for mail being forwarded: SPF will fail but DKIM should survive.
This is a strong anti-phishing feature.
DMARC is not a complete solution
Many email clients don't preserve message integrity when mail is forwarded, which breaks DKIM as well as SPF.
Mailing lists still alter message fields which break DKIM and they forward mail, which breaks SPF. Mailing list software is being brought up to date to deal with this, by adjusting the "From" header and re-signing the message with updated DKIM information.
Many users are still not evaluating their email with an educated, critical eye. All the automated checking in the world still can't stop a user clicking on a message sent from "PayPa1" <no_really_this_is_paypal@hotmail.com>.
ARC
What is ARC
Authenticated Received Chain is an email authentication system designed to allow an intermediate mail server like a mailing list or forwarding service to sign an email's original authentication results. This allows a receiving service to validate an email when the email's SPF and DKIM records are rendered invalid by an intermediate server's processing.
What ARC is good for
ARC allows us to check that mail sent via a trusted forwarding service was authenticated when it arrived at that service. We can then use this additional information to safely override any DMARC policies associated with that email should the DMARC checks fail.
With ARC, if a message is sent from outlook.com to Fastmail, and the Fastmail account is set to forward all mail to Gmail, Fastmail is able to sign the forwarded message to confirm that other standards such as SPF were intact when the message arrived, and the message will arrive successfully.
ARC is not a complete solution
ARC is an experimental standard, and not all mail forwarders are using ARC yet. Until the list of ARC enabled forwarders grows we maintain lists of known good email forwarders and mailing lists, which we use to whitelist known-good mail forwarders, and apply this list when evaluating DMARC policies for inbound mail.
SPF, DKIM, DMARC, and ARC at Fastmail: inbound mail
Fastmail checks SPF, DKIM, DMARC, and ARC on all inbound mail. Passing or failing these checks only alters a message's spam score; we do not outright reject mail, only mark it as more or less suspicious. We add a standard Authentication Results header to all received mail explaining the results of the authentication checks.
SPF, DKIM, DMARC, and ARC at Fastmail: outbound mail
We publish a relaxed SPF policy and DKIM-sign all outbound mail from our domains.
If you have a custom domain, you can set your SPF, DKIM and DMARC policy on your DNS screens. We have instructions if you host your DNS with us. If you use another DNS provider, you need to publish the correct records at your DNS provider.
Fastmail domains have a DMARC policy of none, which means recipient mail servers should report whether the message passes or not, but not change deliverability. This allows users to send mail using our domains from anywhere, for legacy reasons.
In the future we will publish a p=reject policy for our domains. This means to send with a From address at one of our domains, youʼll have to send through our servers.
Any mail which passes through our system on the way to another provider will be ARC sealed with the authentication results when the mail was received.
Want to know more?
There's a longer blog post about SPF, DKIM and DMARC which explains in more detail how email sender authentication came to be in this state, options for the future and why Fastmail uses the configuration it has.