The email delivery process

Between the time an email is sent and the time it is received at the other end, it will have gone through a number of servers and checks in order to make sure it is a valid message and is being organized into the right place. This process can be complicated, but this page details the precise process of email delivery at Fastmail, which can be useful for filtering your mail, or just understanding how email works in general.

Email delivery: a brief overview

This is a general overview on how an email is delivered on a Fastmail account with no custom Sieve scripts and standard spam protection. Some custom Sieve scripts might change the order of some of these steps.

When an email arrives at Fastmail, it comes in via SMTP. SMTP is designed to move your email across different networks, and helps transfer messages between different mail providers.

When we receive a message, we immediately check for two things: the sender's email address and the recipient's email address. If either of these is missing, we reject the email.

After we check that the addresses are included, we look further at the target address (the address the message is sent to, which might be the To:, Cc:, or Bcc: address). We look up the domain in our system, then the address at the domain, and check that these match an address at a user account (or an external address). If we can't find a matching account, the message is rejected.

We begin adding headers to messages that contain server information on the message (for a more detailed explanation on this process, see the technical information below). If the message is redirected to an external mail service through an alias, we send it through to their servers. Spam checking is run on messages and we add headers with the results. If the message is detected as spam, we then sort it into the Spam folder on the user's account.

If the email passes the spam checking process, we run any Forwarding and Organize rules that the user has saved on the account. If one of these rules match, we take the rule's action, or if no rules match, the email is then sorted into the Inbox or into a custom folder if the message was sent to a matching plus or subdomain address.

A technical deep dive

A complete technical guide to the email delivery process is as follows:

Email arrives via SMTP. The SMTP protocol requires two parameters during the transaction, a "MAIL FROM" value and a "RCPT TO" value.

The "MAIL FROM" parameter is supposed to contain the email address of the sending party. If the email is from the "postmaster" address at the remote site, an empty value of <> is used. The value used here is completely independent of what is shown in the actual email header From line.

The "RCPT TO" parameter specifies the recipients of the message. The value used here is completely independent of what is shown in the actual email header To/Cc lines. This is how emails that are BCC'ed to you, or have forged headers, can arrive at your account even though your address is not in the header.
Any aliases, domains and subdomains are matched and translated to the "Target" values. This is done repeatedly until no more translations occur. The order is as follows:

a) Any address of the form user@subdomain.srcdomain.com is translated to subdomain+user@srcdomain.com. A more detailed explanation of this can be found in our documentation of subdomain addressing.

b) For an address of the form name+plus@srcdomain.com, if an alias name@srcdomain.com exists, then the address is replaced with the "Target" of the alias.

c) If address contains a "+" part (e.g. name+srcplus@srcdomain.com) and the "Target" also contains a "+" part (e.g. target+trgplus@targetdomain.com) then the two are combined into target+trgplus.srcplus@targetdomain.com.

d) If no specific name@srcdomain.com alias exists, but a 'catch all' alias exists (e.g. *@srcdomain.com), then the target of that is used. If the "Target" contains an asterix (*) in it, between the "+" part and the "@" part, then it is replaced with the original name part of the address. For example, suppose:
- The alias *@srcdomain.com exists with a "Target" of yourname+*@targetdomain.com.
- An email sent to john@srcdomain.com will be rewritten to yourname+john@targetdomain.com.
- If you have a folder in your account called "John", then this email will automatically be filed straight into that folder. You can also deliver straight to a subfolder by using a "Target" of yourname+parentfolder.*@targetdomain.com.
e) If the "Target" contains commas, then it is separated on the commas into multiple target values. Each of the actions described above occurs on each of the target addresses.
If the resultant target (or targets) is an external address, the email is then queued for sending to the external address.
If the address is local, it is delivered to the appropriate local mailbox. During delivery, the following also occurs:
- The original SMTP "MAIL FROM" parameter is added to the email as an X-Mail-from header.
- The original SMTP "RCPT TO" parameter is added to the email as an X-Delivered-to header.
- The result delivery address based on the alias and subdomain translations above is added to the email as an X-Resolved-to header.
- If the email has any attachments, the names of the attachments are added as X-Attached headers.
- The email is scanned by our content filter.
- The matching user account's Sieve script is checked for matching Discard rules, and if any tests match, the message is discarded.
- If the user has spam checking enabled, the email is spam checked.
  - Any SpamAssassin "hits" are put in the X-Spam-hits header. See SpamAsassin's documentation for a list of the main tests performed and their scores.
  - A "spam score" is put in the X-Spam-score header. It's clamped to be always >= 0, even if the actual score was negative.
After that, the user's "Sieve scripts" for Forwarding and Organize rules are run on the email, in addition to any custom Sieve added after spam filtering. This is a script generated from the Settings → Rules screen which controls the discard, forwarding, and file-into-folder actions for the email.
If the final delivery address has a "+" in it, and no particular Sieve rule catches the email to file it into a specific folder, then the email is delivered to the folder name after the '+' part. Folder name matching is not case sensitive, and with the characters '_', '-', and the space character all being regarded as the same. For example, email to username+sent_items@domain.com will be placed into the "Sent Items" folder. If no such folder exists, then the email will be delivered to the standard Inbox. You can deliver to subfolders by using '.' to separate folder names.
The 'envelope' value seen by the Sieve script is the resultant target value, not the original 'MAIL FROM' or 'RCPT TO' parts.

What headers are added by Fastmail?

While your email is being processed, a number of headers are added:

X-Spam-score: a number greater than or equal to 0. It is the score assigned by SpamAssassin, after any adjustments have been made. It will not have more than one decimal place. This header is used by the normal spam filtering.
X-Spam-hits: a list of the SpamAssassin tests that this message triggered and the score assigned by each one. Definitions of these tests can be found in SpamAssassin's documentation.
X-Spam-source: contains some processed information about the message source, such as the source IP address, hostname and country of the message (after tracing back through any trusted hosts in Received headers), and the TLD of the From header and SMTP MAIL FROM envelope addresses.
X-Spam-charsets: contains a list of character set encodings found in the message, both in headers (e.g. Subject and From) and in MIME bodies.
X-Spam: (deprecated) can be "high" or "spam". If the SpamAssassin score is greater than the threshold you have set, this will be "spam". If the score is greater than twice that threshold X-Spam will be set to "high". This header is only present for compatibility with old Sieve scripts. It is better to use X-Spam-score instead.
X-Attached: one of these headers is added for each attachment. The value of the header is the name of the attachment.
X-Mail-from: the sender address from the original SMTP envelope.
X-Delivered-to: the recipient address from the original SMTP envelope (i.e. the email address the sender actually sent to).
X-Original-Delivered-to: the for <..> address from the oldest Received: header before the message was delivered to Fastmail's servers (i.e. the address an email was delivered to before it was forwarded to a Fastmail address or fetched by a Fastmail account). This header may not always be present as not all servers include the for <..> information and it is not included if the email was delivered to multiple addresses.
X-Resolved-to: the final recipient address, after any address rewriting, and after aliases have been resolved. It is added at the LMTP delivery stage.
X-Spam-known-sender: if this message should be considered as a known sender/whitelisted. It is added at the LMTP delivery stage. See below for more information.
X-ME-VSScore: a number, which is a score assigned by our content filter. This is used in combination with X-ME-VSCategory in message filtering.
X-ME-VSCategory: a category as determined by our content filter.
Received-SPF: the SPF evaluation of incoming email.
X-ME-VSSU: we add this header if our content filter discovered unsubscribe details in an inbound email.
X-ME-VSCause: this header contains an encoded string with details relevant to our content filter. This is used to train the filter when email is marked as spam/not spam.
Authentication-Results: this header is added to inbound email and contains details of the authentication status of an email when received.
ARC-Authentication-Results, ARC-Message-Signature, ARC-Seal: we seal messages with ARC, which allows other providers to use our authentication results when messages are forwarded.
DKIM-Signature: all outbound mail is DKIM signed with a messagingengine.com key, and additionally (if possible) with a key from the domain of the sending email address.
X-ME-Sender: this is a header which we can use to find the sending account for any email sent by a user. This is used when handling spam and email abuse. As this header is encrypted it is not possible for third parties to use this to find a sending account.
X-ME-Proxy, X-ME-Proxy-Cause: we add these headers when sending mail out to other providers. These are useful to our support team when diagnosing delivery problems.
X-ME-...: From time to time we will add additional headers while we are evaluating other solutions, e.g. X-ME-ZSResult relates to the category and reputation of the sending IP address.
BIMI-...: A set of headers added to inbound mail when a Brand Indicator (Logo) was requested by the sender using BIMI.

X-Spam-known-sender

The X-Spam-known-sender header tells whether an email should be considered as a known sender and whitelisted. Currently we check the user's contacts against the MAIL FROM address and From header address as well, with the localpart replaced by * (e.g. foo@bar.com → *@bar.com). The checks done may be expanded in the future.

Currently the following values are possible, though these may change in the future. In general you should only rely on the field starting with "yes", containing "in-addressbook" or containing one of the contact group UUIDs.

no
no ("$CheckLocation == Resent-From, likely forwarded email, ignoring"), in-addressbook
no ("Email failed DMARC policy for domain"), in-addressbook
no ("From == To and no Authentication-Results header, likely forged"), in-addressbook
no ("From == To and no DKIM or SPF for from domain, likely forged"), in-addressbook
yes ("Address $CheckEmail in $CheckLocation is in addressbook"), in-addressbook
yes ("Self sent message"); in-addressbook, self-send

Where:

$CheckLocation => "SMTP MAIL FROM" or "From header"
$CheckEmail => "foo@bar.com" or "*@bar.com"

In addition, on the end of that list (comma separated), we include any contact groups the address was in as:

group-uuid ("group name")

Example headers

An example set of the above headers as they might appear on a message (although it's unlikely these would all appear on all messages):

Return-Path: <hannahm@teachdev.io>
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by sloti2d1t19 (Cyrus 3.1.6-731-g19d3b16-fmstable-20190627v1) with LMTPA;
Tue, 16 Jul 2019 00:43:28 -0400
X-Cyrus-Session-Id: sloti2d1t19-1563252208-3122307-2-17030829279071251430
X-Sieve: CMU Sieve 3.0
X-Spam-known-sender: no
X-Spam-score: 0.0
X-Spam-hits: FREEMAIL_FROM 0.001, HTML_MESSAGE 0.001, ME_ZS_CLEAN -0.001,
RCVD_IN_DNSWL_NONE -0.0001, SPF_HELO_NONE 0.001, SPF_PASS -0.001,
LANGUAGES en, BAYES_USED none, SA_VERSION 3.4.2
X-Spam-source: IP='209.85.166.47', Host='mail-io1-f47.google.com', Country='US',
FromHeader='com', MailFrom='com'
X-Spam-charsets: plain='UTF-8', html='UTF-8'
X-Resolved-to: hannah_martin@fastmail.com
X-Delivered-to: hannah_martin@fastmail.com
X-Mail-from: hannahm@teachdev.io