Why we no longer ask for SPF records
Last month we released a big update to the email authentication workflow for domains. The main goal of the update was to encourage more people to set up email authentication (like custom Return-Paths and DKIM) by making the process easier. Overall it was a huge success (more on that in another post).
Subscribe to receive future product updates.As part of this update, we made a change that, while not obvious, contradicts a pattern for most ESPs: We removed the requirement to add an SPF record for your domain.
Wait, what? Is SPF dead? It’s not. SPF is alive and well and an important part of email delivery. However, the way ESPs ask you to implement SPF is not technically correct.
To explain this, we need to look at some email headers first. When you send an email, there are two types of From addresses. Laura Atkins explains this well, but I will briefly summarize it:
- “Mail From” Address: The address used for bounces or delivery reports, displayed as Return-Path in the headers.
- “Display From” Address: The address that you usually see in the email client, and the email address of the user in most cases.
If you use a mailbox provider like Gmail, Outlook, or Fastmail for your personal email these two headers will usually look the same and any bounces will return directly to your address. However, if you use an ESP, the Mail From will be an address at the ESP so they can collect bounces. For instance, here is what the relevant headers look like for a general email account:
Return-Path: <user@domain.com>
From: User Name <user@domain.com>And here is what those same headers look like by default for emails coming from Postmark:
Return-Path: <some_bounce_address@pm.mtasv.net>
From: User Name <user@domain.com>The domain pm.mtasv.net is what we use to collect bounces. And this is where the confusion starts. Inbox providers check SPF against the Return-Path address, not the From address. Back in the day when SPF came out, Microsoft decided to come out with their own standard called SenderID. The difference was that SenderID was based on the “From” address, where SPF was based on the “Return-Path” address.
In other words, SPF looks up the DNS record (and approved IPs) at the Return-Path domain (the ESP’s domain) whereas SenderID looks up the DNS record of the From domain (customer’s domain).
So when it came to ESPs like Postmark recommending DNS entries, we naturally told everyone to add an SPF record to their own domain, and we made sure we had one on our Return-Path domain. This happily covered both cases. And we just kept doing that - for many years.
The trouble is that SenderID is a dead standard now. SPF is still alive, so the DNS record lookup only happens on the Return-Path domain. So the question is:
Why are we all asking our customers to support an obsolete standard?
When it came to revisiting our Email Authentication process, and how we could make it easier, we asked ourselves this question. The result was to get rid of the SPF DNS requirement for customer domains, making it one less DNS record you have to insert into your DNS.
Instead, we place much more emphasis on DMARC by asking customers to add a custom Return-Path using your own domain. Since the purpose of DMARC is to align DKIM and SPF to your brand’s domain, both of these help support the effort toward domain reputation. With a custom Return-Path, you can set your own domain in place of ours using a CNAME, providing full SPF support at the Return-Path domain.
Return-Path: <some_bounce_address@pm.yourdomain.com>
From: User Name <user@yourdomain.com>By using a CNAME, the domain pm.yourdomain.com will point to our pm.mtasv.net domain, inheriting both the SPF record and MX records to handle SPF and bounces at the same time. The end result is one less DNS record to insert and removing an obsolete practice.
Email is hard. As a service provider we put a ton of effort into finding a balance between removing confusion and educating our customers.
By making this small change we simplified the set up process and removed an obsolete practice. And hopefully this makes the process of authenticating your email make that much more sense.
Oh, btw, what about SPF at my own domain for G Suite, Fastmail, Outlook, etc?
I want to clarify one important aspect. For your main mailbox provider, and any provider who uses your domain for the Return-Path, you still very much need an SPF record. A good example is G Suite email. Since the Return-Path and From addresses are the same (your domain) you should still keep Google’s SPF include in there. Here is an example:
"v=spf1 a mx include:_spf.google.com ~all”