Considerations for evaluating transactional email service providers

All email providers are the same, right? As it turns out, that’s not quite accurate. Like any other product or service, there are huge differences, primarily in the quality of delivery, troubleshooting capabilities, API’s, and customer support. (And with infrastructure tools like transactional email providers, customer support tends to be a lot more important than other tools.)

All things being equal, it would be incredibly tempting to go with the cheapest provider, but you’ll want to make sure you consider much more than costs. We’ve put together a handful of considerations to help you more thoroughly evaluate email providers for your application.

One of the key factors influence your choice of email providers is the types of emails you plan on sending. In general, there are two groups of business emails: transactional and promotional. Some providers specialize in one or the other, and other providers support both. While there are some operational efficiencies in using one platform for both, those efficiencies can be outweighed by the reputation risks of not sufficiently separating your bulk and transactional sending. That could lead to the reputation for your promotional sending dragging down deliverability for your more critical transactional sending. For example, if you use a single provider and send your transactional and promotional emails from the same dedicated IP addresses, you’re probably not separating them enough. Using different domains and email addresses can help, but different IP addresses can further insulate your transactional email from spam complaints more common with promotional emails.

While marketing will generally manage the promotional side of email delivery, the transactional side is critical application infrastructure that you’ll likely want to be managed by your development team. Both teams have very different needs and priorities. For the purposes of this article, we’ll focus on the key factors affecting transactional email and developers.

So what factors should play into choosing your transactional email provider?

1. Delivery Rates & Speeds #

Most providers focus on inbox rates, and of course, every provider in the world claims great delivery. However, many providers don’t live up to those claims, especially with their shared IP pools. More importantly, even if an email does eventually arrive in an inbox, if it’s a minute or two late, that can be almost as bad as not arriving at all.

  1. Look for information about delivery speeds and rates. For instance, at Postmark, we constantly test and share the results for our 3rd-party-monitored inbox rates and delivery speeds for the top five inbox providers—AOL, Apple, Gmail, Hotmail, and Yahoo—right on our public status page. We also run our own tests on delivery times and include this with the delivery rates. This is the same data that we use to monitor our systems and react to any potential performance or delivery issues.
  2. Test delivery speeds for yourself. This includes testing delivery speeds to all providers, not just the service that you use for your email. Setup a test account with Gmail, Yahoo, Hotmail/Outlook, Apple/iCloud, and other popular providers, and send them test emails. We even built an open-source tool just for testing actual inbox delivery speeds.
  3. Search Twitter for mentions of real-world delivery from customers. Searching for brands plus words like “delivery,” “deliverability,” “blacklist,” and “spam,” can help reveal patterns of delivery problems with providers. No provider is perfect, but if you see recurring patterns, that should raise some flags. Almost as important, look at the responses to tweets about issues, and see how responsive or concerned providers are about delivery issues.
  4. Do they offer and encourage authentication with SPF, DKIM, and DMARC? These days, these are fairly common, but not all providers make it easy. The level of support for authentication will tell you a lot about their dedication to great delivery. At Postmark, we’ve even created a free tool to monitor and process DMARC reports for your domain. These days, manually setting up SPF has become less necessary because more and more email providers handle it in-house automatically.
  5. Do they charge extra for dedicated IP addresses? Remember that you’re buying email delivery not email sending. If a provider offers “improved deliverability” by charging extra for dedicated IP addresses, that’s kind of like paying extra for wheels for your new car. You can buy a car without wheels, but it won’t go anywhere. May ESPs offer incredibly low-cost sending through their shared IP pools, but then when you discover that the delivery is problematic, they encourage you to pay them more money for a dedicated IP address. Dedicated IP addresses can help if you’re sending incredibly high volume, but otherwise, a dedicated IP may not materially improve your delivery. At Postmark, we offer dedicated IP addresses at no cost when a sender’s volume warrants it.
  6. Who handles IP address warmup? When you get a dedicated IP address, many providers write instructional documentation for wraming up your dedicated IP address and put the warmup burden on you. At Postmark, we take full responsibility for IP address warmup so you don’t have to do anything except start sending.
  7. Check their IP addresses against known blacklists. We frequently meet with teams who are looking for new providers simply because their existing provider continually ends up on blacklists, thus, hurting their deliverability. With shared IP addresses, providers need to be very careful about policing their IPs and protecting their reputation. When providers allow marketing email to be sent from their IP addresses, they see higher spam complaint rates and thus more frequent blacklisting. At Postmark, in addition to monitoring for spam, we also don’t allow promotional email as this can also be damaging to your reputation. As a result, we very rarely encounter blacklisting of any of our IP addresses. Gmail officially recommends separating your bulk and transactional email to prevent your bulk reputation from dragging down your transactional delivery. 

2. Troubleshooting Capabilities #

No matter how great your email service provider’s reputation and delivery is, there are too many variables outside of their control. It’s inevitable that at some point, you’ll run into delivery issues. Even when it’s not your provider’s fault, you’ll need the tools and capabilities to act quickly when troubleshooting. So what do you need?

With bulk email sending, losing an email here or there isn’t a big deal, but with transactional email like password resets and receipts, even a single missing email will likely lead to a support request and increased support costs. You’ll want to be able to research and trace the life of every email individually so you can track down any delivery problems easily.

  1. Does the provider capture and store extensive message history and events? You’ll want a provider that tracks and stores events and full details like bounces, spam complaints, and delivery so you can see exactly what happened to the email and work with the recipient to get to the bottom of things. Most providers store events, but the length of time those events are available varies widely between providers.
  2. Does the provider maintain at least 30 days, but ideally more, of full content message history? When email delivery problems show up in support, they’re often a couple of weeks old by the time the recipient realizes something is wrong. In these cases, you’ll want to have a solid history of emails accessible to search and track down the missing emails to help the recipient address the problem. Postmark leads the industry by storing 45 days of full content history for every email sent.
  3. Does the provider store the full content of sent emails? Sometimes, when you encounter a delivery issue, it can be caused by incredibly obscure issues with content triggering spam filters. For instance, we’ve seen cases where a phone number in the email was the trigger. Without historical records of the full content of emails, these kinds of problems are incredibly difficult to troubleshoot. Unfortunately, most providers don’t store any historical content, and those that do only store it for a few days. Postmark stores a full 45 days of both message history and content for troubleshooting.
  4. Does the provider accept the deliverability burden? With many provides, when you encounter delivery problems, the first thing they’ll do is push the blame onto you. They’ll tell you that it’s related to your content rather than digging in to see if the problem is on their end. Then they’ll tell you that switching providers won’t matter because it’s your content that’s problematic. At Postmark, we accept full responsibility for delivery, and we’ll never tell you it’s your fault without actionable evidence that shows you exactly what’s wrong and how to fix it.

3. Developer API & Documentation #

If you’re working with an email service provider, chances are pretty good you’re going to want to use their API, and stumbling through incomprehensible documentation can destroy your productivity. So make sure to spend a little time looking over developer documentation. These might seem obvious in hindsight, but it’s easy to forget how much great documentation and libraries can improve your productivity.

  1. Is the documentation organized and easy to find? Setting up an ESP is fairly straightforward, but integrating more advanced features takes work. Make sure to spend some time glancing through provider documentation before committing.
  2. Are there extensive client libraries to minimize development effort, and are the libraries well-maintained and current? It probably goes without saying, but officially supported client libraries are usually the best bet. They’re generally quicker to receive updates when new features are released, and the provider will stand behind them and fix any bugs in the implementation. The more official client libraries, the better.
  3. Are there easily searchable help docs? With Postmark, we’ve strived to unify all of our documentation, including our guides, developer documentation, support docs, blog posts, and any information that could be useful and make it all searchable from one place. Not all searches are created equal, the better the search and the more help docs, the faster you’ll be up and running.
  4. Can you monitor delivery performance via API? You can’t spend your days manually checking Twitter or a status page for issues. With transactional email, nothing beats a publicly available status API to automate monitoring of your email service provider. Visit their status page, and see how informative and transparent it is. When you’re having email troubles, a less-than-helpful status page can complicate things quickly.
  5. Can you seamlessly integrate with your platform? (Bounce handling, inbound, open tracking, click tracking) Sending email is only part of solution. Monitoring bounces, receiving and parsing inbound email, and tracking opens and clicks are central to managing transactional email for your application. Make sure that your provider supports all of these so you have a full picture of your sending.
  6. How’s the templating? When you’re setting up your transactional emails, the design, testing, and development can be incredibly tedious. Having a provider that starts you off with pre-made and well-test templates can save you weeks of development and testing. At Postmark, we offer an extensive set of pre-made templates and best practices guides to help you get up and running as quickly as possible. In all honesty, though, we’ve open-sourced our transactional email templates, and, with some slight modifications, you can use them with any provider. We also built and open-sourced MailMason, a complete toolset to streamline building and updating a set of consistent transactional emails. 

4. Customer Support #

When all else fails and your email service providers tools can’t help you get to the bottom of delivery or other issues, you’ll need help, and customer support can be a huge difference.

  1. Does the provider respond to support emails in a timely manner? If you’re having problems, and your provider takes a day or two to respond, you can be in a tough spot. Send them some test support emails before you set up a paying account to see how serious they are about taking care of customers.
  2. Does the provider staff their support team with knowledgeable technicians? It’s never any fun to ask a technical question and get back a non-answer, and with email delivery, most questions will be technical. Send a reasonably challenging technical question to gauge their effort and competency.
  3. Does the provider have a robust and informative status page? Is it updated frequently? Are they transparent, or do they gloss over issues? Does the status page provide deliver information? With Postmark, we not only share our system status, but we expose our delivery data from our third-party monitoring service. We’re so confident we have nothing to hide, we even put the data right on our home page.
  4. Search Twitter to gauge customer support. Do they respond to tweets in a timely manner? Are they proactive and helpful? Are there consistent patterns of praise or criticism from end users? Nobody’s perfect, but consistent trends of negative comments should raise some flags.

5. Cost & Value #

Of course, you don’t want to ignore cost, but once you’ve reviewed providers based on the above facets, you may find that lower-cost providers have some hidden costs in terms of reliability, support, hidden charges for support, or productivity.

  1. Do you have to pay extra for a dedicated IP address? Some providers appear cheaper on the surface by offering lower quality shared IP delivery but then advising you pay extra for a dedicated IP address once you run into delivery issues. At Postmark, we monitor our shared IPs closer than anyone, and since we don’t allow promotional emails, we receive fewer spam complaints than other providers. And we have incredible deliverability to back it up. However, if the time comes where it makes sense for your account to be on a dedicated IP address, we’ll handle it, and, more importantly, we won’t charge you extra for it.
  2. Do you have to pay extra for priority support? Many providers charge extra for better or more responsive support. The per-email costs may be lower, but once you add in the cost of upgraded support, the total costs go up quickly. At Postmark, we offer the same incredible support to everyone.

6. After the Decision #

Regardless of who you choose, make sure to stay on top of your email game. Deliverability is a moving target. So don’t just evaluate deliverability when choosing, watch your provider like a hawk for as long as you use them.

  1. Don’t assume that great delivery is a given. Continue monitoring blacklists and delivery with tools like 250okGmail Postmaster Tools, and MX Toolbox. Great delivery isn’t perpetual, and you’ll want to know as soon as problems arise.
  2. Separate bulk and transactional sending. Send your application/transactional emails from a different domain/subdomain and IP address from your bulk promotional email to ensure that your bulk messages don’t hurt the delivery of your transactional messages. That’s advice straight from Gmail.
  3. How easily can you switch providers? Abstract the vendor interfaces (bounce handling, sending, inbound processing, link tracking, open tracking, etc.) from your codebase so you can more easily switch providers if they turn out to be sub-par. If a catastrophic delivery issue occurs with your provider, your ability to switch providers in a timely manner could be the difference in great delivery or thousands of lost emails.
  4. Do you need a backup provider? If you run a mission-critical application, consider maintaining a backup provider in the event there are long-standing or regular intermittent delivery issues. At Postmark, we offer a complete status API to help facilitate better monitoring for easier switching to a backup provider in the event of delivery issues.

Transactional email is critical infrastructure #

Sending emails to customers may seem simple on the surface, but there’s quite a lot to consider. Email delivery isn't a commodity, and all email providers aren't created equally. The differences become especially apparent once you dig into whether a given provider focuses on promotional email or transactional email. Hopefully these tips make it a little easier to compare providers and choose one that will serve your needs best.

Still have questions?

  • Dana Chaby Dana
  • Marek Loder Marek

Ask us anything! We’re eager to help you with any problem or question you have…