What we've been up to...
It’s been way too long since we published anything on the blog. This, of course, does not mean we’ve been on vacation. We’ve had some incredible growth in the last few months which has put a lot of our focus inward on our infrastructure and support to help the new (and existing) customers. I wanted to give you a quick update on what we’ve been doing behind the scenes.
Improving bounce and sent tracking #
When we launched sent tracking, we started using MongoDB to store the millions and millions of emails that are archived. Overall it’s a nice write once and read often situation, which means we can take advantage of indexing and caching. We have discovered some issues along the way which many of you have seen. Since then, we moved MongoDB into Amazon EC2 with some high memory instances to help support the load. So far it is working well. One advantage we have is that our data center is literally across the street from the AWS DC, so latency is low and throughput is great.
Even though things are working, we’ve decided to move on from Mongo in exchange for another solution. We’ll post about this when it is up and running.
Better redundancy and failover #
When designing a server architecture you have to figure out what you are designing for first. For instance, in Beanstalk we can afford a little downtime, but can’t afford data loss. In Postmark, we can afford some data loss for archived emails, but can’t afford downtime because emails would be lost. We strive for both, but setting priorities is key. From day one we’ve had good redundancy and sharding built into Postmark, but what happens if our Rackspace data center goes down? We’ve been working on a new big picture strategy that includes a mix of our dedicated environment at Rackspace, Amazon EC2 and geographic load balancing through our awesome DNS provider Dynect. We’ll keep you posted as we make progress.
Connecting through SMTP #
As we grow, we’ve noticed some ways that we can improve performance in how we process emails through SMTP. We’ve had a few cases where some emails were queued and delayed, so we want to ensure it doesn’t happen again. We’re working on a completely new system for accepting and processing emails through SMTP that will not only improve performance, but help with our redundancy strategy above. We’ll also get some nice add-on features like TLS and SSL support.
A lot of this is right around the corner. I just wanted to give you a quick review and let you know what we’ve been doing. We have some new features coming as well, but you will hear about them when they are released.
