Demystifying inbound email processing with JazzHR

Sean: 00:06 That was just trying to get into a very simple boolean decision based off content because things would come through poorly formatted, completely randomly formatted. All of the issues with routing and subdelegation of MX records, all of that stuff was just a nightmare.

Marek: 00:23 Hello and welcome to Talking Email with Postmark. I'm your host Marek Loder and in today's episode my guest Sean McAfee, director of Tech Ops at JazzHR and Patrick Graham, Technical Lead on Postmark's Customer Success team, talk to us about inbound email.

Marek: 00:38 We'll first discuss how inbound processing and parsing works and then explore how JazzHR, a leader in the recruiting software industry utilizes this functionality to power two way messaging for their platform. We hope that you'll come away with some ideas on how you might be able to implement inbound email for your business or side project. Hope you enjoy the episode.

Marek: 00:57 Thanks for joining me today guys.

Patrick: 00:59 Thanks for having me, Marek.

Sean: 01:01 I'm glad to be here. I'm very excited for being first guest in your podcast.

Marek: 01:06 We're excited to have you. Sean, let's get started with some quick intro. You want to just tell us a little bit about yourself and your path to Tech Ops over at JazzHR?

Sean: 01:14 Yeah. My name's Sean McAfee, I've been doing this for about 15 years. Started at a startup that was involved in public health disaster preparedness. We got bought by a healthcare IT company that got a little old a little quick.

Sean: 01:28 I landed at what was at the time The Resumator, we've rebranded it a couple of times since then. In my role of director of Tech Ops, my team is in charge of compliance, system operations, Dev ops work, automation, Bill Process, those types of things as well as what vendors do we use, what's the platform that we provide to the developers to go from code they write to what users see. That's how we get involved with Postmark and similar vendors in my current role.

Marek: 02:01 Cool. We're certainly going to get into that in a moment. Patrick, you've been a customer advocate at some pretty noteworthy companies from the likes of Uber to DocuSign. Do you want to tell us about your path to technical support and how you ended up at Postmark?

Patrick: 02:16 Sure. Back in 2012, I started working for a legal services company called ABC legal in Seattle. I worked for them for a year or two, just doing general support. As I got more interested in the technical space, I started working for Uber as one of their first remote support reps.

Patrick: 02:34 When eventually I wanted to get a little bit more technical, I joined DocuSign as a Technical Customer Success Manager, worked for them for a couple of years and eventually found that my work-life balance was getting a little bit out of whack, so luckily came across Wildbit and started working for them on the Postmark Customer Success Team.

Marek: 02:53 Cool. Well, thank you guys for that. Sean, I'd like to just extend an extra special thanks to you and then rest of the JazzHR team for being the first Postmark customer brave enough to actually join us on the show. Thank you.

Sean: 03:05 You're welcome.

Marek: 03:06 All right. Well, Sean, do you want to kick things off and just tell us a little bit about JazzHR and give us a brief overview of the company's history?

Sean: 03:15 Yeah. JazzHR is... we call ourselves an Applicant Tracking System. It's recruiting software and we are focused on helping companies grow by exceeding their recruiting goals. What we do is we replace the manual hiring efforts, inbox recruiting is something we use fairly frequently, with tools and automation to help spread the reach of your job and then deal with the avalanche of candidates that you hopefully get because we're so good at what we do.

Sean: 03:46 We started, 2009 ish as the Resumator, for a while there it was pretty buzzy company before a bunch of other first mover disadvantage. We rebranded sometime around 2015 to Jazz. We started getting the perception in the marketplace that we are a resume library or something like that. Resumator is a nice, cute name, but it implied things we weren't. We rebranded the Jazz in 2015 and then augmented that to JazzHR a little later when we found it hard to do SEO improvements on the word Jazz.

Sean: 04:23 We're focused on the small business arena. That's where a lot of people have those frustrations. We're not trying to solve every HR problem. We're not going to be ADP and building like this all in one HR IS. But there is a underserved market for tools like that where you only need something to help you do what you want to do.

Sean: 04:44 You don't want to do things... You don't want to mold yourself to the tool. That's where we fit in. Very easy to start using us. Very easy to start getting results and it's fairly lightweight in terms of what you actually have to do to use it. That's what we do. That's what we're focused on. That's our go to market and it's working.

Marek: 05:04 Well, thanks for that. I'm certainly excited to learn a little bit more about the workflow of the platform itself and we'll get into that in a moment. Before we touch on that, I'd like to clarify for our listeners, what we mean when we talk about inbound email processing and parsing.

Marek: 05:22 Patrick, do you want to give us just a quick overview of what we're talking about here?

Patrick: 05:27 Sure. We're talking about the ability for companies to receive inbound email from their users and then bring that into their own application and do additional processing or whatever they need to do with that to parse email once it's inside of their own system.

Marek: 05:43 Cool. I guess just to make sure that we're all on the same page. How is this type of receiving different than let's say how you'd be receiving an email in a regular mail client?

Patrick: 05:54 Sure. When an email is sent out the sending mail server, it'll check the domain of the recipient and check the MX record of that domain and then send the email over to the receiving mail server using that MX record.

Patrick: 06:07 Typically what would happen is that mail server would then populate that email into somebody's email client in their inbox, hopefully. What happens when you use a inbound processing and parsing with a service like Postmark or another ESP is instead of that final step going to the email client up to the recipient's inbox, that way what would happen is your own application essentially becomes the email client where you accept that email with your own application, process it and that displayed your users however you see fit.

Marek: 06:39 Cool. It sounds like based on what you're sharing, there's actually a fair amount of similarities between how a traditional mail client receives a message and what inbound processing and parsing is with the service like Postmark or another ESP. Can we just take it one step further and perhaps you could just give us a high level overview of what's going on behind the scenes to process and parse an email in a service like Postmark?

Patrick: 07:04 Sure. To process emails with a service like Postmark, how we handle it at Postmark is you would actually use an MX record like normal, but you actually point to inbound up postmarkapp.com and that tells the senders who are sending an email to your domain, "Hey, send this over to postmarkapp.com, I'm with that inbound subdomain."

Patrick: 07:24 What happens is Postmark will receive the email instead of it going to your own mail server or some other hosted mail server, we'll take it, we'll parse it on our end and then turn it into a well formatted JSON. Then we'll post that to your own URL, so then you have the data in your application and then you can show to your users, trigger some additional processes based on the incoming email, anything like that.

Marek: 07:49 Cool. Thanks for that overview, Patrick. So getting back to you, Sean, I'd love to learn a little bit more about the role that email plays for JazzHR. Can you speak to that?

Sean: 07:59 Certainly. I mentioned earlier that what our application really does is helps conduct your hiring process. There is basically two audiences for email, our users and then applicants or candidates to a job.

Sean: 08:12 The very different use case for the users it's primarily transactional password resets, things like that, you have a new applicant, traditional messages, you can get a daily digest of the applicants you've gotten.

Sean: 08:25 For the candidate side, that's how all the communications routed. For example, internally, our VP of HR, Corey is the one who actually does the phone screens. What happens is a candidate comes in, we all get everyone on the hiring team. Everyone involved in hiring gets a message that says you have a new candidate and that's all sent through our ESP.

Sean: 08:46 When that comes in, you log in and you can put feedback and you can at mention, you can do all these things and it keeps the communication log in the app and based off your settings, it may send those emails out to the other people in the hiring team.

Sean: 08:59 I could say at Corey, "Hey, really liked this guy. He'll get an email that says I've left feedback. I really liked this guy. From the candidate side, that's how the communication's done. Someone applies and we have the ability to send an automated email. Maybe you say how much experience you have in customer service, one to two years, three to five years, five to seven.

Sean: 09:22 If you don't select five to seven, you automatically get rejected and you get an email that we allow you to schedule so it doesn't look like you automatically got rejected immediately. It says, "Hey, sorry, here's... we're looking for someone else." Whatever the Preqin template is. If it is someone that you're interested in talking to, you can send them a message and say, "Hey, let's talk." They can reply.

Sean: 09:44 When they reply, it goes into their profile in the app instead of just sitting in that HR managers inbox. You can actually see the life cycle of the communication inside of the candidate profile in the app. It really opens up hiring, no more folded email chains, "Hey, hiring manager, this is what this person on the interview thought." All of that just lives inside the app.

Sean: 10:09 That's really the primary thing is that communication with the candidate, that's the important thing that we capture replies. If you reply to your password reset email, you get a no reply. We don't even have that pointed at Postmark because we're just going to reply with the no reply.

Sean: 10:23 It's really on the candidate side where Postmark brings that value because it does open up email is either one-on-one communication or use awkward mailing lists and list serves and things like that. This really just opens it up and makes it be a native communication in the app. No different than like if you're looking at a Facebook thread.

Marek: 10:41 Wow. So it... Clearly email plays a fairly central role in the value that you're bringing to the table. Yeah?

Sean: 10:49 Yeah. Especially, I mentioned the go to market, our biggest competitor is inbox recruiting, which is people live in an email. Since we're asking people to make the leap from email, which is still very communication driven, it's email to an app, they want to at least have that piece of it where they're able to easily communicate with the candidate in a format that's acceptable to the candidate.

Sean: 11:14 I've been on the other end of hiring processes where I'm expected to create an account in order to communicate with an employer and there's such a burden to people doing that on the candidate side, you may lose a good candidate because you chose to use a solution that uses a secondary or tertiary form of communication that you're not used to.

Marek: 11:37 For sure. Basically, the recipient or the applicant for the job, they get to live in their inbox as if they are sending an email back and forth with the company that they're applying to. It's just that you folks are effectively obfuscating that by a third party ESP, like a Postmark to basically manage that back and forth. Correct?

Sean: 11:59 Yeah. Because it's a thousand times easier.

Marek: 12:01 Cool. I just wanted to make sure we were all on the same page. I'd love to get a... You guys have been around for some years. I'd love to get a brief history of how you guys have managed the sending and receiving process since you guys were founded.

Sean: 12:17 This predates me by far. It's way before my time, but one of the guys on my team was employee number four or something like that. I talked to him and what he expressed was they thought about doing email for two minutes and did like a feasibility thing and said, "Oh man, this is complicated." Then just went and found an ESP and said, "We're done with this."

Sean: 12:40 We've actually been doing inbound parsing through multiple vendors. We're on you guys and we like you guys now, for 10 years now. It's been the crux of how we made those inroads the inbox recruiting, like I mentioned earlier, 2009 that was the predominant way, even Indeed was it Indeed just yet?

Sean: 13:02 It was just the job board. There's monster or you sit in your inbox or you buy something really expensive and cumbersome from ADP. Being able to tell people that we're not changing things too much, but we're making it easier was very important. One of the ways that we could make it easier for them is by running a mail server for them, but we're not going to do that. They pretty much went directly to the ESP model just because of the horizon.

Sean: 13:32 You can see the horizon and where it ends when you implement something with ESP. Mel server, it never ends. It never ever ends. There's always something. There's always going to be some edge case. There's always going to be something that breaks. There's always going to be some auto reply, auto response fight with someone that fills up your queue.

Sean: 13:53 You're always going to screw up a post super command and actually do post super dash D all instead of doing post queue. All of those things just happen. I'm glad they made that decision, but they've been on it from the start.

Sean: 14:08 That's always been a core feature of ours, is how you do that communication. Because think about hiring prior to HR's systems and things like that. It would be sending in a resume, you get a phone call, you have a conversation, you show up, you get hired.

Sean: 14:24 We've kept as much of the spirit of that as possible. We're not trying to dehumanize it or make it be impersonal, but we also need it to be usable. Early on they took that approach of we're trying to get people away from using email, but candidates still need to use email. What's the good balance between the legacy operating model and efficiency, both for the customers and ourselves?

Marek: 14:52 It sounds like you weren't necessarily part of the original decision making team that chose to go with a third party ESP, but do you have any experience yourself managing mail servers and managing perhaps something similar to let's say inbound processing and parsing on an internal mail server?

Sean: 15:08 Oh my, yes. I mentioned before we were in... I was in public health disaster preparedness, so we would actually be delegated .gov domains and things like that and we'd have to... we had a handful of Mel appliances and home rolled servers and things like that, that would manage basically auto-reply. This was a while ago.

Sean: 15:31 Are you available to help out in this public health emergency? Yes, no. We're doing all that manually. It was terrible. It was absolutely terrible. That was just trying to get into a very simple bullying decision based off content because things would come through poorly formatted, completely randomly formatted, all of the issues with routing and subdelegation of MX records, all of that stuff was just a nightmare. I keep saying it, but it's just never ends.

Sean: 16:02 It's just... There's always something. We had the one Mel appliance vendor we spent a significant amount of money on. They were pretty good. We liked them. Their domain expired and it's six 30 in the morning and I come in and it's weird. I can't check the mail cues. What's happening? Because it's my pimple face newbie, I'm on email dump box duty. What's going on here?

Sean: 16:26 It's around 7:00 AM by the time I pick up the phone and I'm calling Israel and it's like, "Hey, you know you're not on the internet anymore. Everything is broken and we can't get email because it can't check for new definitions that it felt closed."

Sean: 16:38 It's... We spent all this money to outsource this to an appliance and even that can't stay up. Email's hard, emails really hard. Even the people who were experts in it, we found ourselves having issues with.

Marek: 16:53 Yeah. It sounds like your team from the get go made a good choice to offload that burden onto an ESP. There's just so many things that can go wrong.

Sean: 17:04 Yeah. If I had to come in and the interview and they'd said, "Okay, you're day to day. You're going to be managing this postfix install. Hey, do you know anything about dove caught?" It's on my resume, but that's not what I'm looking for.

Sean: 17:14 It probably would've been a very different conversation, if we had been talking about managing email again. It's nice to have G Suite and Postmark I don't deal with email. It's amazing. It's so relaxing. Well, no, I should... I do have an email server that I use for testing because we can actually see the full cycle and I hate it.

Sean: 17:35 But it's a necessary evil email would throttle like automated testing user because of suspicious unreasonable activity and all these things so we'd have automated tests that would hang.

Sean: 17:47 Postmark's really good at delivering. Unfortunately, SMTP transaction Gmail says "250 OK, thanks." That's good, but it doesn't show it to the user for 10 minutes for who knows, whatever tarpits and reasons.

Sean: 18:03 The only reason why I will ever defend running your own email is if you need absolute certainty and tracking the email from send to showing up in a user inbox. That's my one disclaimer.

Marek: 18:16 All right, well, so just getting back to the history of how you guys have managed your sending, you started with the third party ESP, but you're clearly no longer with that ESP because you folks are now sending and receiving through Postmark. I'm assuming that there were some things that were either missing or broken. Can you tell us about your journey to Postmark?

Sean: 18:33 Yeah. I'm going to try to be somewhat sensitive in what I say. Assume that I'm actually twice as outraged as I'm expressing. Until 2018 if you wanted to export a list, a simple list, recipient, subject date, you had to download the mobile app. That was their official recommendation.

Sean: 19:01 You want a copy of an email, you want to see what the content is of this email. Well, here's this knowledge base article that involves things like setting up a Gmail account, setting up Zapier to insert a record into bring your own objects storage, all of these table stakes features, I can't export a table from a report as a CSV in 2018 unless I get an IOS app. That's stuff, that was just... I didn't get it. After talking to Postmark, it's very focused on... in the industry you build per transaction.

Sean: 19:38 You build per message, that's the standard. You either get a trans or tranche or per message or whatever it is. But the game for an ESP is essentially volume when you're at a certain size.

Sean: 19:51 Their features were not created to help us send email. They were created to help more email be sent. It was very obvious once we moved away that that's what it was about. It was to the point where the big final straw for us was they sent an email because some analysts saw three messages that had suspicious looking, spammy content.

Sean: 20:19 They were spammy content. It was someone who got through our fraud review and started sending on a trial, sending out messages. It was three messages out of 1 million in the last 14 days that he saw.

Sean: 20:31 He was able to, as junior support analyst, press suspend on our account and completely shut down our account. We would get fail transactions trying to send messages. Obviously that's not great for us, not even getting into the qualitative assessment of whether or not they should have been able to do that, but just blindly shutting off is bad.

Patrick: 20:54 Did he... Sean, so just quick question on that. They didn't even cue up the messages or anything. They just went ahead and shut it down to where you were making send requests and they just were not being sent at all and not stored for sending later?

Sean: 21:06 Yeah.

Patrick: 21:07 Oh, wow. Okay.

Sean: 21:07 They were just SMTP username, password fail. That was just... that was bad. It's the middle of the day, so we call support, we get put on hold and we're telling them this is unacceptable. Turn it on and we'll figure this out, but you cannot have us turned off.

Sean: 21:25 They hung up on our CTO and then made his number route directly to voicemail. This is the support line of a company that we're paying thousands of dollars a month to, and they basically put phone calls from our CTO into the circular file immediately.

Sean: 21:43 That was the final straw and why we started looking. Usability is one thing. In this world you have to deal with the things you can't change, but being treated the way we were treated was the final straw.

Marek: 21:57 Well, so even with all the technical considerations of email, it was really... it came down to almost the support you received from the company that was the main impetus to move over to Postmark.

Sean: 22:09 No empathy-

Marek: 22:10 Yeah.

Sean: 22:11 ... is the problem.

Marek: 22:13 I want to keep us moving. It sounds like most of the emails, just to make sure that we're on the same page, most of the emails that you folks are receiving are coming directly from the applicants themselves. Correct?

Sean: 22:24 Yeah. Some minor stuff with internal threads, but it's the vast majority is candidate facing.

Patrick: 22:32 I had one quick question on that too, Sean. When you guys receive an inbound email from a candidate, do you just send out a general alert to people interested in the candidate saying, "Hey, there's a new message from the candidate?" Or do you actually extract the content of that email and put that into an additional email alert that you send out to the hiring manager or whoever interested?

Sean: 22:52 The latter. That's something that we've had to talk to our success team about as well because when they help troubleshoot, since Postmark actually has good troubleshooting tools, crazy.

Sean: 23:04 You can actually see the SMTP message that the recipients server gave you. Since I've had to work with them a lot, I've had to explain to them that we send an email, someone replies to any email, it ceases to be an email. It's an internal message. Then we decide whether that internal message will result in further emails.

Sean: 23:21 Generally they get the content. We basically proxy out back to everyone else on that team, the contents of the message, if it meets the business rules that say this should be rebroadcast. There's some things where it's... a candidate has completed a questionnaire, they just get an email that says candidate's completed the questionnaire. But generally it looks like it's a native email formatted and styled for ourselves.

Marek: 23:48 Yeah. Sean, I definitely want to explore those business rules in a moment. Patrick, do you want to just give us a quick one, two, three, four how this inbound processing is working with Postmark specifically and then we can jump back into Sean's use case.

Patrick: 24:04 Sure. With with Postmark hen you have a Postmark server, it's going to have an inbound message stream and a transactional message stream. In the inbound message stream you can tell Postmark, "Hey, when an email comes into this domain or subdomain, go ahead and turn it into well-formated JSON and then post the details of that email at my webhook URL that I'm going to specify in this inbound message stream settings."

Patrick: 24:29 What happens is an email comes into that inbound domain or subdomain Postmark receives that email we convert it into JSON. We then post it to the URL that you've told us to post that domain's emails to.

Marek: 24:42 Cool. It sounds like one of the first steps in building out an inbound workflow is choosing the right domain or subdomain for that inbound forwarding. Sean, can you tell us what your decision process was with respect to choosing that domain or subdomain for JazzHR?

Sean: 25:00 Yeah, we actually... We don't really present a lot of JazzHR branding to candidates. We run what we call career pages, which are lists of the jobs that the company has posted. We actually run that on an alternate domain called applytojob.com. That's a newer thing that we did at rebrand. Previously, everything was on the resumator.com.

Sean: 25:21 One of the things that had been in place since 2009 was the domain that was chosen was dropbox.theresumator.com. That caused all kinds of confusion because of the company Dropbox, right?

Marek: 25:35 Sure.

Sean: 25:36 We'd have customers being like, "Oh, why are you sending my stuff to Dropbox? You haven't told us that you share it with them." And all these things. There's this challenge of trying to make something seem innocuous, but also not making it seem shady.

Sean: 25:52 There's that fine balance. We actually ended up, I can't even remember now off the top of my head. I think it's MSGS, it's something like that, some abbreviation of like messages.

Sean: 26:05 Because that we did a couple of, I can't remember the service, where you just give the poll. It's like a UX testing thing and we put out a, which one of these seems the least shady option and that one by far it was no real close and we even put super shady looking things in there as the test to make sure people were being honest.

Marek: 26:29 Forgive me if this is just a silly question, but is there any real reason why you folks wouldn't have chosen, let's say, a subdomain of JazzHR?

Sean: 26:41 That just has to do with the separation. We actually don't even run our primary app off of JazzHR. We do have some separation there, but the big thing and why we chose the non Jazz domain was for the non branded, unbranded thing on the candidate facing side.

Sean: 27:02 There are still some contexts where we send out messages that have the Jazz branding on it to internal users, but for the most part we use that obscure but not shady solution. Just because we're not trying to supersede an employer's brand with our own.

Patrick: 27:21 Okay. You're almost white labeling for your own clients. The people doing the hiring. You don't want JazzHR going on all their emails that they're sending to their candidates and faculty?

Sean: 27:31 Flutter logo and part of that is because we've also had issues where we've been glowing flint up on Glassdoor or there's been some misunderstanding where people thought they were applying for a job with us and they weren't.

Sean: 27:48 Part of it is the separation of candidates for its customers and trying to be somewhat brand neutral. I wouldn't go so far as white labeling, but we are fairly conservative. We have a flutter logo and that's about it.

Marek: 28:02 Okay. So it's just to mitigate confusion for the applicants for your customers companies?

Sean: 28:08 Yeah. You should always defer to your customers branding.

Marek: 28:12 For sure. Patrick, are there any other considerations to be mindful of there when choosing a subdomain or domain for inbound forward?

Patrick: 28:20 Yeah. Definitely the biggest one is a lot of times people want to receive inbound emails still like normal with their Gmail. With they've got their own root domain. It's myapp.com. They still want to have their employees able to receive emails with their generic john@myapp.com addresses without those emails instead being routed through Postmark.

Patrick: 28:43 You'll definitely want to use a subdomain like inbound@myapp.com or whatever your domain is so that only those emails going to that subdomain get processed by Postmark, so you can continue receiving your normal day to day communication emails internally with something like G Suite or or Exchange or office 365 or whatever.

Sean: 29:03 That's actually a good point on the corporate side. We also tried to provide some insulation. Email is messy and you can't control what other people do. Because of the nature of our business, you can sign up for a trial and communicate with candidates. Even some candidates, have applied to jobs, we know they've applied to a job and then they start reporting these messages from someone who wants to hire them as spam.

Sean: 29:23 We get the FBL, the feedback loop emails, although no longer from AOL since they migrated all their architecture to Verizon or whatever. It would be these people are reporting spam, like job offers.

Sean: 29:34 We wanted to make sure that we didn't end up co-mingling our corporate enterprise email with basically user injectable garbage in the email, you know, in the global email infrastructure. Just because, say we did have an issue where we got RBL to on our domain, we don't have email, who was supposed to email. We can't even email people to fix it because our email's broken.

Sean: 29:57 Now we're relying on our own Gmail and things like this that look unofficial. Because we have a shared risk pool with the sum of all of our customer's activities, insulating ourselves a bit from our primary corporate email, and this is ESP aside was somewhat of a consideration when we did the rebrand.

Marek: 30:20 It sounds like absolutely at the very least a subdomain. In your case, Sean, it sounds like you guys decided to take it one step farther and actually choose a completely different domain that's separate from your company domain.

Sean: 30:34 Is definitely consideration. There's the concept in the ops word of out-of-band and if you put your customer facing customer usable email in your normal email, you no longer have an out-of-band channel should something go wrong.

Marek: 30:48 Cool. All right, well just moving on here, I take it that the next big step as Patrick had mentioned, is to receive that email. Can you walk us through the process that you guys went through Sean to create the webhook and maybe talk us through what happens next?

Sean: 31:07 Yeah. We already had the framework for the webhook done. What the underlying data model is the primary fields, the headers of an email. The data model is already built for you.

Sean: 31:18 It's who's sent it? Who are they replying to? What's the message I'd? That kind of thing. When we went to Postmark, we took... I'm not going to say we did a full rewrite, but we took the opportunity to clean it up a bit. We already had that model. Thankfully, the data model was already in place.

Sean: 31:35 We built it knowing that there would be more functionality provided on the call backs. It's not just inbound parsing, but it's facts about outbound parsing. We built something that was adaptable. Previously it was very targeted to just accept this message and this format and dump it into the database.

Sean: 31:56 But now it is able to communicate with Postmark more than just the dumb webhook that receives email address and turns it into relational data inside a database. That's where we went with that. We did take the opportunity to do some improvements on outbound side make it more asynchronous, moved fully away from SMTP to HDP.

Sean: 32:15 There are other outbound based things that we did. But on the inbound side, a lot of it was already prior art that we just improved and were able to simplify some of the error handling because we were getting inconsistent behavior from the previous vendor.

Marek: 32:35 Are there any, I guess key takeaways that came out of that cleanup that you'd be able to share with us?

Sean: 32:41 The big thing would be being able to write the same code and use it for multiple things. Even though there's a different callback address, Postmark... One of the things I like about Postmark is that you actually do different callbacks for different events.

Sean: 32:53 We're able to make what is actually one piece of code but with different end points to invoke separate things. Even though the same piece of code, we give it a fake end point at virtual URL that indicates when it's coming inbound, that it's a spam complaint or it's a readability hit or something like that or read hit.

Sean: 33:13 Being able to make it, not just receive a message but receive... not receive email content but receive any sort of inbound data from Postmark was probably the biggest win we have, and I'll actually name drop Nick Amoscoto. He was the engineer that built it. He did a very good job making sure that we could take advantage of the more full Postmark feature set as it related to end to end visibility.

Marek: 33:41 That's great. You took the opportunity while you were working on the inbound webhook to go ahead and implement a webhook mid spacely for all the other webhooks that Postmark offers, like bounces, spam complaints, opens, that kind of stuff.

Sean: 33:52 Yeah. We've got bounces in a feature guarded state right now. Everything else is... it's primarily UX consideration, which is why we haven't implemented it yet. We start to show how to use it, or you show it in the app, explain what that means, keeping content.

Sean: 34:08 Nick banged it out real quick and he's like, "Okay, here. Now we know whether or not a message bounced." It's the dog catching up to the car. Now what do you do? You start changing your tabular data displays and doing all this content.

Sean: 34:21 It's there and a technical level. It's just trying to make users understand email and you're seeing it here. It's... we're all close to email experts. We're having a hard time explaining how email works.

Sean: 34:33 Tracking that into some UX that we designed and then explaining to customers what this means in the app and also what this means for email, that's a pretty steep hill. That goes way beyond writing a webhook to receive well-formatted JSON.

Marek: 34:49 I guess, with that in the mess that email is, I imagine that with all of the emails that you folks are sending that are going out the door and then the replies that you guys are getting back from applicants, I take it that this can get messy really quickly.

Marek: 35:04 Sean, can you talk to us a little bit about how you guys go about keeping the house clean and linking conversations back up and creating those message threads. It sounds like you folks are still sorting through some of how to do that, but I would just love to get an overview from you on that.

Sean: 35:20 Yeah. We have... There's two different things you can respond to. You can respond to a message or you can send to what we call the candidate Dropbox and use that Dropbox word again. But it would allow you to send... You can just send a message to an email address and it shows up.

Sean: 35:42 You can Cc on an out-of-band communication. Say you wanted to conduct some outpost hiring process and you emailed your benefits broker to set up payroll, you could Cc that address and it would show up in that employee record.

Sean: 35:57 But the primary one is related to just replying to communications. We actually just have a unique id and when you reply to it, it's the unique id @subdomain.com. That's how it links it.

Sean: 36:12 In the data model it's this is this message, this is its ID, it is in reply to this message, where reply to it can be no. That's how we construct the chain.

Sean: 36:23 That actually became usable for us because we are doing some texting features and we wanted to see what a conversation cadence looked like, how many back and forths were there in any average communication and then take a weighting factor. "Okay. Now how many text messages would it take to replace that email?"

Sean: 36:43 It was really good that we were able to chase that stuff back because we did build that, receive it then proxy it back out, rather than just having the replies directly go. If we had not been doing the way we've been doing it and we'd just been doing an email, it'd be incredibly hard to reconstruct the flow of a message and the cadence of the conversation.

Sean: 37:04 It's used... We use the dedicated email address it's either the ID of the candidate or the ID of the message that you're replying to, and then that kicks it off through multiple inbound processing.

Sean: 37:18 We get the inbound parse and we say, "Oh, this is in response to communication. Let's go look up that communication. Insert it. It's in response to this one. It's new ID. Okay, so now what do we need to do? Do we need to resend it?" That kicks off that cycle again.

Sean: 37:33 Email's terminated, initiated and terminated on the Postmark side and then we just based off of the context in which the message was sent and the settings that the customer has configured for the behavior, either at the account level or on the job or in the hiring team, we decide whether we need to generate additional emails or not.

Sean: 37:55 Those additional emails would then essentially be seen as unique. They're not actual reply to... you don't see the same message thread inside the headers, the SMTP headers which has caused some issues with outlook.com automatic threading and an old versions of a fat client outlook's automatic threading because it doesn't have that SMTP transaction ID trail. But that's how we do. It is just unique ideas on the email address.

Patrick: 38:23 Yeah. I'm curious, Sean, do you guys... You said you use an ID, do you use any part before that hash, like a reply plus the ID @subdomaindotdomain.com or you just straight use just the ID @subdomain.domain.com for the address to process those inbound messages.

Sean: 38:43 It's funny you should say that, I mentioned before you use reply to headers. We send things out as no reply plus unique id and it's a short integer id that relates to a specific job.

Sean: 38:56 But the reply to is actually a more structured and longer one that's a little more deterministic. It starts with, for example, starts with C, if it starts with C, then we know that that is in response to a communication record.

Sean: 39:10 Its response to an email. That's one of the shortcuts that we have is, I know string parsing is not the best thing in the world, but you can cut out a bunch of more complicated logic and database records by putting the database lookups, by just putting in the little hint inside that email address.

Sean: 39:28 Instead of accepting an email and figuring out which path that should take, it comes in and self identifies the path that it wants.

Patrick: 39:35 Cool. That's awesome. I think maybe Marek the main takeaway there is don't try to rebuild threads based on in reply to headers on the messages or references headers and things like that.

Sean: 39:50 Yeah. That's the balance is you don't want to... you need to maintain email behaviors. But if you've recreated how SMTP works, if you've recreated how email works, then what was the point? You just don't have to run a demon.

Patrick: 40:06 Yeah, exactly.

Sean: 40:07 It's a very fine line and you break towards convenience. Like, "Why did we do that?" Like essentially namespace based addressing because it's easier. We don't have to write additional logic to say, "Oh, okay, so this has this header. Was this a communication response or is it to the candidate Dropbox? Hmm, how should we do this?"

Sean: 40:27 It self identified it because we set the header when we send the message. Why not on the outbound message, say, "Oh, and this is the route that you'll take coming back in." By just inserting a tiny one character hint into the beginning of the string.

Marek: 40:43 That's pretty cool. One of the things that you had touched on earlier, Sean, was some of the business rules and some of the logic that you guys have once you get this parsed email and the JSON back. Are there any things that you can share with us in terms of how you guys... what kinds of things that you guys are using to make decisions as to what to do with the replies that you guys are digesting back?

Sean: 41:08 Yeah, and I mean since this existed for so long, a lot of it is new development always built on previous development. A lot of that logic in terms of should we proxy out, should we reset and things like that, when we moved to Postmark was just, we built the end point to receive it and then we injected into the same space.

Sean: 41:26 All of that decision making and things like that is essentially portable because all you're doing is saying, "Hey, here's a class, here's a piece of code or a piece of a text that represents a message. Do what you will." That was good that that was already built and it evolved over the years. I'm not going to say what we have is perfect, but it is complicated. Like I said earlier about like recreate an email, you still need the idea of reply all.

Sean: 41:54 Because when you're doing inbound parsing, you're not actually replying to an email message, you're replying to a webhook that was communicated to you via email. You do have to recreate that and get that stuff down.

Sean: 42:09 It's hard because you do have to maintain the behavior of email without being email. There's some awkward parts where things don't necessarily work as you'd expect. Some of the things that we come across are stuff like we mask emails. When I talked about the auto reject emails, some people choose to put those under the company name instead of under the employer name.

Sean: 42:36 The problem is the employee's name, so the HR guy doesn't want his name on it. The problem with that kind of thing is that the email address is unique for that interaction essentially, but if you're just interacting with that... And when I said we use that mask outbound address and then the reply to is the real meat of the address.

Sean: 42:58 Email clients behave differently. I send out an email initially to you that says, "Hey, I'd like to talk to you." It comes from a no reply email address that's pseudo unique. Then later on I decide I'm not going to hire you. I send a message from my company saying, "Hey, we're not hiring you."

Sean: 43:16 It still has my name attached to it because of what the email client on the recipient side decided to do. We get all kinds of things like that where the email client tries to be helpful and injects garbage and does this and does that, Yahoo Mail, there's a specific bug and their next gen version, this is a year ago, that reply to just didn't work. They said, "Oh, we'll make reply to headers work someday."

Sean: 43:41 In the meantime candidates aren't getting jobs because they use Yahoo. I went off the rails there, but just being able to parse email and insert it doesn't mean that you're not going to have these issues because it's just systemic to email on the other side of what you're doing.

Marek: 43:59 Email's complicated.

Sean: 44:01 Emails are like printers. Just be glad they work.

Marek: 44:05 Email is still complicated despite using a service like a Postmark to streamline it a bit. Have you guys had challenges explaining what's going on behind the scenes to perhaps your less technical team members who are actually trying to troubleshoot some of these things?

Sean: 44:22 Oh my, yes. I've done many, many one-on-ones. It's really hard because everyone learns different and email is as complicated as you need it to be. The general concepts of email are similar, but that breaks down when we start to get in details.

Sean: 44:43 One of the big challenges we have with inbound specifically is when you go to outbound and you're looking to see someone... the customer says, "Hey, I was trying to hire this employee and they said they weren't getting my emails." We can go and we can go in the inbound or the outbound logs and Postmark and we can see when email was sent.

Sean: 45:02 We can see when their email server accepted it. We can see what SMTP exit code was given when they accepted it. We can see that the guy opened it a bunch of times.

Sean: 45:11 Yes, he definitely received it or someone who logs in to his email account received it. But then on the inbound side, it just... the trail seems to go cold.

Sean: 45:22 You see Postmark received the message, Postmark successfully put it into your application. When a candidate replies to the hiring manager and the hiring manager says, "I didn't get the email." The trial goes cold when you look on inbound because it's no longer email at that point.

Sean: 45:38 Postmark doesn't know that we don't give SMTP codes to Postmark, we just say, "Hey, we got it, we're good." Having to explain that candidates response did not go to the hiring manager, it went to our system. Our system decided that it was going to then send out another message that looks like it's a reply from the candidate, but it's actually a new message that we've copied the text from the previous reply into and generated and started the cycle over.

Sean: 46:03 That's where it breaks down is understanding that you have to explain email and people have to understand how email works and then you have to tell them, but forget all of this stuff in this case because it's entirely different than we do something else. No, it's not email anymore. That's been the difficult part. Is understanding that email terminates on your side and ceases to be email, bi-directionally one to take it off to Postmark or handed back to us.

Sean: 46:30 We have had some challenges with that. We do a lot of troubleshooting for customers. I've had an unfortunately high amount of IT managers who don't actually understand how their own email system works. MX records, if your mail server said that they don't know that domain, I can't do anything about that.

Sean: 46:53 I looked up food.com it said go here, I said, "I have a message for user@food.com." It said, "I don't know what you're talking about. I don't know what else I can do." That's where it ends.

Marek: 47:03 Sure.

Sean: 47:04 Oh, this message didn't hit my inbox while we logged into Postmark we saw 250 or 250 SMTP OK, message received, talk to your IT admin. It's no different than back in the day you call and you say, "Hey, is Johnny home?" "No he's out playing. "Can I leave a message?"

Sean: 47:21 Mom doesn't give the message. You don't know that. She said she'd take care of it. That's the two difficulties is that it's no longer full cycle email. Also that the recipient's email server is the authoritative final answer. Please, please, please stop opening tickets when someone did get a message when their email server said, "250 OK."

Marek: 47:42 It sounds like certainly there's still some points of confusion, but I'd be curious to know, Sean, are there some things that have been made easier since you folks have migrated over to Postmark to manage some of your sending?

Sean: 47:55 Yeah. Just on the activity log dashboard side light years away, previously what happened, in the old vendor you go and you hover over the information icon, that little eye circle that everyone has and it gives you the first line of the SMTP output.

Sean: 48:13 That's fine except SMTP 451 is an unofficial code that people use for different reasons. I want to see what the output is. I want the SMTP transaction id, so I can say... success team can reply and say, "Oh, you're an email server. Except that here's what our side saw."

Sean: 48:32 They can go look that up. We're able to say, "Here's everything we saw. You have to go figure it out." That just was not possible before.

Sean: 48:40 We couldn't even get IT harping on this. We could even export the logs of messages sent to an email address. You can copy paste 50 rows at a time and hit next or you can install a mobile app.

Sean: 48:51 In terms of troubleshooting, I've gone from trying to negotiate why it's not our problem with our success team to being able to teach them how to respond to a customer with, "Hey, sorry you didn't get it but your email server got it. Here's the message. Talk to your IT guy."

Sean: 49:10 That's been great to not have to negotiate with them on whether or not I'm going to look into something. They're able to go do it. They're able to provide fairly definitive final word, authoritative answers on why the person did or didn't get an email, a single email message.

Marek: 49:29 Just to be clear, that in it itself was a challenge with some of the other vendors that you guys have worked with.

Sean: 49:35 Yeah. You get no information there. They're meant for sending out newsletters. Being able to actually troubleshoot on an individual basis rather than on aggregate is just not possible.

Marek: 49:48 Cool. Well, before we wrap this up, do you guys have any just high level recommendations for maybe those listeners who are just starting to consider implementing inbound processing and parsing for their application and I guess specifically things that maybe they should be aware of or some of the requirements that they'll need to sort out as they get going with that?

Sean: 50:10 The end point for the webhook is the easiest part. It is entirely a UX and architecture decision. Now you've got this content, where are you going to do with it? That's the tricky part. So planning that out is key. The end Point is the end point. It is simple. I'm sure that someone has a postman collection that mocks out what you need to do to receive a message on inbound parsing from Postmark. What you do with it after that is that's the meat of the problem.

Marek: 50:42 Any tips on that side of it?

Sean: 50:45 Hire really good software engineers like Nick Amascato, but don't hire him, he's taken

Marek: 50:51 Well, this has been a great conversation guys. I think we can wrap things up here. I just want to thank both of you for taking the time to be with us today.

Sean: 50:59 Thank you.

Patrick: 51:00 Thanks Marek.

Marek: 51:02 To our listeners, thank you for joining us today for this episode of Talking Email with Postmark. If you enjoyed the episode, please leave us a review on iTunes and subscribe to receive updates on all future episodes.

Marek: 51:13 Be sure to check out the resources section for this episode where you'll find useful tools and helpful articles on inbound processing and parsing.

Marek: 51:20 Lastly, if you're ever looking for advice on how to set up inbound processing for your business or side project, be sure to reach out to us at support@postmarkapp.com and we'd be happy to help. See you soon.