An introduction to inbound email parsing (What it is, and how you can do it)
Although email may seem simple at surface level, each email that is sent actually contains quite a bit of data that can be useful. This data can include basic fields like the Subject line, To and From addresses, and when the message was sent, but can also include detailed information such as the full message content, full message headers, and even attachments.
Processing all of this data manually can be very difficult and time consuming. Email parsing can remove much of this complexity and help you start utilizing the data in your inbound mail streams.
What is email parsing? #
Email parsing allows you to provide an email message, often by sending it to a specific email address, and then have the email message represented in a data structure that is easy to work with in code, such as JSON. There are many potential uses for this data, for example:
- Your application allows people to send or reply to tickets and you need a way to take action on their response in code
- You want to maintain a record of inbound messages in your database
- You have a social application that allows users to post and comment on other posts via email
- Automating order fulfillment in e-commerce
- We even built an app that utilizes Postmark’s Inbound Webhook to parse incoming messages, which are then passed along to two AI models for sentiment analysis and a content summary
Here is an example of an email that has been parsed into JSON:
{
"From": "myUser@theirDomain.com",
"MessageStream": "inbound",
"FromName": "My User",
"FromFull": {
"Email": "myUser@theirDomain.com",
"Name": "John Doe",
"MailboxHash": ""
},
"To": "451d9b70cf9364d23ff6f9d51d870251569e+ahoy@inbound.postmarkapp.com",
"ToFull": [
{
"Email": "451d9b70cf9364d23ff6f9d51d870251569e+ahoy@inbound.postmarkapp.com",
"Name": "",
"MailboxHash": "ahoy"
}
],
"Cc": "\"Full name\" <sample.cc@emailDomain.com>, \"Another Cc\" <another.cc@emailDomain.com>",
"CcFull": [
{
"Email": "sample.cc@emailDomain.com",
"Name": "Full name",
"MailboxHash": ""
},
{
"Email": "another.cc@emailDomain.com",
"Name": "Another Cc",
"MailboxHash": ""
}
],
"Bcc": "\"Full name\" <451d9b70cf9364d23ff6f9d51d870251569e@inbound.postmarkapp.com>",
"BccFull": [
{
"Email": "451d9b70cf9364d23ff6f9d51d870251569e@inbound.postmarkapp.com",
"Name": "Full name",
"MailboxHash": ""
}
],
"OriginalRecipient": "451d9b70cf9364d23ff6f9d51d870251569e+ahoy@inbound.postmarkapp.com",
"ReplyTo": "myUsersReplyAddress@theirDomain.com",
"Subject": "This is an inbound message",
"MessageID": "22c74902-a0c1-4511-804f-341342852c90",
"Date": "Thu, 5 Apr 2012 16:59:01 +0200",
"MailboxHash": "ahoy",
"TextBody": "[ASCII]",
"HtmlBody": "[HTML]",
"StrippedTextReply": "Ok, thanks for letting me know!",
"Tag": "",
"Headers": [
{
"Name": "X-Spam-Checker-Version",
"Value": "SpamAssassin 3.3.1 (2010-03-16) onrs-ord-pm-inbound1.wildbit.com"
},
{
"Name": "X-Spam-Status",
"Value": "No"
},
{
"Name": "X-Spam-Score",
"Value": "-0.1"
},
{
"Name": "X-Spam-Tests",
"Value": "DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,SPF_PASS"
},
{
"Name": "Received-SPF",
"Value": "Pass (sender SPF authorized) identity=mailfrom; client-ip=209.85.160.180; helo=mail-gy0-f180.google.com; envelope-from=myUser@theirDomain.com; receiver=451d9b70cf9364d23ff6f9d51d870251569e+ahoy@inbound.postmarkapp.com"
},
{
"Name": "DKIM-Signature",
"Value": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=wildbit.com; s=google; h=mime-version:reply-to:message-id:subject:from:to:cc :content-type; bh=cYr/+oQiklaYbBJOQU3CdAnyhCTuvemrU36WT7cPNt0=; b=QsegXXbTbC4CMirl7A3VjDHyXbEsbCUTPL5vEHa7hNkkUTxXOK+dQA0JwgBHq5C+1u iuAJMz+SNBoTqEDqte2ckDvG2SeFR+Edip10p80TFGLp5RucaYvkwJTyuwsA7xd78NKT Q9ou6L1hgy/MbKChnp2kxHOtYNOrrszY3JfQM="
},
{
"Name": "MIME-Version",
"Value": "1.0"
},
{
"Name": "Message-ID",
"Value": "<CAGXpo2WKfxHWZ5UFYCR3H_J9SNMG+5AXUovfEFL6DjWBJSyZaA@mail.gmail.com>"
}
],
"Attachments": [
{
"Name": "myimage.png",
"Content": "[BASE64-ENCODED CONTENT]",
"ContentType": "image/png",
"ContentLength": 4096,
"ContentID": "myimage.png@01CE7342.75E71F80"
},
{
"Name": "mypaper.doc",
"Content": "[BASE64-ENCODED CONTENT]",
"ContentType": "application/msword",
"ContentLength": 16384,
"ContentID": ""
}
]
}Your application can easily access any of these fields as needed, or the entire message can be saved to your database to be accessed at any time.
Structured vs. Unstructured Emails #
A structured email will typically be an email generated by a process or application that contains the same layout and types of information each time it is sent. Examples of structured emails include order confirmations, new replies to a message or support forum, and form submissions on a website.
Unstructured emails can come from any source and don’t follow a set template. They can also include any type of content. Some examples of unstructured emails are promotional emails, direct one to one messages, and newsletters.
Postmark’s inbound parsing does not differentiate between structured and unstructured email - it can parse both! Structured emails are typically easier to work with after being parsed due to the predictable nature of the content, but there are many uses for unstructured emails as well such sentiment analysis.
How to use Postmark for email parsing #
In order to start parsing emails with Postmark you will need to set up an inbound webhook. Each Postmark Server comes with an associated inbound message stream, which has an inbound email address that you can use to receive emails as JSON. The inbound email address will be in the format of a GUID@inbound.postmarkapp.com, such as 482d8814b3864b2c8ba7f7679fc116bf@inbound.postmarkapp.com.
You can access your inbound email address in the Setup Instructions and Settings pages of the server's inbound message stream.
Each inbound message stream can have one inbound webhook URL. Setting the inbound webhook URL for a stream tells Postmark where to send the JSON for emails received at the inbound email address for that stream or the stream's inbound forwarding domain.
When logged into Postmark, select the inbound message stream and go to the Settings page. The Webhook field is where you input your webhook URL.
Once you enter a URL for receiving inbound webhooks you can use the Check button to confirm that your URL is working as expected. When you check the URL we will send an example inbound webhook to your URL and let you know if we get back a 200 HTTP response code (success) or an unsuccessful response from your URL.
Once you have configured your inbound webhook URL the next step is to test its ability to correctly process inbound emails by sending an email to your inbound email address. You can also try sending multiple emails with different formats (plain text, HTML, attachments, etc…) to ensure that your application is handling all scenarios correctly.
Next steps #
In addition to parsing messages by forwarding them to the email address provided by Postmark, you can also set up Inbound Domain Forwarding. This feature allows you to configure your MX records so that all email sent to a domain or subdomain is processed by Postmark. For most use cases we recommend using a subdomain with this feature, as modifying MX records on your main domain will modify how all email sent to your domain is processed.
Here are some additional resources to help you get started with inbound message processing with Postmark:
Implementing Inbound Processing from the Postmark manual
Inbound processing user guide from the API user guides
Inbound webhooks from the API documentation
If you are already using Zapier and want an email parser or just need a place to host your webhook endpoint and don’t want to write any code then Zapier webhooks can be used as your endpoint. Using Zapier as your email parser endpoint also allows you to easily take advantage of the other tools and integrations provided by Zapier using the data parsed from your email.
