Is Poor Message Formatting Hurting Your Inbox Placement?

The fight against spam continues to wage onwards, and there doesn’t seem to be an end in sight. Here at SocketLabs, we follow this war closely in order to understand how the mailbox providers are adjusting their defenses. Monitoring changes by mailbox providers allows us to better understand common issues our customers may be encountering while trying to get their own messages delivered.

A trend that started a few years ago which seems to be building momentum is more strict enforcement of the RFCs that dictate the syntax of email messages and their content. Unfortunately, the standards and requirements that outline how a message should be formatted are commonly violated by both legitimate senders and malicious spammers.

With their own users in mind, mailbox providers originally deployed systems that were very generous to those poorly formatted messages. Rather than rejecting a message that may have been legitimate, mailbox providers would often try to fix the formatting and syntax issues and allow the message to reach to the recipient.

Spam in general has a greater tendency to be malformed and not adhere to standards. This is partially from the sloppy untested code used to distribute these messages, but also some spammers have moved to purposefully distributing messages violating standards in hopes of confusing the filters. For this reason, mailbox providers are pushing the burden of adhering to standards back onto message senders. Here are some of the most common errors we find legitimate senders encountering.

Line length limitations and line feeds

In an email message, there is a maximum length of 998 characters on any given line. Exceeding this limitation will cause failures at an increasing number of mailbox providers. You can find more information about line length limits in RFC5322 Section 2.1.1 and RFC5321 Section 4.5.3.1.6.

If your messages exceed line length limitations you can expect to encounter the following error codes and bounces:

• 550 Maximum line length exceeded
• 500 Line too long
• 550 Message is not RFC5322 2.1.1 compliant

The limitation of 998 characters may seem like an odd number. The real limitation is actually 1000 characters, but every line of a message is expected to be terminated with a standard pair of ASCII characters. The termination sequence is defined as CRLF or Carriage Return Line Feed. Failing to properly terminate every line in an email with this sequence is another of the common mistakes. This will lead to messages being rejected by recipient mail servers with errors codes similar to:

• 550 5.6.0 Lone CR or LF in headers (see RFC2822 section 2.3)
• 451 stray newline detected. see http://cr.yp.to/docs/smtplf.html
• 451 See http://pobox.com/~djb/docs/smtplf.html

Note: The hyperlinks referenced in the above error codes can be helpful additional resources to understand line feed issues.

When a mailbox provider “fixes” line feed issues they commonly break the DKIM signature, which is an important message authentication mechanism. Handling of present but malformed authentication is a grey area for mailbox providers, and one of the attack vectors of which malicious users have taken advantage in the past.

Mistakes in header fields

When looking at the raw headers of an email message, it may seem like there is an infinite number of different line items, but there are actually only a few required lines. Often some of the most simple can be missed when generating a message. Some of the most commonly missed are Date and Message-ID.  This is again something that a mailbox provider could easily fix by adding a date of when the message was received and giving the message an identifier. Unfortunately the sloppy creation of messages missing these fields will result in a host of issues. Here are some common failure codes that you might find in your failures and bounces if you don’t properly set these fields:

• 550 Missing Date Header, Message headers not compliant with RFC.
• 550 Date header far in the past/future
• 554 Message is not RFC compliant; missing “Date” header
• 550 Senders clock is broken – fix and resend

Even if the message does get accepted, popular anti-spam systems like SpamAssassin look for these mistakes and will penalize you for the poor formatting. Below is the anti-spam report of a message sent through SpamAssassin with only the bare minimum fields to be considered an email message.

Non-specific RFC-based rejections

While some error codes and responses like those listed above help senders narrow down the issue with their message formatting, not all mail receivers are so nice. Many times the error response issued by a recipient’s mail server is generic. The following error response codes may be encountered any time a message does not meet standard RFC specifications:

• 554 Malformed mail denied!
• 550 5.7.1 empty subject header
• 550 Headers unacceptable – rejected
• 550 5.7.1 This message does not comply with required standards.
• 554 Message is not RFC compliant
• 521 5.2.1 : AOL will not accept delivery of this message.
• 550-5.7.1 [IP ADDRESS] Our system has detected that this message is not RFC 2822 compliant. To reduce the amount of spam sent to Gmail, this message has been blocked. Please review RFC 2822 specifications for more information.
• 550 5.7.0 (XXXX-XXX-XXX) Message could not be delivered. Please ensure the message is RFC 5322 compliant.
• 554 Message not allowed

If you encounter any of these error codes in the failed messages report of your SocketLabs server, please contact our support department for more information and assistance in troubleshooting and diagnosing the root cause.

Other Considerations

An often overlooked limitation that is established in the RFCs defining email messages is the 64 character limit of the local portion of an email address. This means you can only have 64 characters before the “@” of an address. This limit applies to addresses in any message field including the from address, reply-to, and return path.

The two most common occurrences of the local address limitation issue both occur with addresses that are generated by code. If you are establishing a reply-to address with unique message identifiers in the local portion, be sure that the maximum number of characters is less than 64. SocketLabs customers may also encounter this issue when utilizing our optional mailingID and messageID parameters. These fields become encoded into the local portion of the email address used in the return path. We recommend keeping the total character count of these values as low as possible.  We have found that exceeding 30 combined characters leads to increased message failures. Here are some sample failures that could be encountered due to addresses exceeding 64 characters in the local address portion:

• 501 Syntax error in parameters or arguments.
• 501 Invalid Address
• 501 Syntax error invalid forward path
• 501 local part too long near {address}
• 501 Syntax error – Badly formatted address

For more information about message formatting or any error codes contained in the article please contact the SocketLabs Support Department.

If you are not currently a SocketLabs customer but are interested in how we can help get your messages delivered, book a free 30-minute consultation with one of our email experts today!