Like most of you who decided to read this post, we use DomainKeys Identified Mail (DKIM) in an attempt to keep email spoofing down. . We noticed that some of the emails we were receiving were marked as spam by spamassassin when they shouldn't. Looking at the email header we saw they were failing the DKIM body integrity test. The entries in the header file looks something like
Authentication-Results mail.example.com (amavisd-new); dkim=softfail (fail, body has been altered) firstname.lastname@example.org
dkim=neutral (body hash did not verify). dkim=fail (message has been altered) domainkeys=pass
That would increase the score of those legitimate emails just above the threshold, causing spamassassin to flag them as spam. After some investigating, we found out emails that contained characters above the original ASCII character set, such as Olivenöl, in the body of the message were failing. In other words, DKIM really does not like 8bit (specifically 8BITMIME) encoding.
I was able to duplicate that by sending an email to myself or to a gmail test account using our default mail client, Thunderbird. So I tried to manually (using netcat; telnet would have worked fine) create a properly configured 8Bit email using our SMTP server (As you can clearly see, our MX server (postfix) is setup to handle 8BITMIME):
raub@desktop:/tmp$ nc -t mail.example.com 25 220 mail.example.com Test Mail Server EHLO desktop.example.com 250-mail.example.com 250-PIPELINING 250-SIZE 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN MAIL FROM:<email@example.com> BODY=8BITMIME 250 2.1.0 Ok RCPT TO:<firstname.lastname@example.org> 250 2.1.5 Ok DATA 354 End data with <CR><LF>.<CR><LF> From: "Mauricio Tavares"
Subject: 8bit test Date: Mon, 30 Jun 2014 11:10:05 -0400 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Olivenöl . 250 2.0.0 Ok: queued as 2BA1F80041 QUIT 221 2.0.0 Bye raub@desktop:/tmp$
The above would work while using Thunderbird didn't. After setting Thunderbird to debug mode, we noticed it was not properly indicating the email was 8BITMIME, so postfix would treat it as a 7bit ASCII email and then DKIM would fail. The DKIM signatures of 8BITMIME mail may break unless all SMTP servers in the path implement and announce 8BITMIME support, and as shown above the mail clients do not lie. Since we cannot guarantee that is the case, it is better to down-convert to quoted-printable before DKIM signing.
Solution is to force the MTA to downconvert to 7bit. That can be done in postfix by creating a null filter in master.cf that is called before the rest of the filters. In other words, something like
smtp inet n - n - - smtpd -o content_filter=smtp-downconvert:127.0.0.1:10027 [...] smtp-downconvert unix - - - - 2 smtp -o smtp_discard_ehlo_keywords=8bitmime,silent-discard 127.0.0.1:10027 inet n - n - - smtpd -o smtpd_authorized_xforward_hosts=127.0.0.1 -o smtpd_client_restrictions= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_relay_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject
should do the trick. If we restart the MX (postfix reload will do the trick) and send the test email again, the email now should show something like
Authentication-Results mail.example.com (amavisd-new); dkim=pass email@example.com
in the header, which means DCKIM is successful and life is good... for now
I would like to thank Wietse Venema for pointing me on the right direction regarding downconverting.
Note: HELO vs EHLO
- HELO: Standard/Original SMTP (7-bit ASCII email) (RFC 821, 5321)
- EHLO: ESMTP (RFC 1869). Think Enhanced HELO; supports MIME and other exciting extensions. A quick discussion is found at http://cr.yp.to/smtp/ehlo.html.