Wednesday, 12 December 2012

DMARC and SPF updates

A while back I added a DMARC entry in DNS for my @gardiner.net.au domain. The existence of this entry then means I get daily email reports which include data from a number of email servers (eg. Google, Yahoo) about emails received from my domain and whether they were regarded as legitimate or spam. I don't receive copies of the emails themselves – just a summary of how many emails the destination site thought were legitimate and how many were rejected because they thought they were spam.

Microsoft have just announced that they too are now sending DMARC reports, so this prompted me to review my current SPF and DMARC settings to ensure that they're working properly.

The trouble with the DMARC reports are that they come via email with an attached zipped .XML file, which means you can't just view them... you have to download them, unzip them, then open it in IE (or Notepad), and scan through the XML to try and make sense of it. Wouldn't it be nice if there was a tool or service that summarised this for you?

Well it turns out there are some. I've decided try try two out - http://dmarcian.com and DMARC Analyzer.

Both of these services allow you to upload existing DMARC reports or set up email forwarding to automatically send the reports directly. You can then log in and view a summary.

I uploaded the data from the last 7 days. Here's some examples of the kind of report you get from each service:

dmarcian.com

Graph of DMARC results for last 7 days

The details for data from the 9th of December:

image

GARDINER.NET.AU - 3 msgs, 3 IPs
  • SPF-Authorized Servers - 2 groups , 2 msgs, 2 IPs, 100% auth'd
  • Other Servers - 1 group , 1 msg, 1 IP, 0% auth'd
  • 65.54.190.25 (bay0-omc1-s14.bay0.hotmail.com), 1 msg, 0% auth'd
  • 1 msg, disposition: None (monitor only) [none], DMARC-DKIM: fail (raw: none, d=none), DMARC-SPF: fail (raw: pass, dom: hotmail.com)
  • DMARC Analyzer

    Graph of DMARC results for last 7 days

    I was curious that both of them flagged a potential problem with an email. Sometimes this can be because it is actually spam – an email sent from an address that was not part of the authorised sender list as defined in the SPF record. But in this case, the error indicated that the email did come from a legitimate source.

    Next step to confirm that my SPF record is correct. A quick trip to the SPF Record Testing Tools confirmed that yes, my SPF record was in effect, but that there was also an error message I hadn't noticed previously:

    PermError SPF Permanent Error: Too many DNS lookups

    So it turns out that there are limits on how many DNS lookups are allowed for SPF records. 10 to be precise.

    My old SFP record was:

    v=spf1 a mx ip4:203.59.1.0/24 include:aspmx.googlemail.com include:hotmail.com include:gmail.com include:live.com -all

    It does looks like there's some redundancy there with two similar includes covering GMail and another two for Hotmail/Live. Simplifying things down (and hopefully not losing any accuracy) I've changed the record to this:

    v=spf1 a mx ip4:203.59.1.0/24 include:hotmail.com include:_spf.google.com ~all

    This now passes validation. Note that I've reverted back to ~all (a 'Soft' fail which means that recipients won't outright reject emails if there is a problem with the new rule). I'll switch back to -all (a 'hard' fail) after a week or two once I'm happy that nothing is broken!

    I'll also be interested to see if the DMARC reports contain passing results for the hotmail emails.

    5 comments:

    Tim Draegen said...

    Very cool! David, about the same day you wrote this, I updated dmarcian.com to include a "Dashboard" view of all of your domains, showing:

    - Domain
    - DMARC
    - SPF
    - DKIM
    - Volume

    Given this view, you would have had a red cell that said "SPF Error", and tooltip would have said "PermError SPF Permanent Error: Too many DNS lookups". I hope this functionality would have saved you time!

    The tools are still maturing, that's for sure. There's a lot more to do to make email authentication easy. Thanks for checking out dmarcian!

    Anonymous said...

    I had a problem when triesd to set DMARC to "reject" while using some systems like Gmail to send in behalf of me / other users.
    problem is posted here http://serverfault.com/questions/488689/how-to-avoid-messages-rejection-because-of-dmarc-when-sent-through-gmail-alias

    if any ideas will be thankful :)

    Dustin said...

    Nice stuff! Before I stumbled on dmarcian.com, I had already decided to implement my own report "aggregator" after discovering DMARC. The weeks prior, I rolled out TLS, SPF and DKIM on our domains (yeah, we were living in the dark ages prior to February 2013). DMARC would've been nice to help design my SPF records, live and learn though.

    Anyhow, my report analyzer is built on a VERY simple SQL back end with two tables. Calendar selector to filter on dates and pretty pie charts for some icing on the cake. Plan to add click event functions to further filter like dmarcian.com. I see that dmarcian is free for <100k emails/month. We're not too far below that threshold and it wouldn't take much for one of our marketing departments to get overzealous with a campaign one month and throw us over. That said, $99/month is nominal for the value dmarcian.com provides, and certainly if you're inclined to start from scratch... but I will offer that I have a much better appreciation for DMARC now after having written my own reporting front end and XML interpreter.

    David Gardiner said...

    Hi Dustin,

    thanks for your comments. 'Building your own' is certainly a great way to better understand something.

    I think it's also worth considering what's the core business you're in. Is it writing DMARC applications (maybe) or is your business something else entirely, in which case getting someone else to provide this service may make sense in the long term.

    -david

    Dustin said...

    David,

    You have without a doubt a valid point! I just noticed a typo in my original comment; I meant to say "certainly if you're *NOT* inclined".

    I added in click event handlers to filter and drill-down on data from my various charts this afternoon in an hour or two. It's been a great ASP experiment/learning experience for me and I'll definitely keep dmarcian in mind as we continue to delve into our DMARC reported data. It goes without saying that they've spent far more time than I'll ever have to perfect the art.

    It's also worth noting that dmarcian offers some really handy tools (with excellent documentation and great explanations) too!

    Dustin