Reduce unnecessary nested includes in your SPF record, to improve DNS efficiency
If you run a service which might be responsible for sending mails on behalf of a customer, and consequently have an SPF record they need to "include:" in their own, I think that you should probably review it and see if you have an excessive number of DNS lookups in your SPF record.
The problem is that if a customer of more than one of these mail service providers, and they have multiple include elements in their SPF record, it’s all too easy to breach the 10 DNS lookup limit, which could lead to random email loss (recipient MTAs giving up on DNS lookups and bouncing/rejecting legitimate emails).
For instance, include:spf.protection.outlook.com resolves to the following: -
"v=spf1 ip4:126.96.36.199/26 ip4:188.8.131.52/24 ip4:184.108.40.206/24 ip4:220.127.116.11/24 ip4:18.104.22.168/25 ip4:22.214.171.124/23 ip4:126.96.36.199/24 ip4:188.8.131.52/24 ip4:184.108.40.206/24 include:spfa.protection.outlook.com -all"
which leads to include:spfa.protection.outlook.com: -
"v=spf1 ip4:220.127.116.11/25 ip4:18.104.22.168/25 ip4:22.214.171.124/24 ip4:126.96.36.199/26 ip4:188.8.131.52/23 ip4:184.108.40.206/26 ip4:220.127.116.11/26 ip4:18.104.22.168/14 ip4:22.214.171.124/24 include:spfb.protection.outlook.com -all"
which in turn, leads to include:spfb.protection.outlook.com: -
"v=spf1 ip6:2a01:111:f400::/48 ip4:126.96.36.199/19 ip4:188.8.131.52/23 ip4:184.108.40.206/24 ip4:220.127.116.11/17 ip4:18.104.22.168/21 ip4:22.214.171.124/21 ip4:126.96.36.199/24 ip4:188.8.131.52/23 ip4:184.108.40.206/16 ip4:220.127.116.11/26 -all"
Now, most of the SPF include records I’ve seen, are perfectly able to live in a single, long DNS record - longer than 255 characters - simply by separating them with '" " ' (an end quote, a space, a start quote and a space) - these breaks are not seen in the final record - See the Internet Systems Consortium Knowledge Base article, Can I have a TXT or SPF record longer than 255 characters? (https://kb.isc.org/article/AA-00356/0/Can-I-have-a-TXT-or-SPF-record-longer-than-255-characters.html)
You can easily check the number of DNS lookups an SPF record requires, using dmarcian - SPF Surveyor (https://dmarcian.com/spf-survey/spf.protection.outlook.com).
In your case, there are too many IPs to contain them all in a single DNS Resource Record (and keep it within the 512 byte DNS UDP response size limit), but you can flatten/minimise the records like so (the first record is 2x <255 character sections & if/when the 2nd record grows >255 characters, that can be similarly split): -
spf.protection.outlook.com IN TXT "v=spf1 ip4:18.104.22.168/26 ip4:22.214.171.124/24 ip4:126.96.36.199/24 ip4:188.8.131.52/24 ip4:184.108.40.206/25 ip4:220.127.116.11/23 ip4:18.104.22.168/24 ip4:22.214.171.124/24 ip4:126.96.36.199/24 ip4:188.8.131.52/25" " ip4:184.108.40.206/25 ip4:220.127.116.11/24 ip4:18.104.22.168/26 ip4:22.214.171.124/23 ip4:126.96.36.199/26 ip4:188.8.131.52/26 ip4:184.108.40.206/14 ip4:220.127.116.11/24 ip4:18.104.22.168/19 ip4:22.214.171.124/23 ip4:126.96.36.199/24 include:spfa.protection.outlook.com -all"
spfa.protection.outlook.com IN TXT "v=spf1 ip4:188.8.131.52/17 ip4:184.108.40.206/21 ip4:220.127.116.11/21 ip4:18.104.22.168/24 ip4:22.214.171.124/23 ip4:126.96.36.199/16 ip4:188.8.131.52/26 ip6:2a01:111:f400::/48 -all"
I know I am but a small fish, in Microsoft’s big pond, but I firmly believe that this small improvement in efficiency (1 fewer DNS lookup), as well as benefiting my company (as our SPF record is overcrowded), it should have a positive effect on the number of DNS queries Microsoft’s DNS service will have to perform (~1/3 fewer).
Now reduced to one, please check.
Shawn Kammerdiener commented
This is still a huge issue, we are maxed at DNS retrievals for our own company as we have several vendors doing this, and MS uses 3/10 lookups for O365 email SPF hits. This should definitely be flattened, and also the IP range should be narrowed to ONLY those that are under control of MS for O365. Don't include IP ranges in Azure or other MS IP space that anyone with an account can utilize, as that defeats the purpose of SPF.
Brian Hampson commented
Three years on and we're STILL burning DNS lookups :(
C Colley commented
Yeah, I just dug into my SPF record to figure out why some email headers indicated a problem and found that Micro$ was burning 3 DNS lookups. This subsequently required me to rework my SPF record to try and cobble together a nested SPF setup.
To those of you in this situation, you could create another spf TXT record like spf1.yourdomain.com which includes all of Micro$ IPs and add spf1.yourdomain.com as an include: in your actual spf and potentially knock out the need to query Micro$. Problem with that is, if they add and IP (seriously doubt they'll retire one) you won't have that in your record.
In that case, you could do this for a less critical include: you have in your SPF - I did this for smtp.shopify since they burn through 3 lookups as well.
Wesley Kirkland commented
I agree this should be consolidated, while many libraries will allow for more than 10. The standard is 10 or less, and string concatenation would be better.
Antal Delahaije commented
I fully agree on reducing SPF records to reduce DNS lookups as they are limited to 10 according to RFC7208.
The include:spf.protection.outlook.com parameters needs 3 DNS lookups which could easily be reduced to 1 if Microsoft Office 365 admins configure a:submission.protection.outlook.com and put al necessary IP address in this domain like a round-robin configuration.
This also reduces DNS traffic for all parties.
- Solution for Microsoft -
Salesforce solved the exact problem (they used to burn up 4 lookups) by using a macro see here:
I am amazed that:
A/ this only has 28 votes...
B/ For something so trivial to get right, Microsoft still hasn't responded with "yep, sorry about that, ill fix that while I wait 2 minutes for my coffee" - because it is literally that easy to solve.