cetanu/sender_policy_flattener

Compact large SPF chains into flat blocks of IP addresses

sender policy flattener

We had a problem in our organisation that caused our SPF records to become invalid:

When customers computers were querying our SPF records, there were more than 10 lookups required after following all of the include: remarks.

Solution? Query them ourselves, and create a much more condense list of SPF records.

But wait... What if the downstream records change?

Part of what the script does is that it creates a JSON file that keeps track of the last list of IP Addresses that your combination of SPF records had.

When the hashsum of your IP Addresses changes, it will send out an email (or just dump HTML if it can't find an email server) with a handy diff & BIND format for viewing what has changed, and promptly updating it.

You could theoretically extract the flat IP records from the resulting JSON file and automatically update your DNS configuration with it.

Installation

via git clone

Clone this repo and run

pip install poetry
poetry install

via pip

pip install sender_policy_flattener

Usage

usage: spflat [-h] [-c CONFIG] [-r RESOLVERS] [-e MAILSERVER] [-t TOADDR]
              [-f FROMADDR] [-s SUBJECT] [-D SENDING_DOMAIN] [-d DOMAINS]
              [-o OUTPUT]

A script that crawls and compacts SPF records into IP networks. This helps to
avoid exceeding the DNS lookup limit of the Sender Policy Framework (SPF)
https://tools.ietf.org/html/rfc7208#section-4.6.4

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Name/path of JSON configuration file
  -r RESOLVERS, --resolvers RESOLVERS
                        Comma separated DNS servers to be used
  -e MAILSERVER, -mailserver MAILSERVER
                        Server to use for mailing alerts
  -t TOADDR, -to TOADDR
                        Recipient address for email alert
  -f FROMADDR, -from FROMADDR
                        Sending address for email alert
  -s SUBJECT, -subject SUBJECT
                        Subject string, must contain {zone}
  -D SENDING_DOMAIN, --sending-domain SENDING_DOMAIN
                        The domain which emails are being sent from
  -d DOMAINS, --domains DOMAINS
                        Comma separated domain:rrtype to flatten to IP
                        addresses. Imagine these are your SPF include
                        statements.
  -o OUTPUT, --output OUTPUT
                        Name/path of output file

Example

spflat --resolvers 8.8.8.8,8.8.4.4 \
    --to [email protected] \
    --from [email protected] \
    --subject 'SPF for {zone} has changed!' \
    --domains gmail.com:txt,sendgrid.com:txt,yahoo.com:a \
    --sending-domain mydomain.com

or

spflat --config spf.json

You can specify a config file, or you can specify all of the optional arguments from the command line.

I've provided a settings.json file with an example configuration file.

Supported Python versions

See the latest result of the build: https://github.com/cetanu/sender_policy_flattener/actions

3rd party dependencies

• netaddr
• dnspython

Example email format

sender_policy_flattener's Issues

Not resolving A records

When I have A records in my config.json, their IPs don't get added to the sums.json. To further test I removed everything except "example.com": "A" and it generates what is basically an empty SPF:

{
    "< my domain >": {
        "records": [
            "v=spf1  -all"
        ],
        "sum": "4e15a0e95214c62159ddf7ffd277b47a85fe7f9aaf8c893924a0c8d69c3e8dea"
    }
}

Getting an error when running the script.

When running:
spflat --resolvers 8.8.8.8 -to [email protected] -from [email protected] -subject 'SPF Flat' --domains _spf.salesforce.com:txt --sending-domain cp42ghtcyvfm.b-dgmleaq.um1.bnc.salesforce.com

I get:
usage: spflat [-h] [-c CONFIG] [-r RESOLVERS] [-e MAILSERVER] [-t TOADDR] [-f FROMADDR] [-s SUBJECT] [-D SENDING_DOMAIN] [-d DOMAINS] [-o OUTPUT]
spflat: error: unrecognized arguments: Flat'

Also i cannot find the config file anywhere??

SPF record length cap

Hiya, been using this application for our business. Works great, we found that some vendors experienced problems with SPF records exceeding the character max of 255, which most entries generated by spflat appear to do.
spflat should probably cap generated record lengths at 255 chars.

Confuse with configuration file

I am confused with your configuration file:
here is your example:

{
    "sending domains": {
        "mydomain.com": {
            "yahoo.com": "a",
            "google.com": "txt",
            "apple.com": "txt",
            "reddit.com": "txt"
        },
        "myseconddomain.com": {
            "cisco.com": "a",
            "oculus.com": "a"
        }
    },
    "resolvers": [
        "8.8.8.8",
        "8.8.4.4"
    ],
    "email": {
        "to": "[email protected]",
        "from": "[email protected]",
        "subject": "[Change Detected] SPF Records for {zone} have changed.",
        "server": "email_server"
    },
    "output": "sums.json"
}

Here is my configuration without problem

{
    "sending domains": {
        "mydomain.com": {
            "mydomain.com": "txt"
        }
    },
    "resolvers": [
        "8.8.8.8"
    ],
    "email": {
        "to": "[email protected]",
        "from": "[email protected]",
        "subject": "[Change Detected] SPF Records for {zone} have changed.",
        "server": "localhost"
    },
    "output": "sums.json"
}

In my understanding,
"sending domains" is target domain to monitor.
But, what is the function of "myseconddomain.com"?

Report which include an address changed for in report

Proposed enhancement: When the report is emailed, it shows which addresses are different from the last run, but does not specify which include the changes came from. This could be useful for seeing which SPF includes are updating frequently.

Concern about SPF of remind mail

Hi. Thank you very for your sharing about the solution for SPF flattener.
I have a concern about the notification mail:
If the From address is [email protected] and I apply this service in a server (IP:1.2.3.4).
For sending the notification successfully, I have to add IP:1.2.3.4 to SPF record of 'test.com'. Am I right?

CRLF in the shebang

$ spflat --help
/usr/bin/env: ‘python\r’: No such file or directory

It's because the file is using Windows CRLF instead of plain newlines.
dos2unix .local/bin/spflat fixed this but pypi should be updated.

too much ''.join() line 33

at first look line 33 https://github.com/cetanu/sender_policy_flattener/blob/master/sender_policy_flattener.py#L33

''.join(sorted(''.join(sorted(iterable)))

should be replaced by

''.join(sorted(iterable))

It's because the original line transform

['bda','ace']

into

'aabcde'

it is unlikely it is the intent because it mix all the characters.

an even better better alternative would be to replace the whole by

json.dumps(sorted(iterable))

as it guarantee that ['abc','def'] lead to different result than ['a','bcdef']

In both cases as the hashed values would be different than the original one, the hash of settings['output']files should be recalculated

Actually, unless the hash is used somewhere else, the whole hash calculation could be all wiped out if line 256 (https://github.com/cetanu/sender_policy_flattener/blob/master/sender_policy_flattener.py#L256 )

if prev_spf[zone]['sum'] != curr_spf[zone]['sum']:

is replaced by

if sorted(prev_spf[zone]['record']) != sorted(curr_spf[zone]['record']):