trellmor/bind-adblock

Use the BIND DNS server to block ads

bind-adblock's Issues

Thanks.

utf-8 char broke script

List: https://hosts-file.net/ad_servers.txt have a utf-8 char which broke script with error:

Traceback (most recent call last):
  File "./update-zonefile.py", line 229, in <module>
    domains = parse_lists(args.origin)
  File "./update-zonefile.py", line 135, in parse_lists
    data = download_list(l['url'])
  File "./update-zonefile.py", line 115, in download_list
    return f.read()
  File "/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc9 in position 297767: ordinal not in range(128)

Python is not my stenght, but replacing:
with cache.open() as f: with with cache.open('r', 1, 'utf-8') as f:
fixed error for me

Can we have a comprehensive list of ads played by youtube (specifically youtube) ?

Add custom domains to blacklist

First things first: THANKS for this project! I refuse to install adblockers on my clients (have so many devices at home it quickly becomes a maintenance nightmare to keep everything properly configured and up to date), so blocking ads at the DNS level makes a lot of sense to me.

Anyway, I've noticed that you document how to create a whitelist. However, what I want to know is how to do the inverse: aside of blocking ads and trackers, I want to add extra domains to the blocklist - I want to block Facebook and friends, and currently I achieve that using a custom zone for redirecting the facebook.com zone to a bogus IP inside my LAN. Is there a way to achieve the same with this script?

RNDC fails to reload when using multiple zones

I have multiple zones to split my cloud network domain from my home domain since I need some internal networks to use a public IP and others to use an internal IP. because of this, I have to specify the rpz blocklist in each zone.

I get the following error when running update-zones.py

rndc: 'reload' failed: multiple
zone 'block.ads.rpzlist' was found in multiple views
Traceback (most recent call last):
  File "/root/bind-adblock/update-zonefile.py", line 288, in <module>
    reload_zone(args.origin)
  File "/root/bind-adblock/update-zonefile.py", line 215, in reload_zone
    raise Exception('rndc failed with return code {}'.format(r))
Exception: rndc failed with return code 1

The workaround is to simply use "systemctl restart named", or "rndc reload". I'm not a programmer unfortunately, however it appears that the code to reload the server needs to reload all zones.

Exclude domain list

Hello
I just saw i go problem with HBO HO at home... after research i find:
HBOGO application connact to: level3cdn-hbogo-eu.secure.footprint.net if connect failed he don`t working.... i try watch Game of Throne S7E5 and this is impossible with this...

Please add option to create exclude domain list.
Best Regards
TaKeN

Are the tracking hosts and ad hosts on a different list or are they combined?

I really am not that concerned with blocking tracking hosts, and it's become an annoyance because so many links will not work since they're facilitated through some kind of tracking service. I figure I could remove the disconnect.me tracking list, but what about Peter Lowe's list? That seems to imply that it contains both tracking and ad hosts.

I know I can just add certain hosts to the whitelist, but editing a whitelist and adding a host every time I can't follow a link seems like it will get annoyingly cumbersome. I'd prefer to just not block tracking hosts.

TypeError: object doesn't support attribute assignment

This might be an end user issue, but I'm not able to get the script to work. Details included below. I've swapped my actual domain name for "example.com" but I am not using "example.com" in any of my actual configuration.

root@server:/tmp/bind-adblock# pip3 list | grep requests
requests               2.22.0
root@server:/tmp/bind-adblock# pip3 list | grep dnspython
dnspython              2.0.0

root@server:/etc/bind# cat named.conf.options
options {
        directory "/var/cache/bind";
        auth-nxdomain no;
        // listen-on-v6 { any; };
        listen-on port 53 { localhost; 192.168.100.0/24; };
        allow-query { localhost; 192.168.100.0/24; };
        forwarders { 10.8.8.1; 192.168.1.1; };
        recursion yes;
        response-policy {
                zone "rpz.example.com";
        };
};

root@server:/etc/bind# cat named.conf.local
zone    "example.com"  {
        type master;
        file    "/etc/bind/forward.example.com";
};


zone    "100.168.192.in-addr.arpa"       {
        type master;
        file    "/etc/bind/reverse.example.com";
};


zone    "rpz.example.com"      {
        type master;
        file    "/etc/bind/rpz.example.com";
        masterfile-format text;
        allow-query { none; };
};

root@server:/etc/bind# cat rpz.example.com
@ 3600 IN SOA @ example.com. 0 86400 7200 2592000 86400
@ 3600 IN NS ns.example.com.

Actually running the script:

root@server:/tmp/bind-adblock# ./update-zonefile.py /etc/bind/rpz.example.com rpz.example.com
Traceback (most recent call last):
  File "./update-zonefile.py", line 263, in <module>
    update_serial(zone)
  File "./update-zonefile.py", line 224, in update_serial
    soa.serial += 1
  File "/usr/local/lib/python3.8/dist-packages/dns/rdata.py", line 129, in __setattr__
    raise TypeError("object doesn't support attribute assignment")
TypeError: object doesn't support attribute assignment

Unable to run in Alpine based container

I have created the following Dockerfile:

FROM alpine:3.17
RUN apk update
RUN apk add --no-cache bind python3 py3-certifi py3-charset-normalizer py3-decorator py3-dnspython py3-idna py3-pathlib2 py3-yaml py3-requests py3-urllib3 py3-validators git
USER named
WORKDIR /tmp
RUN git clone https://github.com/Trellmor/bind-adblock.git
WORKDIR bind-adblock
RUN ./update-zonefile.py --no-bind blocklist.zone blocklist.zone
HEALTHCHECK --interval=60s --retries=1 CMD ["nslookup", "google.com", "127.0.0.1"]
EXPOSE 53/udp
CMD ["named", "-c", "/etc/bind/named.conf", "-g"]

I get the following error:

 => ERROR [7/7] RUN ./update-zonefile.py --no-bind blocklist.zone blocklist.zone                                                                                                                               0.5s 
------                                                                                                                                                                                                              
 > [7/7] RUN ./update-zonefile.py --no-bind blocklist.zone blocklist.zone:                                                                                                                                          
#0 0.482 Traceback (most recent call last):
#0 0.482   File "/tmp/bind-adblock/./update-zonefile.py", line 283, in <module>
#0 0.482 Zone file "/tmp/bind-adblock/blocklist.zone" created.
#0 0.482 
#0 0.482 Add BIND options entry:
#0 0.482 response-policy {
#0 0.482     zone "blocklist.zone";
#0 0.482 };
#0 0.482 
#0 0.482 Add BIND zone entry:
#0 0.482 zone "blocklist.zone" {
#0 0.482     type master;
#0 0.482     file "/tmp/bind-adblock/blocklist.zone";
#0 0.482     masterfile-format text;
#0 0.482     allow-query { none; };
#0 0.482 };
#0 0.482 
#0 0.482     zone = load_zone(args.zonefile, args.origin, args.raw_zone)
#0 0.482   File "/tmp/bind-adblock/./update-zonefile.py", line 208, in load_zone
#0 0.483     return dns.zone.from_text(zone_text, origin)
#0 0.483   File "dns/zone.py", line 1111, in dns.zone.from_text
#0 0.483   File "dns/zone.py", line 1116, in dns.zone.from_text
#0 0.483   File "dns/zonefile.py", line 451, in dns.zonefile.Reader.read
#0 0.483   File "dns/zonefile.py", line 214, in dns.zonefile.Reader._rr_line
#0 0.483   File "dns/transaction.py", line 145, in dns.transaction.Transaction.add
#0 0.483   File "dns/transaction.py", line 407, in dns.transaction.Transaction._add
#0 0.483   File "dns/transaction.py", line 489, in dns.transaction.Transaction._checked_put_rdataset
#0 0.483   File "dns/zonefile.py", line 47, in dns.zonefile._check_cname_and_other_data
#0 0.483   File "dns/transaction.py", line 126, in dns.transaction.Transaction.get_node
#0 0.483   File "dns/transaction.py", line 90, in dns.transaction._ensure_immutable_node
#0 0.483   File "dns/_immutable_ctx.py", line 41, in dns._immutable_ctx._immutable_init.nf
#0 0.483   File "dns/_immutable_ctx.py", line 64, in dns._immutable_ctx.immutable.ncls.__init__
#0 0.483   File "dns/node.py", line 296, in dns.node.ImmutableNode.__init__
#0 0.483   File "dns/node.py", line 295, in dns.node.ImmutableNode.__init__
#0 0.483 TypeError: __init__() takes exactly 2 positional arguments (1 given)
------
failed to solve: process "/bin/sh -c ./update-zonefile.py --no-bind blocklist.zone blocklist.zone" did not complete successfully: exit code: 1

I've tried several methods, such as binding the blocklist file instead of using one in tmp, making it run as root instead of the named user, and more. I can't get it to create a fresh file on container creation, I think i will configure the container to have the update portion as a cron job, but I first need to get it to even generate the zone file.

This does not install correctly on various ubuntu versions

Was hoping to get this working, but its failed on ubuntu 14/16

pip install -r requirements.txt

Collecting certifi==2020.6.20 (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl (156kB)
100% |████████████████████████████████| 163kB 4.2MB/s
Collecting chardet==3.0.4 (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
100% |████████████████████████████████| 143kB 6.0MB/s
Collecting dnspython==2.0.0 (from -r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/67/d0/639a9b5273103a18c5c68a7a9fc02b01cffa3403e72d553acec444f85d5b/dnspython-2.0.0.zip (324kB)
100% |████████████████████████████████| 327kB 2.8MB/s
Collecting idna==2.10 (from -r requirements.txt (line 4))
Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
100% |████████████████████████████████| 61kB 7.8MB/s
Collecting pathlib==1.0.1 (from -r requirements.txt (line 5))
Downloading https://files.pythonhosted.org/packages/ac/aa/9b065a76b9af472437a0059f77e8f962fe350438b927cb80184c32f075eb/pathlib-1.0.1.tar.gz (49kB)
100% |████████████████████████████████| 51kB 7.8MB/s
Collecting PyYAML==5.3.1 (from -r requirements.txt (line 6))
Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz (269kB)
100% |████████████████████████████████| 276kB 3.2MB/s
Collecting requests==2.24.0 (from -r requirements.txt (line 7))
Downloading https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl (61kB)
100% |████████████████████████████████| 71kB 8.0MB/s
Collecting urllib3==1.25.10 (from -r requirements.txt (line 8))
Downloading https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl (127kB)
100% |████████████████████████████████| 133kB 6.2MB/s
Building wheels for collected packages: dnspython, pathlib, PyYAML
Running setup.py bdist_wheel for dnspython ... done
Stored in directory: /root/.cache/pip/wheels/33/c7/d2/367b914384b2ca7372006799bd16b52e307683bea1642da890
Running setup.py bdist_wheel for pathlib ... done
Stored in directory: /root/.cache/pip/wheels/f9/b2/4a/68efdfe5093638a9918bd1bb734af625526e849487200aa171
Running setup.py bdist_wheel for PyYAML ... done
Stored in directory: /root/.cache/pip/wheels/a7/c1/ea/cf5bd31012e735dc1dfea3131a2d5eae7978b251083d6247bd
Successfully built dnspython pathlib PyYAML
Installing collected packages: certifi, chardet, dnspython, idna, pathlib, PyYAML, urllib3, requests
Successfully installed PyYAML-5.3.1 certifi-2020.6.20 chardet-3.0.4 dnspython-2.0.0 idna-2.10 pathlib-1.0.1 requests-2.24.0 urllib3-1.25.10
You are using pip version 8.1.1, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@dns7:/bind-adblock# ./update-zonefile.py --help
Traceback (most recent call last):
File "./update-zonefile.py", line 33, in
import dns.zone
ImportError: No module named 'dns'
root@dns7:/bind-adblock#

THANK YOU (youtube ads?)

Great tool! I installed it yesterday and life at home is already much better.
I still get Youtube ads however. Is there a feed to block youtube ad servers?
Many thanks, -t

ImportError: No module named 'dns'

i have this problem on banana pi ubuntu 16.04
Traceback (most recent call last): File "./update-zonefile.py", line 33, in <module> import dns.zone ImportError: No module named 'dns'

Bind zones can handle wildcard

Unlike host file, bind can handle wildcard so why don't we optimize the zone file for domains?

For example

$ cat adblock.rpz | wc -l
134973 (total lines)

$ grep .302br.net adblock.rpz | wc -l
17943
$ grep .hpg.com.br adblock.rpz | wc -l
878

Replacing these entries with followings will make things smaller, faster & much efficient.

*.302br.net IN CNAME .
*.hpg.com.br IN CNAME .

issue with: FROM python:latest

>  => ERROR [adblock 8/9] RUN pip install --upgrade pip &&     pip install -r requirements.txt                                                                                                               8.6s
> ------
>  > [adblock 8/9] RUN pip install --upgrade pip &&     pip install -r requirements.txt:
> 1.823 Requirement already satisfied: pip in /opt/venv/lib/python3.12/site-packages (24.0)
> 2.919 Collecting certifi==2022.12.7 (from -r requirements.txt (line 1))
> 3.036   Downloading certifi-2022.12.7-py3-none-any.whl.metadata (2.9 kB)
> 3.277 Collecting charset-normalizer==3.0.1 (from -r requirements.txt (line 2))
> 3.295   Downloading charset_normalizer-3.0.1-py3-none-any.whl.metadata (27 kB)
> 3.439 Collecting decorator==5.1.1 (from -r requirements.txt (line 3))
> 3.452   Downloading decorator-5.1.1-py3-none-any.whl.metadata (4.0 kB)
> 3.574 Collecting dnspython==2.3.0 (from -r requirements.txt (line 4))
> 3.589   Downloading dnspython-2.3.0-py3-none-any.whl.metadata (5.2 kB)
> 3.721 Collecting idna==3.4 (from -r requirements.txt (line 5))
> 3.739   Downloading idna-3.4-py3-none-any.whl.metadata (9.8 kB)
> 3.981 Collecting pathlib==1.0.1 (from -r requirements.txt (line 6))
> 4.000   Downloading pathlib-1.0.1-py3-none-any.whl.metadata (5.1 kB)
> 4.168 Collecting PyYAML==6.0 (from -r requirements.txt (line 7))
> 4.190   Downloading PyYAML-6.0.tar.gz (124 kB)
> 4.290      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.0/125.0 kB 1.2 MB/s eta 0:00:00
> 4.384   Installing build dependencies: started
> 8.081   Installing build dependencies: finished with status 'done'
> 8.082   Getting requirements to build wheel: started
> 8.299   Getting requirements to build wheel: finished with status 'error'
> 8.305   error: subprocess-exited-with-error
> 8.305   
> 8.305   × Getting requirements to build wheel did not run successfully.
> 8.305   │ exit code: 1
> 8.305   ╰─> [54 lines of output]
> 8.305       running egg_info
> 8.305       writing lib/PyYAML.egg-info/PKG-INFO
> 8.305       writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
> 8.305       writing top-level names to lib/PyYAML.egg-info/top_level.txt
> 8.305       Traceback (most recent call last):
> 8.305         File "/opt/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
> 8.305           main()
> 8.305         File "/opt/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
> 8.305           json_out['return_val'] = hook(**hook_input['kwargs'])
> 8.305                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 8.305         File "/opt/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
> 8.305           return hook(config_settings)
> 8.305                  ^^^^^^^^^^^^^^^^^^^^^
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
> 8.305           return self._get_build_requires(config_settings, requirements=['wheel'])
> 8.305                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
> 8.305           self.run_setup()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
> 8.305           exec(code, locals())
> 8.305         File "<string>", line 288, in <module>
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
> 8.305           return distutils.core.setup(**attrs)
> 8.305                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
> 8.305           return run_commands(dist)
> 8.305                  ^^^^^^^^^^^^^^^^^^
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
> 8.305           dist.run_commands()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
> 8.305           self.run_command(cmd)
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
> 8.305           super().run_command(command)
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
> 8.305           cmd_obj.run()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
> 8.305           self.find_sources()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
> 8.305           mm.run()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run
> 8.305           self.add_defaults()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults
> 8.305           sdist.add_defaults(self)
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults
> 8.305           super().add_defaults()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
> 8.305           self._add_defaults_ext()
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
> 8.305           self.filelist.extend(build_ext.get_source_files())
> 8.305                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 8.305         File "<string>", line 204, in get_source_files
> 8.305         File "/tmp/pip-build-env-wa0przg1/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
> 8.305           raise AttributeError(attr)
> 8.305       AttributeError: cython_sources
> 8.305       [end of output]
> 8.305   
> 8.305   note: This error originates from a subprocess, and is likely not a problem with pip.
> 8.307 error: subprocess-exited-with-error
> 8.307 
> 8.307 × Getting requirements to build wheel did not run successfully.
> 8.307 │ exit code: 1
> 8.307 ╰─> See above for output.
> 8.307 
> 8.307 note: This error originates from a subprocess, and is likely not a problem with pip.
> ------
> failed to solve: process "/bin/sh -c pip install --upgrade pip &&     pip install -r requirements.txt" did not complete successfully: exit code: 1

had to change to FROM python: 3.11

[Request] Add Internet Storm Control 'Suspicious Domains' list

Hey,

I happened to stumble across the following list:

https://isc.sans.edu/feeds/suspiciousdomains_Low.txt

(See https://isc.sans.edu/suspicious_domains.html for more info).

Anyway when I added it to the script it pulled in an additional 3138 domains to my block (out of 3490 on the ISC list.) So it seems to have a lot of unique entries not already covered.

I'm not sure how good it is as a list, I've had no trouble so far and they seem like a legit outfit. Just a suggestion anyway. BTW thanks for the project it is really useful and well put together !

Cathal.

Optional check_zone & reload_zone

Greetings,

I am currently running bind-adblock inside a minimal Docker container. This container does not have access to the bind binaries. I would be nice to have options to skip both the check_zone and reload_zone steps. This way I can just receive a completed zone and overwrite my existing one.

Thanks!

someonewhocares.org changed URL

https://www.someonewhocares.org/hosts/zero/

Hello
someonewhocares.org has recently being timing out for he. I changed the URL in your existing code to the URL listed above and got it working again. Please verify and change if you agree to my finding.
Thank you for providing this great script to the public.
With kind regards
Thorstne

"out of memory"

I haven't gone into details, but if using all sources the host name lengths of some of them cause the zone not to load with "out of memory" message. I just excluded everything that was > 50 char line length and all is well. I'll do some additional testing to see what I find as the tip point. bind 9.10.3 (ubuntu).

Typo in whitelist example

https://github.com/Trellmor/bind-adblock/wiki/whitelist

rpz-whitelist.exam_p_le .com misses the "p" ;-)

// For AdBlock
response-policy {
zone "rpz-whitelist.examle.com" policy passthru;
zone "rpz.example.com";
};

Allow option to return NULL (0.0.0.0) rather than NXDOMAIN

According to PiHole experiments when clients query return NXDOMAIN they don't immediately give up resolving the domain name. I'm unsure how complicated would be implementing this option.

dnspython version 2.0.0 breaks script

Hello,

It appears changes are needed to support dnspython 2.0.0. It's probably related to Rdata is now immutable. Use dns.rdata.Rdata.replace() to make a new Rdata based on an existing one.1

Error:

  File "/opt/bin/update-zonefile.py", line 263, in <module>
    update_serial(zone)
  File "/opt/bin/update-zonefile.py", line 224, in update_serial
    soa.serial += 1
  File "/usr/lib/python3.7/site-packages/dns/rdata.py", line 129, in __setattr__
    raise TypeError("object doesn't support attribute assignment")
TypeError: object doesn't support attribute assignment