lynxthecat/adblock-lean

Lean and powerful adblocking solution for OpenWrt

⚔ adblock-lean

If you like adblock-lean and can benefit from it, then please leave a ⭐ (top right) and become a stargazer! And feel free to post any feedback on the official OpenWrt thread here. Thank you for your support.

adblock-lean is a super simple and lightweight adblocking solution that leverages the major rewrite of the DNS server and domain handling code associated with dnsmasq 2.86 that drastically improves performance and reduces memory foot-print, facilitating the use of very large blocklists for even older, low performance devices.

adblock-lean was originally designed primarily for use with the dnsmasq variants of the popular hagezi and oisd blocklists used by major adblockers and which are intended to block ads without interfering with normal use.

adblock-lean is written as a service and 'service adblock-lean start' will download and setup dnsmasq with a new blocklist file. Various checks are performed and, in dependence upon the outcome of those checks, the script will either: accept the new blocklist file; fallback to a previous blocklist file if available; or restart dnsmasq with no blocklist file.

adblock-lean includes, inter alia, the following features:

• support for local blocklist and one or more blocklists to be downloaded from urls
• suport for local allowlist
• check individual blocklist file parts and total blocklist size do not exceeed configurable maximum file sizes
• generate blocklist file from local blocklist and allowlist and the one or more downloaded blocklist file part(s)
• check for rogue entries in blocklist file parts (e.g. check for redirection to specific IP)
• check good lines in blocklist file exceeds configurable minimum (default: 100,000)
• set up dnsmasq with new blocklist file and save any previous blocklist file as compressed file
• supports blocklist compression by leveraging the new conf-script functionality of dnsmasq
• perform checks on restarted dnsmasq with new blocklist file
• revert to previous blocklist file if checks fail
• if checks on previous blocklist file also fail then revert to not using any blocklist file
• user-configurable calls on success or failure
• automatically check for any updates and self update functionality

Installation on OpenWrt

wget https://raw.githubusercontent.com/lynxthecat/adblock-lean/main/adblock-lean -O /etc/init.d/adblock-lean
chmod +x /etc/init.d/adblock-lean
service adblock-lean gen_config # generates default config in /root/adblock-lean/config
vi /root/adblock-lean/config # modify default config as required
uci add_list dhcp.@dnsmasq[0].addnmount='/bin/busybox' && uci commit # to enable use of compressed blocklist
service adblock-lean enable

Config

adblock-lean reads in a config file from /root/adblock-lean/config.

A default config can be generated using: service adblock-lean gen_config.

Each configuration option is internally documented with comments in /root/adblock-lean/config.

Variable	Setting
blocklist_urls	One or more blocklist URLs to download and process
local_allowlist_path	Path to local allowlist (domain will not be blocked)
local_blocklist_path	Path to local blocklist (domain will be blocked)
min_blocklist_file_part_line_count	Minimum number of lines of individual downloaded blocklist part
max_blocklist_file_part_size_KB	Maximum size of any individual downloaded blocklist part
max_blocklist_file_size_KB	Maximim size of combined, processed blocklist
min_good_line_count	Minimum number of good lines in final postprocessed blocklist
compress_blocklist	Enable (1) or disable (0) blocklist compression once dnsmasq loaded
initial_dnsmasq_restart	Enable (1) or disable (0) initial dnsmasq restart to free up memory
rogue_element_action	Governs rogue element handling: 'SKIP_PARTIAL', 'STOP' or 'IGNORE'
download_failed_action	Governs failed download handling: 'SKIP_PARTIAL' or 'STOP'
report_failure	Used for performing user-defined action(s) on failure
report_successs	Used for performing user-defined action(s) on success
boot_start_delay_s	Start delay in seconds when service is started from system boot

For devices with low free memory, consider enabling the initial_dnsmasq_restart option to free up memory for use during the memory-intensive blocklist generation process by additionally restarting dnsmasq with no blocklist prior to the generation of the new blocklist. This option is disabled by default to prevent both the associated: dnsmasq downtime; and the temporary running of dnsmasq with no blocklist.

Selection of blocklist(s)

An important factor in selecting blocklist(s) is how much free memory is available for blocklist use. It is the responsibility of the user to ensure that there is sufficient free memory to prevent an out of memory situation.

Here are two examples for low and high memory devices.

Example blocklist selection for low memory devices:

blocklist_urls="https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/light.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/native.winoffice.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/native.apple.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/native.amazon.txt"

Example blocklist selection for high memory devices:

blocklist_urls="https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/pro.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/tif.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/native.winoffice.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/native.apple.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/native.amazon.txt"

An excellent breakdown of highly suitable lists and their merits is provided at:

https://github.com/hagezi/dns-blocklists

Selection of blocklist download and processing parameters

The parameters described in the config section above relating to the intermediate sizes, good line count and duplicate removal should be set in dependence on the selected blocklist and available memory. These are considered self-explanatory, but if in any doubt please post on the OpenWrt thread at:

https://forum.openwrt.org/t/adblock-lean-set-up-adblock-using-dnsmasq-blocklist/157076.

Automatically deploy blocklist on router reboot

Providing the service is enabled, the service script should automatically start on boot.

Automatically update blocklist at 5am following delay by random number of minutes

Set up the following Scheduled Task:

0 5 * * * /etc/init.d/adblock-lean enabled && export RANDOM_DELAY="1" && /etc/init.d/adblock-lean start

This tests whether the adblock-lean service is enabled and if so launches the start function, which updates to the new blocklist list.

The random delay serves to prevent a thundering herd: from an altruistic perspective, amelioerate load on the blocklist server; and from a selfish perspective, increase the prospect that the server is not loaded during the download.

User-configurable calls on success or failure

adblock-lean supports user-configurable calls on success or failure.

The following config paramters:

report_failure="" 	 
report_success=""

Are evaluated on success or failure, and the variables: ${success_msg} and ${failure_msg} can be employed in the calls.

Example below for Brevo (formerly sendinblue), but use your favourite smtp/email (or SMS) method.

• install mailsend package in OpenWRT
• sign up for free Brevo account (not affiliated!) - provides 300 free email sends per day
• edit /root/adblock-lean/config lines with Brevo specific user details (user variables in CAPITALS below): report_failure="mailsend -port 587 -smtp smtp-relay.sendinblue.com -auth -f [email protected] -t [email protected] -user [email protected] -pass BREVOPASSWORD -sub "$failure_msg" -M " "" report_success="mailsend -port 587 -smtp smtp-relay.sendinblue.com -auth -f [email protected] -t [email protected] -user [email protected] -pass BREVOPASSWORD -sub "$success_msg" -M " ""
• the Brevo password is supplied within their website, not the one created on sign-up.
• with each adblock-lean start call an email with a header such as "New blocklist installed with good line count: 248074." should be sent on success or a failure message sent on failure

Checking status of adblock-lean

The status of a running adblock-lean instance can be obtained by running:

service adblock-lean status

Example output:

root@OpenWrt-1:~# service adblock-lean status
Checking dnsmasq instance.
The dnsmasq check passed and the presently installed blocklist has good line count: 736225.
adblock-lean appears to be active.
Generating dnsmasq stats.
dnsmasq stats available for reading using 'logread'.
The locally installed adblock-lean is the latest version.

Keeping adblock-lean up-to-date

adblock-lean automatically checks for any version updates both at the end of the start and status routines.

adblock-lean can be updated to the latest version by simply running:

service adblock-lean update

Preserve service file and config across OpenWrt upgrades

Just add the files:

/root/adblock-lean
/etc/init.d/adblock-lean

to the list of files to backup in the Configuration tab in LuCi here:

http://openwrt.lan/cgi-bin/luci/admin/system/flash

🌠 Stargazers

adblock-lean's Issues

New blocklist file check failed.

What is file check failed and indicator of?

Started adblock-lean.
No local blocklist identified.
Downloading new blocklist file part(s).
Downloading new blocklist file part from: https://small.oisd.nl/dnsmasq2.
Download of new blocklist file part from: https://small.oisd.nl/dnsmasq2 suceeded (downloaded file size: 1012 KB).
Cleaning whitespace and formatting blocklist file part as local=/.../.
Successfully generated preprocessed blocklist file with 40183 line(s).
Processing and checking new blocklist file.
Preprocessed blocklist file size: 1012 KB.
Removing duplicates from blocklist file.
Duplicates removed.
No local allowlist identified.
Checking for any rogue elements.
Good line count: 40183 below 100000.
New blocklist file check failed.

Rogue Element identified in new blocklist

Hi noob question, may I know why I'm getting this error, what does rogue element means?

Rogue element: '447216: local=/antonio' identified in new blocklist.

By the way, I'm using hagezi blocklist. The rest of his blocklist are working but if I include the threat intelligence feed dnsmasq link provided, I got that error. Is this a compatibility issue?

Better way to wait for dnsmasq restart completion?

Right now we simply issue a sleep 10 call to wait for a dnsmasq restart before checking the new dnsmasq instance - see here:

adblock-lean/adblock-lean

Line 217 in 90f3d50

sleep 10

This is rather crude.

Surely there is a better way to wait until the dnsmasq restart has completed to avoid waiting unnecessarily long or, worse, not long enough?

@dave14305 and @Wizballs, any thoughts?

how to change Upstream DNS

Thanks for creating Adblock-lean. How to test; if it s working on lan side or not and I'm also wondering if there is way to change upstream encrypted or non encrypted DNS?

Not an issue just wanted to thank you

As the title says.

Just wanted to thank you here because I forgot the password https://forum.openwrt.org. LoL

It's been about 2 weeks using adblock-lean on MikroTik RBSXTsq2nD (SXTsq Lite2) without any problems.

It's a 16/64 device.
But for the first time I can use a "large" blocklist (Hagezi Multi NORMAL).
Note: even I tried Multi PRO (but people suggested Multi NORMAL).

Usually when I use the "large" blocklist on simple-adblock or now adblock-fast or adblock or dnscrypt-proxy2 I run out of RAM and adblock doesn't work anymore.

But not with adblock-lean!!!
I use it with wireguard to access public wifi without any problems.
I didn't even enable "initial_dnsmasq_restart".

And I usually need another router (actually an android tv box with openwrt) to run "ad blocker" with "large" blocklists.
Or a dns with blocklist.

Thank you this is really amazing.