slyfox1186/pihole-regex

Custom RegEx, Exact, and Adlist filters for Pi-hole's FTLDNS

pihole-regex's Issues

Error in documentation on the main page

Please note that the commands in section "Add/Update the RegEx filters to Pi-hole" are in reversed order. The first command shouldn't install the filters, but uninstall them. Uninstall the filters first, then install them.

Blocks google.com and www.google.com

This block regex blocks google landing for search:
^[\w-.]*?google(?:adservices|apis|syndication|usercontent)?.com$

Is there a step I am missing besides installing and adding whitelist?
Also... what URLs are these intending to block. I would like to help but don't want to break the intent this block was used for.

Blacklist domain pass through as Whitelist

Some whitelist regex might need rethinking, given regex “as is” are
letting blacklist domain pass through as whitelist.

Cheers

Amazon # Close to 1800 blacklist domains check attach.

(.|^)(amzn|amazonaws|amazontrust|apzones|digicert|marketwatch|mzstatic|sym[b-d]+?|wp|yimg|youtube|ytimg).(com|to)$

amazon.txt

AdultTime Necessary for the site to deliver non ad content

(.|^)(dns|)algolia(|net).(com|io|net)$

a0ef2haqr0-1.algolia.io
algolia.com
analytics.algolia.com
analytics.de.algolia.com
analytics.fve0otglpf.algolia.net
analytics.rcsnjttpht.algolia.net
analytics.us.algolia.com
analytics.uwxbmb9os2.algolia.net
insights.algolia.io
insights.de.algolia.io
insights.us.algolia.io
jn1rdqrfn5-1.algolianet.com
jn1rdqrfn5-2.algolianet.com
jn1rdqrfn5-3.algolianet.com
jn1rdqrfn5-dsn.algolia.net
logs.algolia.net
recommendation.us.algolia.com
telemetry.algolia.com
track.rcsnjttpht.algolia.net
tracker.rcsnjttpht.algolia.net
tracking.rcsnjttpht.algolia.net

(.|^)gammacdn.com$

free-at-assets.gammacdn.com
gammacdn.com
images01-buddies.gammacdn.com
images01-fame.gammacdn.com
images02-buddies.gammacdn.com
images02-fame.gammacdn.com
images03-buddies.gammacdn.com
images03-fame.gammacdn.com
images04-fame.gammacdn.com
kosmos-assets-prod.react.gammacdn.com
kosmos-prod.react.gammacdn.com
kosmos-prodv2.react.gammacdn.com
static01-cms-buddies.gammacdn.com
static01-cms-fame.gammacdn.com
static02-cms-buddies.gammacdn.com
static02-cms-fame.gammacdn.com
static03-cms-buddies.gammacdn.com
static03-cms-fame.gammacdn.com
static04-cms-buddies.gammacdn.com
static04-cms-fame.gammacdn.com
trailers-buddies.gammacdn.com
trailers-fame.gammacdn.com
transform.gammacdn.com
videothumb.gammacdn.com

^([a-z0-9]+.|)(appsync|execute)-api.us-east-1.amazonaws.com$

0bxxaty1ad.execute-api.us-east-1.amazonaws.com
1g26ewet37.execute-api.us-east-1.amazonaws.com
1s7vmel6xi.execute-api.us-east-1.amazonaws.com
2m1ln5gmga.execute-api.us-east-1.amazonaws.com
30dckcweuf.execute-api.us-east-1.amazonaws.com
4uklew74b1.execute-api.us-east-1.amazonaws.com
815jpfypwc.execute-api.us-east-1.amazonaws.com
99kz2a2ob8.execute-api.us-east-1.amazonaws.com
9w2zed1szg.execute-api.us-east-1.amazonaws.com
9w2zed1szg.execute-api.us-east-1.amazonaws.com
bin5y4muil.execute-api.us-east-1.amazonaws.com
bx7jwhkpb4.execute-api.us-east-1.amazonaws.com
e6yeun02cb.execute-api.us-east-1.amazonaws.com
fc01np5u7i.execute-api.us-east-1.amazonaws.com
fqicudrbaf.execute-api.us-east-1.amazonaws.com
l026e7vji8.execute-api.us-east-1.amazonaws.com
pn8sm7rjuc.execute-api.us-east-1.amazonaws.com
qe6evcafs0.execute-api.us-east-1.amazonaws.com
v76ndo1am9.execute-api.us-east-1.amazonaws.com
w6x8q98np4.execute-api.us-east-1.amazonaws.com
wl96h214rb.execute-api.us-east-1.amazonaws.com

Required to log into general website user account pages

^accounts..*.(com|net|org|uk|br|ly|gov)$

accounts.abgsex.net.daraz.com
accounts.adcolony.com
accounts.adespresso.com
accounts.adiant.com
accounts.adjust.com
accounts.adtelligent.com
accounts.advanced-store.com
accounts.adx1.com
accounts.amobee.com
accounts.api.affiliatewindow.com
accounts.api.binance.com
accounts.api.getadmiral.com
accounts.api.kochava.com
accounts.appdynamics.com
accounts.appier.com
accounts.apple.securelink.auh1.com
accounts.attentivemobile.com
accounts.auctiondrop.com
accounts.baileysfarmsinc.com
accounts.bi.tt.omtrdc.net
accounts.bighow.net
accounts.binance.com
accounts.ccminer.org
accounts.cmp.optimizely.com
accounts.comscore.com
accounts.conduit.com
accounts.craigslist.org-securelogin.viewpostid8162-bmayeo-carsandtrucks.evamata.com
accounts.crazyegg.com
accounts.cryptonight.net
accounts.decibelinsight.net
accounts.deepintent.com
accounts.despegar.com
accounts.dev.api.binance.com
accounts.doubleclick.net
accounts.effectivemeasure.net
accounts.emerse.com
accounts.eu.api.binance.com
accounts.eyereturn.com
accounts.fgl.com
accounts.firstpromoter.com
accounts.gocsooglc.com
accounts.gooacogle.com
accounts.google.com.notecia.inf.br
accounts.google.com.serviceloginservicemailpassivetruerm-falsecontinuemail.google.com.mail.ss1scc1tmpldefaultltmplcache2emr1osid1.financetrendnews.com
accounts.google.comads.yahoo.comafs.moatads.com
accounts.google.comafs.moatads.com
accounts.googlyoutube.com
accounts.gooog1e.com
accounts.goooglesecurity.com
accounts.growingio.com
accounts.gumgum.com
accounts.haravan.com
accounts.hotels-in-israel.com
accounts.hsoub.com
accounts.infusionsoft.com
accounts.inntelligentcrm.com
accounts.insigit.com
accounts.instagram.com.days-sa.com
accounts.int2-pmgt.api.tt.omtrdc.net
accounts.kaizenplatform.net
accounts.kidoz.net
accounts.kiosked.com
accounts.letsdoeit.com
accounts.live.resonate.com
accounts.livefyre.com
accounts.longmusic.com
accounts.loopme.com
accounts.mail.wesfrgpay.com
accounts.mantisadnetwork.com
accounts.meltwater.com
accounts.mobidea.com
accounts.mobvista.com
accounts.monumetric.com
accounts.mytaphouse.com
accounts.mywellsfargnaccount.gocoogle.resortfulelevation.com
accounts.na.global.global.tt.omtrdc.net
accounts.net.daraz.com
accounts.novaonx.com
accounts.ns.kubernetesvision.net.daraz.com
accounts.omniconvert.com
accounts.pega.com
accounts.petametrics.com
accounts.pkr.com
accounts.pqa-np.api.api.tt.omtrdc.net
accounts.qa-ext.livefyre.com
accounts.qocple.epizy.com
accounts.qwilr.com
accounts.rentshop.org
accounts.retargetly.com
accounts.revcatch.com
accounts.revenuenetwork.com
accounts.revprotect.com
accounts.salesloft.com
accounts.sattadon0001.net.daraz.com
accounts.secure.googlaidmin.com
accounts.signifyd.com
accounts.sisudata.com
accounts.skimlinks.com
accounts.slickstream.com
accounts.smaato.com
accounts.sundaysky.com
accounts.support.net.daraz.com
accounts.synchronizing.googlemail.www2.vectorstrategies.com
accounts.system-ns.org
accounts.t402.livefyre.com
accounts.targetspot.com
accounts.techibits.com
accounts.tellkomsel.com
accounts.testlib-ln-1.net.daraz.com
accounts.tradingmakeracademy.com
accounts.us-east-1.salesloft.com
accounts.us.api.binance.com
accounts.walkme.com
accounts.wordpress-catalog.com
accounts.wozbtc.com
accounts.yektanet.com
accounts.yoc.com
accounts.yotube.com
accounts.zywave.com

Blocking YouTube or Twitch Ads

I'm just curious, do the filters in this project currently help with blocking ads on either Twitch or YouTube? Those are 2 big services I'm wondering if someone already has a technique for. I've heard that blocking YouTube is currently not possible since they serve from the same servers as the videos, but maybe you've found a regex that works? Thanks!

Most of the regular expressions in this project don't work for pihole

The Pihole REGEX implementation doesn't suport some of the expressions you used in the youtube regex blocklist as stated here https://docs.pi-hole.net/regex/tutorial/

Especially your expressions in brackets use the common regex syntax but pihole has some different syntax there. This results in the targeted domains not being matched. You can use the command found here https://docs.pi-hole.net/regex/testmode/ to test domains against the active set of expressions..

My updated versions of your regex':
Displaying regex blacklist:
1: ^[\w-.]?google(?:adservices|apis|syndication|usercontent)?.com$ (disabled, last modified Tue, 09 Nov 2021 14:25:27 +0100)
2: ^[\w-.]?l.google.com$ (disabled, last modified Tue, 09 Nov 2021 14:25:43 +0100)
3: ^clients\d+?.google.com$ (disabled, last modified Tue, 09 Nov 2021 14:25:46 +0100)
4: ^r[-0-9]+sn[-.a-zA-Z0-9]+googlevideo.com (enabled, last modified Tue, 09 Nov 2021 17:24:57 +0100)
5: ^r[.0-9]+sn.*googlevideo.com$ (enabled, last modified Tue, 09 Nov 2021 17:27:59 +0100)
6: ^[-.0-9a-zA-Z]double(?:lick|click).net$ (enabled, last modified Tue, 09 Nov 2021 17:29:47 +0100)
7: ^[-.0-9a-zA-Z]?manifest.googlevideo.com$ (enabled, last modified Tue, 09 Nov 2021 19:36:57 +0100)
8: ^beacons.gv[-.0-9a-zA-Z]+.com$ (enabled, last modified Tue, 09 Nov 2021 19:41:19 +0100)
9: ^clients[0-9]+.youtube.com$ (enabled, last modified Tue, 09 Nov 2021 20:53:06 +0100)

1-3 are unchanged from your list but disabled, since I didn't see any use for blocking youtube ads so far. #1 even blocked google.com which obviously isn't desired.

I hope this is helpful to you 👍

Wrong regexes in youtube-regex-filters.txt

Hi there,

Pi-Hole v5.7 is unable to use the following regexes:
^[\w-.]?google(?:adservices|apis|syndication|usercontent)?.com$
^[\w-.]?l.google.com$

The error message is - Invalid character range.
Could you fix them please?

Pihole RegEx and Exact Lists: Script error

run.sh: line 20: Script error: The shell scripts were not found.: command not found

When I download the page contents manually and then run ./run.sh locally I get:
./run.sh: 65: Syntax error: "(" unexpected

Facebook

Hello,

I run all 4 scripts however, it seems to block to much. All profile pictures and posts on Facebook become blocked

How can I resolve this?
Thank you

Whitelist Unavailable: 404 Not Found

Hi there, just letting you know that the whitelist file linked in the wiki appears to have been removed, not sure if intentional or not so thought I'd log an issue.

Thanks for your time with this project :)

Please add description to *.list files in README.md

Hello!

Stumbled on this project via reddit. Very interesting! Had some bad luck with youtube ad blocking via Pi-hole in the past (overblocking - video not starting cause ad and video is located on same server). Haven't tested your lists yet, but I'm looking forward to.

Could you please provide a description in README.md for each list to see what it's ment for?

Cheers!