clearurls/addon

ClearURLs is an add-on based on the new WebExtensions technology and will automatically remove tracking elements from URLs to help protect your privacy.

ClearURLs

ClearURLs is an add-on based on the new WebExtensions technology and is optimized for Firefox and Chrome based browsers.

This extension will automatically remove tracking elements from URLs to help protect your privacy when browsing the Internet, which is regularly updated by us and can be found here.

Application

Many websites use tracking elements in the URL (e.g. https://example.com?utm_source=newsletter1&utm_medium=email&utm_campaign=sale) to mark your online activity. All that tracking code is not necessary for a website to be displayed or work correctly and can therefore be removed—that is exactly what ClearURLs does.

Another common example are Amazon URLs. If you search for a product on Amazon you will see a very long URL, such as:

https://www.amazon.com/dp/exampleProduct/ref=sxin_0_pb?__mk_de_DE=ÅMÅŽÕÑ&keywords=tea&pd_rd_i=exampleProduct&pd_rd_r=8d39e4cd-1e4f-43db-b6e7-72e969a84aa5&pd_rd_w=1pcKM&pd_rd_wg=hYrNl&pf_rd_p=50bbfd25-5ef7-41a2-68d6-74d854b30e30&pf_rd_r=0GMWD0YYKA7XFGX55ADP&qid=1517757263&rnid=2914120011

Indeed most of the above URL is tracking code. Once ClearURLs has cleaned the address, it will look like this: https://www.amazon.com/dp/exampleProduct

Features

• Removes tracking from URLs automatically in the background
• Blocks some common ad domains (optional)
• Has a built-in tool to clean up multiple URLs at once
• Supports redirection to the destination, without tracking services as middleman
• Adds an entry to the context menu so that links can be copied quickly and cleanly
• Blocks hyperlink auditing, also known as ping tracking (see also this article)
• Prevents ETag tracking
• Prevents tracking injection over history API (see also: The replaceState() method)
• Prevents Google from rewriting the search results (to include tracking elements)
• Prevents Yandex from rewriting the search results (to include tracking elements)

Permissons

Reasoning for needed permissions can be found under here.

Screenshot

CI/CD Artifacts Download (for Firefox- and Chrome-Dev only)

Here you can download the packed files for the Firefox- and Chrome-Dev:

• ClearURLs.zip

Test

If you want to test whether ClearURLs works correctly on your system, you can go to this test page: https://test.clearurls.xyz/

Contribute

If you have any suggestions or complaints, please create an issue.

Note: If you have any suggestions or complaints regarding the rules, please create an issue in this repo or email us rules.support (at) clearurls.xyz (this mail will automatically create a new issue in this repo).

Translate ClearURLs

You want to help translating ClearURLs into many languages? – Nice

You can choose between two options to contribute. You can create a merge request, or you can use the POEditor to translate ClearURLs.

Hint: The description field in the translation files are only an information for what the translation is used. It is not necessary to translate the description field; in the most cases it is empty.

Merge request

If you want to create a merge request, you must open the path _locales/en/messages.json in the ClearURLs repo and translate the english terms into terms of your language. Once you have translated all the terms, you make a pull request of your translation. Please push your translation into the folder _locales/{language code}/messages.json.

POEditor

Projects that use parts of ClearURLs

• Uroute used ClearURLs to filter/clean URL before launching browser
• Scrub used ClearURLs to filter/clean URLs as cog for the Red Discord bot
• Unalix small, dependency-free, fast Python package for removing tracking fields from URLs
• Unalix-nim small, dependency-free, fast Nim package and CLI tool for removing tracking fields from URLs
• UnalixAndroid simple Android app that removes link masking/tracking and optionally resolves shortened links

Recommended by...

• ghacks-user.js
• Awesome Humane Tech List
• PrivacyTools
• New York Times Wirecutter
• ClearURLs is part of Mozilla's recommended extensions program

Permissions

Reasoning for needed permissions you can find under this wiki page.

Copyright

We use some third-party scripts in our add-on. The authors and licenses are listed below.

• WebExtension browser API Polyfill | Copyright by Mozilla | MPL-2.0
• Bootstrap v4.3.1 | Copyright 2011-2016 Twitter, Inc. | MIT
• jQuery v3.6.0 | Copyright JS Foundation and other contributors | MIT
• DataTables v1.10.20 | Copyright (c) 2008-2015 SpryMedia Limited | MIT
• Pickr v1.7.0 | Copyright (c) 2018 - 2020 Simon Reinisch | MIT
• Font Awesome v5.12.0 | Copyright (c) @fontawesome | Font Awesome Free License

addon's Issues

search on google maps does not give you suggestions

open maps.google.com and then start typing a place into search
normally it shows you expanded suggestions
if ClearUrls is enabled then this feature does not work

cheers

ClearURLs: explain that not all enhancements/fixes result in a change of version

At #14 (comment) a few hours ago:

ClearURLs load the new rule file only on browser start :)

The descriptive text at https://addons.mozilla.org/addon/clearurls/ might benefit from a short paragraph to explain that behaviour.

Previously, I imagined that each improvement to ClearURLs would result in a version change; an update from addons.mozilla.org. My confusion at #8 (comment), for example.

Thanks!

Search on zillow.com [reported over addons.mozilla.com]

An issue with the search on zillow.com, reported over the mozilla addon page.

It seems to block the map on the left side of the property search result on zillow.com. This occurs when looking at a saved map search or a new map search.

[suggestion]Please increase the default items per page in the log

currently it is 5 which is a little low. it should be one of the selectable numbers. 10 fit on most screens without scrolling. if you want to keep 5 then it should be in the drop down menu so you can switch back and forth.

ClearURLs 1.1.2.5 and Google Hangouts frame /webchat/frame3

At https://hangouts.google.com/ the frame --

https://hangouts.google.com/webchat/frame3

-- will not load with ClearURLs Filters enabled.

Screen recording

2017-12-09 08:22.ogv.zip

A frame from that recording:

Available on request: two more screen recordings (a little earlier, a little longer).

Environment

Waterfox 56.0 on FreeBSD-CURRENT.

Login with Google+ at a WordPress site

https://pikeralpha.wordpress.com/2018/02/15/a-new-start/ whenever I aimed to reply with ClearURLs 1.2.1.4 filtering enabled, a click on the g+ button caused something to occur but nothing visibly Google-related.

With filtering disabled, after a reload of the page a click on the button leads to:

Connecting to Google+

– with an indeterminate progress bar (barber pole) … that seems to be endless, so I don't know whether there's something else at play here.

ClearURLs 1.1.2.4 versus Google authentication for various non-Google services: error 400, invalid_request, missing required parameter: client_id

Steps

1. have a not logged in (NLI) view of, for example, https://apple.stackexchange.com/
2. enable ClearURLs filters
3. aim to log in, e.g. https://apple.stackexchange.com/users/login?ssrc=head&returnurl=https%3a%2f%2fapple.stackexchange.com%2f
4. click the G+ Google button

Expected

• continuation of the authentication routine

Actual result

400. That’s an error.

Error: invalid_request

Missing required parameter: client_id

…

Workaround

• disable ClearURL filters
• revisit https://apple.stackexchange.com/ and follow the log in routine successfully with Google.

Other examples

If I recall correctly, services such as GitLab are similarly affected.

Add Custom URL

Hello

Possible to add custom URL ?

Or add :

https://www.facebook.com/?ref=logo
https://www.facebook.com/groups/000000000000/?ref=bookmarks
https://www.facebook.com/?sk=h_chr

Thanks

[SUGGESTION] Modifying Rules-Status checking

Hi,

Running ClearUrls 1.2.1.4 / Firefox 59.0.2 (x64) / Windows 7 (x64) flawlessly.

ClearUrls checks the Rules-Status (calling 151.101.12.133) not once a week, not once a day or hour, but at each and every Firefox start.

• These rules aren't modified at a rhythm which requires a check on every start of the browser, last update of data.json took place 2 months ago...
• The fact of being aware of connections established by the browser, hence by its extensions, on start seems to be, IMO, a valid caution, even when the connections are legitimate and healthy.

This is why I'd like to suggest an option that would let the user check manually and/or modify the automatic checking schedule.

Thanks.

Doesn't seem to filter links opened in a new tab

Prerequisites

• Can you reproduce the problem in a fresh firefox profile with only ClearURLs?

So you want me to uninstall those 10 other extensions? Sorry won't do.

• Are you running the latest version of ClearURLs?

Yes

• Are you have changed settings under about:config?

Yes but I don't remember every single change unfortunately. Btw wording for this question is wrong, isn't it "Are you have"?

Description

When I visit a link in the email, it calls firefox binary with the link as an argument. That opens a new tab on existing Firefox instance and CleanURLs won't filter utm_source, utm_campaign, etc. The same happens when I create a new tab and paste the full URL there - the original URL is used. Only after I refresh the tab, URL gets cleaned.

In contrast the 'pure-url' extension https://addons.mozilla.org/cs/firefox/addon/pure-url/ is able to filter URLs for new tabs so it looks like a ClearUrls-only problem.

Steps to Reproduce

1. Click on some stupid marketing link in an email client
2. Link opens as it is in Firefox without removing marketing tracking

Expected behavior:
CleanUrls cleans URLs on new tabs

Actual behavior:
URL is not cleaned until I hit refresh.

System

• Version: 1.2.1.4, latest as I have just installed it
• Platform: Linux xxx 4.16.10-1-k3a #2 SMP Tue May 22 00:00:23 CEST 2018 x86_64 GNU/Linux

Few Questions

1. Is there any specific reason you included doubleclick.net domain?, cause there are some websites which detect such kind of blocking and flags you as adblock user. Being a user of such awesome addon mostly means he/she is already using an adblock addon which is already blocking doubleclick.net domain.

visit https://raw.githubusercontent.com/anarki999/Adblock-List-Archive/master/Better.fyiTrackersBlocklist.txt and search "anti" on the page.

As you can see, in the above better.fyi filterlist an exception to some google ad domains are added to make adblock undetectable, I think these can be managed more efficiently through an adblock extension which supports whitelisting such as ublock origin or ghostery etc.

2. Does this extension support t.co redirection and intermediate tracking by twitter cause its reach and tracking methods are far more invasive than bit.ly or similar domains. If not, are there any chances of including it?

3. Is there any chance the functionality of Link Cleaner addon https://addons.mozilla.org/en-US/firefox/addon/link-cleaner/ can be included: it removes facebook's l.facebook.com redirection and that too without injecting javascript in webpage as it hangs the whole page and increase memory and cpu usage like affiliate killer userscript? is there any chance you can include affiliate and immediate redirection functionality just like this addon?

4. Does this addon completely remove any information sent to target domain with removed parameters or are there any chances of connecting to domains with those parameter and referrer filled domains if you hover over such kind of links and also does it just changes the url AFTER you have connected to the parameters filled domain or it let us connect only after removing the parameters?

Being said that, a huge respect and love for your out of the box approach towards this thing which sets this addon apart from the rest.

These questions are asked out of curiosity and to request to add support for features so as to minimize the installation of such kind of addons which are doing almost similar function so as to decrease the page hangs, memory and cpu consumption, incompatibility between each other and separately configuring each and every addon.

[SUGGESTION] Statistics removed or as an option

I don't understand the pertinence of the Statistics feature. It doesn't bring anything, knowing that x% of elements have been blocked is a non-information IMO, and meanwhile it writes to disk again and again.

I'd suggest removing this non-feature or setting it as a toggle option under Features - Logging - Badges.

Who reads the stats? What does it bring, an element of satisfaction? Nonsense IMO.

Otherwise, ClearUrls does a swell job : powerful, efficient.

Rule incorrectly matching asyncRefreshAnalysis=1

ClearURLs is stripping asyncRefreshAnalysis=1&asyncRefreshLocalMediaAgent=1& based on the rule [a-zA-Z_]*ref[a-zA-Z_]*=[^\/|\?|&]*(\/|&(amp;)?)?

As far as I can tell, it's matching the Ref in the parameter names, but shouldn't this rule only match lowercase ref? This is breaking a website for me. Hoping for some insight as to why these parameters are getting caught by the regex.

ClearURLs 1.1.2.4 maybe incompatible with http://www.allthatnerdystuff.com/2014/09/22/student-microsoft-office-365-free/

http://www.allthatnerdystuff.com/2014/09/22/student-microsoft-office-365-free/

When ClearURLs filters are enabled at the page, I find the page endlessly reloading.

In the midst of a long screen recording: for a while, I wondered whether AdBlock and/or Privacy Badger were contributory to the issue – and I do have many other extensions installed – but it seems to me that with ClearURLs filters disabled, I can enable both AdBlock and Privacy Badger without any looping of the affected page.

When I disable both AdBlock and Privacy Badger and enable ClearURLs filters, there's looping.

Please, can you reproduce the symptoms?

Environment

• Firefox 56.0.1 (64-bit)
• FreeBSD-CURRENT
• KDE Plasma 5
• KWin
• XRender

root@momh167-gjp4-hpelitebook8570p-freebsd:~ # uname -v
FreeBSD 12.0-CURRENT #0 r320869: Mon Jul 10 13:57:55 UTC 2017     [email protected]:/usr/obj/usr/src/sys/GENERIC 
root@momh167-gjp4-hpelitebook8570p-freebsd:~ # 

Google Docs

Hi there!

This report is for Firefox nightly. This one seems to be similar to the one I reported about Google Mail attachments.

I noticed this morning that CleanURLs breaks the use of Google Docs, it throws up an error and documents cannot be created or edited. I checked to ensure that CleanURLs was the underlying issue by disabling it and attempting to create or edit a document.

After disabling the extension the Google Docs functions without an issue.

I'm not sure if there is anything you can do about this but I figured that I would let you know regardless!

You missed an amazon paremter.

ref=

It would be good if we could add our own parameters :)

[SUGGESTION] Allow rule customization

Thank you for the helpful tool. I was wanting to customize some of the definitions (for example, removing more from Amazon URLs -- such as keywords=... and hopefully also the ref=..., but I'm not sure yet if the syntax supports that on the "path" part of the URL). What about having a way that users can specify their own data.json style rules and that gets merged with what is downloaded from github? My initial thought was something like a data.json that users can drop in a specific directory under their firefox profile, but I'm guessing extensions can't read files from the local filesystem for security reasons. So maybe entries under about:config instead? (I was just trying to think of something simple that wouldn't require new UI elements or anything overly complicated to implement.)

add ability to see which rules were triggered

It would be nice to see a detail in the notification of what rule was triggered and what the original url's were and what they are now.

Like on hover or simply log results to console. Something informative would be nice.

if I could see what the rules did I could understand the concept and offer additional elements to strip out

Possible Bug with Facebook

I tested 5 times ClearUrls with both, FF 56.0.2 and 57.0.2 (64-bit):

When scrolling down Facebook main page, at maximum speed, for more than 15 second... then the whole browser freezes (not 100%, but 90%). RAM and CPU go up. It seems like a kind of loop.

Facebook' tab also freezes, and bottom posts go in a kind of blank space in an endless upload (as you can see in the above attached gif image).

I can't say if it is an incompatibility issue with other add-ons. But I did same test with similar add-ons (Neat URL and Pure URL), and Facebook worked perfectly, no bug, no freezes.

https://i.stack.imgur.com/hBbTY.gif

ClearUrls makes Firefox 58.0b3 crash on Ubuntu 17.04

Hi, I use ClearUrls 1.1.2.4 (thanks for your work) and Firefox Beta on Ubuntu Zesty 17.04 with the official Mozilla ppa :

https://launchpad.net/~mozillateam/+archive/ubuntu/firefox-next

Up to Firefox 57, all worked well, but since this morning update and version 58.0b3, ClearUrls makes Firefox always crash, either at startup, either just when loaded and click on "New Tab", or "Addons".

FF Window becomes grey, unresponsive and has to be stopped. It happens also even if ClearUrls is the only addon activated.

Please tell me if I can provide useful crash logs :
https://crash-stats.mozilla.com/report/index/45ab8ad3-2a99-4b12-ab6b-367650171115

Simple question

Hi, please just two simple questions:

1. What are the main differences between ClearURLs and PureURL in terms of system performance, quantitative and qualitative URL garbage removing, etc?

2. Is there any reason we don't see URLs adds-on unifying functions such as "clearing", "redirecting", "skipping" etc all-in-one?

Thank you!

Feature request/suggestion

Color the icon to reflect whether filters are enabled (green) or disabled (orange or red) and possibly add an option to replace badges with a different color (e.g. yellow) when active or use a different icon completely (e.g. can with lid open).

Few Facebook referer trackers

These are some parameters not removed when browsing facebook:
hc_ref, pnref, see_more_ref, ref, hc_ref, fref, timeline_context_item_type, timeline_context_item_source

YouTube search suggestions not working

Using ClearUrls 1.2 on Firefox 57.0.2

The search suggestions on YouTube aren't working as one or more of the gs_ parameters are required.

Before processing:

https://clients1.google.com/complete/search?client=youtube&hl=en&gl=de&sugexp=ytcle%2Cryff%3D1%2Crwff%3D0%2Cyrl%3D10%2Ccpf%3D1%2Ccfro%3D1%2Cccl%3D1%2Ccpf%3D1%2Cryff%3D1%2Crwff%3D0%2Cyrl%3D10&gs_rn=64&gs_ri=youtube&ds=yt&cp=7&gs_id=t&q=searchstring&callback=google.sbox.p50&gs_gbg=some_id

After processing:

https://clients1.google.com/complete/search?client=youtube&hl=en&gl=de&sugexp=ytcle%2Cryff%3D1%2Crwff%3D0%2Cyrl%3D10%2Ccpf%3D1%2Ccfro%3D1%2Cccl%3D1%2Ccpf%3D1%2Cryff%3D1%2Crwff%3D0%2Cyrl%3D10&ds=yt&cp=7&q=searchstring&callback=google.sbox.p50&

Rule:

gs_[a-zA-Z]=[^\/|\?|&](/|&(amp;)?)?

trkid= not cleaned from Netflix links when codex.nflxext.com is blocked

Prerequisites

• [Yes, with the exception of uBo and uMatrix] Can you reproduce the problem in a fresh firefox profile with only ClearURLs?
• [Yes] Are you running the latest version of ClearURLs?
• [Yes & No] Are you have changed settings under about:config?

Description

Not sure if this is a bug or not, but codex.nflxext.com script has to be allowed in order for a trkid Netflix link to be cleaned. For example, with uBo and uMatrix installed, going to https://www.netflix.com/title/80166314?trkid=1234567&MSG_TITLE=80166314&lnktrk=EMP&g=6E5416B8D93DD75DB84131269FB4725A6455A8C2&lkid=W2W_ROW_1_MDP_ will only convert the link to https://www.netflix.com/title/80166314 once it has been allowed. It seems the conversion shouldn't be reliant on requiring script to run on the destination page.

I was going to make a separate issue for the fact that google search results aren't cleaned by ClearURLs, but I now suspect it's likely due to the same cause. It seems very counter-productive to globally require 3rd-party scripts just to clean links, so I'm hoping this is just a bug and not a requirement for it to function.

Steps to Reproduce

1. Install uBo and uMatrix and set to block 3rd-party (alternately, you could just use uBo and forego uMatrix, as the result would be the same, since uMatrix basically just allows more fine-grained control)
2. Go to above link and notice it's not cleaned
3. Allow the above in uBo and, if applicable, allow scripts for it in uMatrix, then go to the link again and notice that it's cleaned

Expected behavior: [What you expected to happen]

The link should be cleaned regardless of what scripts are allowed to run on the site

Actual behavior: [What actually happened]

Netflix links (and others?) aren't cleaned until certain 3rd-party scripts are allowed/unblocked.

System

• Version: 1.2.1.4
• Platform: Win10x64

Log

There were no log entries created with the script was blocked. Both entries are from loading the page a single time after unblocking it. Also, I've noticed many entries have the "This domain is blocked" statement in the rule column.

Before processing	After processing	Rule	Time

https://adtech.nflximg.net/adtech_iframe_target_03.html?data=%7B%22membership_status%22%3A%22NON_REGISTERED_MEMBER%22%2C%22country%22%3A%22US%22%2C%22source%22%3A%22https%3A%2F%2Fwww.netflix.com%2Ftitle%2F80166314%3Ftrkid%3D13710079%26amp%3BMSG_TITLE%3D80166314%26amp%3Blnktrk%3DEMP%26amp%3Bg%3D3E5616B8D73DD75DB84131269FB4725A6455A8C2%26amp%3Blkid%3DW2W_ROW_1_MDP_1%22%2C%22fbaId%22%3A%226ef0941b-6a2f-47a9-89db-e12565dc2d3e%22%2C%22is_member%22%3A%22anonymous%22%2C%22wasFormerMember%22%3Afalse%2C%22referrer%22%3A%22title%22%7D		.*	5/8/2018, 6:31:24 AM
https://adtech.nflximg.net/adtech_iframe_target_03.html?data=%7B%22membership_status%22%3A%22NON_REGISTERED_MEMBER%22%2C%22country%22%3A%22US%22%2C%22source%22%3A%22https%3A%2F%2Fwww.netflix.com%2Ftitle%2F80166314%3Ftrkid%3D13710079%26amp%3BMSG_TITLE%3D80166314%26amp%3Blnktrk%3DEMP%26amp%3Bg%3D3E5616B8D73DD75DB84131269FB4725A6455A8C2%26amp%3Blkid%3DW2W_ROW_1_MDP_1%22%2C%22fbaId%22%3A%226ef0941b-6a2f-47a9-89db-e12565dc2d3e%22%2C%22is_member%22%3A%22anonymous%22%2C%22wasFormerMember%22%3Afalse%2C%22referrer%22%3A%22title%22%7D	https://adtech.nflximg.net/adtech_iframe_target_03.html?data=%7B%22membership_status%22%3A%22NON_REGISTERED_MEMBER%22%2C%22country%22%3A%22US%22%2C%22source%22%3A%22https%3A%2F%2Fwww.netflix.com%2Ftitle%2F80166314%3Ftrkid%3D13710079%26amp%3BMSG_TITLE%3D80166314%26amp%3Blnktrk%3DEMP%26amp%3Bg%3D3E5616B8D73DD75DB84131269FB4725A6455A8C2%26amp%3Blkid%3DW2W_ROW_1_MDP_1%22%2C%22fbaId%22%3A%226ef0941b-6a2f-47a9-89db-e12565dc2d3e%22%2C%22is_member%22%3A%22anonymous%22%2C%22wasFormerMember%22%3Afalse%2C%22referrer%22%3A%22title%22%7D	This domain is blocked.	5/8/2018, 6:31:24 AM

sponsored links on amazon

Sponsored links don't work on amazon. We examine this.

Example:
https://www.amazon.de/gp/slredirect/picassoRedirect.html/ref=sspa_dk_detail_15?ie=UTF8&adId=A01928193EG7FPZM5KMNC&qualifier=1511903202&id=5497001537359525&widgetName=sp_detail2&url=%2Fdp%2FB073Z7FBWC%2Fref%3Dsspa_dk_detail_15%3Fpsc%3D1

Facebook comment plugin [Archiving purposes]

[Archiving purposes]:

Ed @addons.mozilla.org:

I am having some trouble with this addon. When used with the Disconnect addon or Firefox's built-in Tracking Protection and the Cookie AutoDelete addon, the Show Comments button at the end of each article on politico.com becomes non-functional. Are there conflicts among these items?

The facebook plugins needs the href field. So we're adding an exception for facebook plugins,
".*(facebook\\.)\\w{2,}.*(\\/plugins\\/).*", in 98d6b45.

ClearUrls is nice, but is killing my browser

A lot of webpages are taking a long time to load (e.g. DuckDuckGo), while some stuck the browser for ever, like this one (/!\ Save your session first, you will have to SIGTERM Firefox if affected):
https://itsfoss.com/best-icon-themes-ubuntu-16-04/

I’ve reproduced this in a clean Nightly profile (just installing ClearUrls and going to above URL), but haven’t had the time to check with release/beta.

BUG: e10s + Facebook issue

Prerequisites

• [yes] Can you reproduce the problem in a fresh firefox profile with only ClearURLs?
• [yes] Are you running the latest version of ClearURLs?
• [yes & no (see below)] Are you have changed settings under about:config?

Description

When running Firefox or Waterfox in multi-process (e10s) mode, when ClearURLs is active (filters enabled), it causes the entire browser to break when using Facebook. I first noticed it in Waterfox 56.1.0 x64 and spent hours narrowing it down to this addon and then checking different variations to see when it happens and when it doesn't. It's definitely ClearURLs, which I've further verified by the fact the same thing happens with a fresh Firefox 59.0.3 x64 install with only this addon installed. And the problem happens regardless of the logging and badges settings; it's only related to whether it's filtering or not. I'm not sure what exactly triggers it, or if other sites are affected, though I tried with Amazon fairly extensively and with some other sites a bit. Facebook works fairly regularly, though not consistently.

What happens is that everything will be fine, then all of a sudden clicking on the icon for any addon in the navigation toolbar or the status/addon bar (in Waterfox with Classic Theme Restorer addon) will result in it only showing a sliver of that addon's pop-up, and the rest doesn't show. The items in the bookmarks bar don't seem to be affected. The other thing that happens is that the entire browser stops loading web pages. All tabs are affected. In fact, when it first started happening I was testing something out which required copy/pasting into Facebook chat and reloading the page, and when it kept breaking I thought for a while it was my crappy internet, until I finally tried another browser. The only way to get everything working again is by restarting the browser or possibly by waiting a while (several minutes or more), since I think that's what I did when it happened at first and I thought it was the internet dropping out.

Steps to Reproduce

1. Install ClearURLs and enable filtering
2. Log in to Facebook
3. The thing that seems to trigger it the most consistently is opening a chat with someone and typing or pasting something into it, then refreshing the page. It's too inconsistent for me to be able to figure out more than this, i.e. I'm not sure if typing something then deleting it before refreshing has more or less of a chance of causing it, or if it's even necessary to type anything. And in fact, when I first tested it in Firefox after doing most of the testing in Waterfox, Firefox seemed to break easier/more consistently, but it still seemed to require a page reload. After refreshing Firefox (to wipe the slate), all it took was to load Facebook and it would instantly break. So I would restart Firefox, and when it reopened it would automatically reload Facebook and it would break right away again. Disabling ClearURLs stopped this. Disabling multiprocess in Waterfox stops it as well.

Expected behavior: [What you expected to happen]

Should be obvious, but using ClearURLs shouldn't break the browser when using Facebook and e10s.

Actual behavior: [What actually happened]

See above

System

• Version: 1.2.1.4
• Platform: Win10x64

Log

Nothing was logged in Firefox. I will try to get a log from Waterfox.

Breaks Fanatical link

Prerequisites

• Can you reproduce the problem in a fresh firefox profile with only ClearURLs? - immediately after refreshing Firefox and installing ClearURLs, the page worked, but then a couple minutes later I tried again and it was broken until disabling filtering
• Are you running the latest version of ClearURLs?
• Are you have changed settings under about:config? - yes, in Waterfox, but not in Firefox

Description

Going to this link with ClearURLs filtering enabled, the page doesn't load properly. Disabling filtering allows it to load. Disabling other addons doesn't have any effect, have similar results in Firefox with clean profile (see above). This is in Waterfox 56.2.0 x64, but since it happens in Firefox too that doesn't seem to matter.

Steps to Reproduce

1. Enable filtering
2. Click the link
3. Disable filtering
4. Refresh the page

Expected behavior: [What you expected to happen]

The page should load properly with ClearURLs active. Also, it might be helpful to have some sort of whitelisting feature.

Actual behavior: [What actually happened]

The page doesn't load properly unless ClearURLs is disabled.

System

• Version: 1.2.1.4
• Platform: Win10x64 Pro

Log

Before processing	After processing	Rule	Time
https://fanatical.cdn.prismic.io/api/v2/documents/search?page=1&pageSize=1&lang=*&ref=Wv7hTCAAAGRi9xhk&q=%5B%5Bat(my.blog_template_groups.uid%2C%20%22win-one-of-ten-top-steam-pc-games%22)%5D%5D	https://fanatical.cdn.prismic.io/api/v2/documents/search?page=1&pageSize=1&lang=*&q=%5B%5Bat(my.blog_template_groups.uid%2C%20%22win-one-of-ten-top-steam-pc-games%22)%5D%5D	[?|&]ref[_]?=[^\/|\?|&]*	5/19/2018, 3:51:50 PM

Question regarding logging

When logging is enabled, where I can find the logs?

Gmail Attachments

Hi there!

This report is for Firefox nightly.

I noticed this morning that CleanURLs breaks the downloading of attachments from GMail. I checked to ensure that CleanURLs was the underlying issue by disabling it and attempting to download the same attachment.

After disabling the extension the attachment downloaded without an issue.

I'm not sure if there is anything you can do about this but I figured that I would let you know regardless!

?ftag=

In response to a notification (from https://www.cnet.com, I guess, where I allow notifications), I found myself at:

• https://www.cnet.com/news/turns-out-pluto-might-really-be-a-giant-comet/?ftag=CAD-03-10aaj8j

Enhancement request

Please, would it be possible/appropriate to apply filters for things such as ?ftag=CAD-03-10aaj8j to be cleared?

https://www.cnet.com/news/turns-out-pluto-might-really-be-a-giant-comet/ appears fine, as far as I can tell.

Environment

Waterfox 56.2.0 on FreeBSD-CURRENT,

$ pkg info waterfox | grep Version
Version        : 56.2.0.13_4
$ date ; uname -v
Thu 24 May 2018 18:38:57 BST
FreeBSD 12.0-CURRENT #2 r333587: Mon May 14 01:59:02 BST 2018     root@momh167-gjp4-hpelitebook8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC 

ClearURLs 1.2.2.8

ClearURLs 1.1.2.5: GUI

Toolbar counter

This switch has a good name.

Reset counter

A better name for this button will be:

• Reset statistics

strange filter result of rule [a-zA-Z]*ref=[^\/|\?|&]*(\/|&(amp;)?)?

Before processing
http://www.facebook.com/plugins/like.php?locale=sl_SI&href=http://24ur.com/bin/jokes.php?joke_id=1491&layout=standard&show_faces=true&width=250&action=like&colorscheme=light&height=28&show_faces=false

After processing:
http://www.facebook.com/plugins/like.php?locale=sl_SI&/24ur.com/bin/jokes.php?joke_id=1491&layout=standard&show_faces=true&width=250&action=like&colorscheme=light&height=28&show_faces=false

Rule:
[a-zA-Z]*ref=[^\/|\?|&]*(\/|&(amp;)?)?

page reload issue on dslreports.com

http://www.dslreports.com/shownews/FCC-Commissioner-Wants-Your-Broadband-Availability-Horror-Story-140621

?ref=producthunt

For example,

https://mapsofmind.com/?ref=producthunt referred from https://www.producthunt.com/posts/mapsofmind

[suggestion] custom data.json update url

the default rules work great so far but some users want to have custom rules as well. since you already have a rule update mechanism i thought the easiest way to allow custom rules would be to allow to change the update url so users can make and share custom rules easily.

related to #38

Google Translate broken

Hello,

It seems that your add-on breaks Google Translate.

Try a simple translation like "hello" with your add-on enabled, and we get a "Translation Error" as result.

When I disable the option "Filters", the translation is correctly done.

[Contribute wanted] Translate ClearURLs

Dear community,

since version 1.2.2.8 I have integrated the i18n API feature from Mozilla (https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/i18n). Now it is possible to easily translate ClearURLs into all languages.

We all know that the translations from "Google Translate" and the like are not very good. So I'm asking the community to help me translate ClearURLs.

Translating ClearURLs is easy. Open the path _locales/en/messages.json (https://github.com/KevinRoebert/ClearUrls/blob/master/_locales/en/messages.json) in the ClearURLs repo and translate the english terms into terms of your language. Once you have translated all the terms, you make a pull request of your translation. Please push your translation into the folder _locales/{country code}/messages.json.

I'm curious into how many languages we translate ClearURLs.

ClearURLs freezes Firefox 58 Beta on Windows 10 on Reddit.com if RES extension is always installed.

I have RES extension and ClearURLs. The browser updates from FIrefox 57 Beta to Firefox 58 Beta. All websites work except for www.reddit.com, visiting which will freeze the whole browser and make it use 22% CPU and 30% GPU. If I disable RES extension and restart the browser, ClearURLs works properly on Reddit. If I disable ClearURLs, the RES works properly. They just don't get along with one another.

Few days ago I noticed that while opening Reddit RES Dashboard, which loads lots of subreddits in one window, the ClearURLs would detect 8,000 entries and filter few hundred of them. Not sure if it's related.

Breaks some Amazon functionality

Prerequisites

• [Yes] Can you reproduce the problem in a fresh firefox profile with only ClearURLs?
• [Yes] Are you running the latest version of ClearURLs?
• [Yes & No] Are you have changed settings under about:config?

Description

For items with a size selection (e.g. https://www.amazon.com/Bionic-DVML-Driving-Gloves-Black/dp/B001ZTRDCI/ref=sr_1_2?keywords=bionic%2Bdriving%2Bgloves&th=1&psc=1), you can't select size. When trying, most of the item description and images and links to other products are all grayed out and the page becomes less responsive (basically it's as if it's stuck in the middle of the change); products with a row of multiple choices where you click the box you want (products with different lenghts, colors, etc) seem to work, so there appears to be something different with the single drop-down selection method for clothing sizes; the above site did work a couple times with ClearURLs enabled, but it failed to work a few dozen times, and it was very consistent about not working and working when ClearURLs was filtering and not filtering, respectively. This is in both Watefox 56.1.0 x64 and Firefox 59.0.3 x64, with Firefox on a clean profile.

Steps to Reproduce

1. Go to above site
2. Change the size (it might take a few tries, but it typically happens pretty quickly)
3. Go to an item with multiple length/color/diameter/etc choices (good options are string, rope, heat shrink, cables, etc) where the different choices are chosen by selecting different boxes instead of different options within a single drop-down. These product pages are unaffected.

Expected behavior: [What you expected to happen]

The page should function normally.

Actual behavior: [What actually happened]

The page breaks, becoming more or less frozen (though not locked up).

System

• Version: 1.2.1.4
• Platform: Win10x64

Log

Nothing was logged

Possibly misleading formulation in wiki page 1.1

The Badges switch activates the badge functionality (See WebExtensions API) of the addon. In the current version (1.2.x.x) the addon counts the blocked elements per url.

shouldn't the last word "url" rather be "tab"?

Android support

I just installed this on Android but even when installed all counters are stuck at 0 so I'm assuming it doesn't work. Can Android support be added?

Some tracking links

Some links which can be cleaned

https://www.reddit.com/r/subreddit?ref=trending_sr_somenum

https://www.reddit.com/r/subreddit/comments/random/title/random/?context=3&utm_content=context&utm_medium=user&utm_source=reddit&utm_name=frontpage

https://www.reddit.com/r/subreddit/comments/random/title/random/?sh=random&st=random

https://out.reddit.com/someRandomCharacters?url=https%3A%2F%2Factual.url.com

https://twitter.com/twitter?ref_src=twsrc^google|twcamp^serp|twgr^author
(when visiting twitter from google or search engines/affiliates)

https://twitter.com/search?q=term&src=tren

https://youtube.com/v=*&feature=youtu.be&a (feature tracks users who clicked the link. also 't' parameter may sometimes be there so might need to take care of that properly)

https://addons.mozilla.org/whatever/firefox/addon/someaddon/?src=hp-dl-mostpopular (addons site has a few of these but I heard they can't be cleared for security reasons. is this still true?)

request users are somehow notified of rule changes

Since updates to data.json are done server-side, for safety it would be nice to know when/how the rules were changed. While I trust you I think I should know when the rules change. Not only from a security perspective but also to aid in testing rule changes.

Your addon brokes Google reverse image search

Hi,

If I search an image on Google with Reverse Image Search and with ClearUrls activated, this will not works (open a new tab with the same image).
It works if I deactivate ClearUrls.
Firefox 55.0.3 x64 on Win7 x64.

Regards.

ClearURLs 1.2.1.4 versus sending messages with riot.im/app

1.2.1.4 with Waterfox 56.1.0_18 on FreeBSD-CURRENT. Many extensions enabled … as far as I can tell, when filters are enabled it becomes difficult, sometimes impossible to send messages in https://riot.im/app.

The first of 94 entries whilst logging was enabled:

• https://matrix.org/_matrix/client/r0/rooms/!vfFxDRtZSSdspfTSEr%3Amatrix.org/send/m.room.message/m1525789170942.22? before
• https://matrix.org/_matrix/client/r0/rooms/!vfFxDRtZSSdspfTSEr%3Amatrix.org/send/m.room.message/m1525789170942.22 after.

At a glance, all other entries are comparable – without the trailing ?. Will it help to provide the entire log? If so:

• where (in the file system) can I find it?

I have a long screen recording of the behaviours with, and without, filtering. In the recording I also experimented with disabling features of some other extensions, including Malwarebytes Browser Extension BETA and uBlock Origin.

I'll test with a cleaner profile (far fewer extensions) …

switching to a new log page doesn't scroll up

reproduce steps:
open log
select "show 100 entries"
scroll to the bottom
select any other page

expected would be to start from top, but intead the page stays at bottom

Opens a blank tab when clicking outside links on Reddit.

If I click a link to another site on Reddit it just opens a blank website (about:blank in the url bar). It works though if I right click on the link and open it.