bongochong/cwp-utilities

Combined Windows Privacy Utilities | Hosts file updater, block list manager, and more. Open source tools for Windows users, to help ensure privacy & security. Block ads, spyware domains, and other malicious activity/traffic, all through a simple interface.

Combined Windows Privacy Utilities

3/02/2023: I had some setbacks recently due to a significant death in the family, but have been working on relatively major additions and optimizations for a new release, which should arrive in a couple of months (or hopefully sooner). I'm still actively maintaining and updating this repo, so bear with me.

Are you tired of ads and trackers affecting your web browsing and eating up bandwidth? Do you want to use file sharing applications in a safer way, with less risk of being snooped on? Do you want to take preemptive measures against spyware? Or, would you simply like to have an easier way of keeping your hosts file and torrent client block lists thorough and up-to-date on Windows? You've come to the right place if you answered yes to any of those questions.

Combined Windows Privacy Utilities (or CWP Utilities) is a free and open source suite of integrated tools for Windows users, to help them guard their privacy and security. It does this through drastically simplifying the work of parsing and structuring data from a number of different trustworthy block lists, then combining them into single, usable files, and putting them in their proper locations. A simple installer is provided, to make everything easily accessible. The program can be installed anywhere on any device that runs Windows, along with USB drives, as all of the utilities dynamically recognize system paths. It creates no registry entries, cleans up after itself, and if one chooses to uninstall it, no files are left behind. Please consider donating to the project if you find it useful. It is also highly recommended to use the "Update Scripts" feature after installing, as this will ensure the utilities are in sync with the current versions present in this repository.

CWP Utilities automates the process of maintaining an up-to-date and comprehensive hosts file, in order to block a variety of trackers, advertisements, and malware, on a system-wide level. It also enables the average user to easily keep the block lists for their file sharing applications up-to-date and thorough, along with facilitating the easy utilization of a Proxy Auto-Configuration (or PAC) file, for ad and tracker-blocking on devices that are less configurable (e.g. non-rooted phones). Think of it as a more capable replacement for HostsMan, even though it's not as pretty. It can also fetch hosts lists from servers HostsMan cannot, thanks to an up-to-date build of wget for Windows.

CWP Utilities currently provides a system tray-based interface and a simple, interactive command line interface, allowing the user to perform the aforementioned tasks hassle free. The hosts file updater automatically installs the resultant list in the correct system location and flushes the DNS cache for immediate use. Along with merging and sorting the data from multiple block lists out of the box, it also reduces the size of the hosts file via an optimization script, allowing for slower machines to utilize larger block lists. Obviously it de-duplicates the data as well, and has an option to create entries that will block malicious traffic over IPv6 too.

The file sharing block list updaters output their lists in a subdirectory of each tool's folder, which can be moved to another directory of the user's choosing thereafter. An explorer window appears upon completion of the block list update, revealing the location of the list. The PAC updater does the same, but a template script is also included that allows for greater flexibility.

All the scripts which perform these routines are configurable directly in a text editor of one's choosing, or via the simple yet capable text editor shipped with CWP Utilities. More useful features are planned for the future, as is a more flexible GUI, but I thought I would start sharing fully functional releases with the public, in hopes that others will find my tools as useful and beneficial as I have.

This set of utilities is inspired by a lot of scripting work I have done over the years to automate repetitive tasks on my personal devices, and during my stints as a SysAdmin. It is built around significantly friendlier Windows versions of the scripts from my Combined Privacy Block Lists repository, which also contains some lists I maintain, and more software. If anyone would like to pitch in - especially in regard to crafting a more comprehensive GUI - I would be more than happy to work with you as time allows.

All of the scripts and underlying binaries have been tested on 32 and 64 bit versions of Windows 7, Windows 8.1, and Windows 10, though it should all work in Vista as well.

TLDR: This is a bunch of batch scripts I wrote, which I hacked together into an integrated utility. The scripts use Windows ports of GNU utilities and some other FOSS software, in order to aid in automating the process of updating and editing various files for blocking undesirable traffic on your computers and phones. It sports both a command line interface, and a simple system tray GUI.

Notes: It is strongly encouraged to use the installer, as it provides useful shortcuts which bring you directly to the main interface and each individual utility. If you prefer to run the program manually, you can run the LauncherLauncher.bat or CWPU-Tray.exe files, both of which are housed in the root folder of the install directory. Alternatively, you may run WinUtilsMultiLauncher.bat from the same location, as long as you do so with administrative privileges. Do not attempt to run CWP Utilities outside of using the the aforementioned methods, as it will not work as intended if you do.

Though one can also simply use pre-made block lists like the ones I provide in my other repo and many others, CWP Utilities cuts out the middle man, allowing you to generate and use your own lists, directly from the sources. I believe that the default configurations are far better than adequate for the vast majority of users, though if you have any experience with shell scripting, then modifying the scripts in CWP Utilities should be a relatively simple process (which is why options for doing so are included).

Disclaimer: It should also be noted that a small number of AV scanners report false-positive results for this suite of utilities. This is solely due to the administrative privileges necessary for CWP Utilities to function properly (specifically the hosts file tools). ClamAV and the vast majority of widely used AV engines report that CWP Utilities is clean software (which it undoubtedly is), and this has been confirmed by the fine folks at Softpedia as well.

Addendum: As of 7/28/2020, Windows Defender is reported to now block changes to the hosts file on some installations of Windows 8.1 and 10. This is a profoundly ridiculous move on the part of Microsoft. If you are affected by this issue, please simply add an exclusion rule for your hosts file (%systemroot%\system32\drivers\etc\hosts) to Defender. This affects users of HostsMan and other block list utilities too. See here: for more info.

---

More Screenshots:

cwp-utilities's Issues

Some suggestions

Thank you for making this useful tool available.

Some suggestions, based on my experience with the latest version of "Update Hosts (Standard)":

Add to the documentation, that it is possible to add your own custom hosts file entries, like this:

• Place your custom hosts file entries in a text file.
• Name this text file hosts.99.
• Place this text file in C:\Program Files (x86)\CWPUtilities\cwpumain\bin.
• In "CWP Utilities SysTray", choose Edit Configs > Hosts (Standard) to open the editor.
• In the editor, search for this line: busybox cat processing/hosts/hosts.* > processing/hosts/hosts-cat.final.
• When this line has been found in the editor, insert the following line above the found line: copy hosts.99 processing\hosts.
• Choose File > Save in the editor to save the change in the script.

Another suggestion: Add the option to the program to use an ignore list, i.e. a user created text file with URLs that are blocked in one or more of the downloaded blocking lists, but which should not be blocked.

Some more suggestions:

For each of the files that are to be downloaded with curl -s -o processing/hosts/hosts.*, check whether this file can be downloaded, and if not, display a warning.

When the file final-hosts.txt has been created, check the size of this file. If this file has a size of zero, abort the script with an error message that says that nothing could be downloaded (or something like that).

When the hosts file in C:\Windows\System32\drivers\etc is a read-only file, at the end of the script, this misleading message appears: Done! Your host file has been updated, while this is not the case. A few lines earlier it says Access is denied. 0 file(s) copied. I guess some people may overlook this earlier message. My suggestion is, to check first whether the hosts file is a read-only file, and if so, abort the script with an error message. (Or, as an alternative, if this is possible: Before trying to write to the hosts file, if needed, the script changes the read-only attribute of the hosts file, so writing to it is possible. When the hosts file was originally a read-only file, after writing to the hosts file, the script makes it a read-only file again.)

With some firewalls or other security programs, it is possible to prevent modifications to the hosts file. When such a prevention has been set, at the end of the script, this misleading message appears: Done! Your host file has been updated, while this is not the case. A few lines earlier it says The process cannot access the file because it is being used by another process. 0 file(s) copied. I guess some people may overlook this earlier message. My suggestion is, to check first whether the hosts file has been blocked by another program, and if so, abort the script with an error message.

You may consider to add https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts to the Defaults Hosts File Sources (and then remove the three sources "MVPS", "PGL Yoyo", and "URL Haus", because they then become duplicates, as they are included in the StevenBlack source).

NOTE: The link in the script to "URL Haus" (https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-hosts-online.txt), links to an old version of the "URL Haus" list, while the link in https://github.com/bongochong/CWP-Utilities/blob/master/MoreInfo/DefaultHostsLists.md links to the current version of the "URL Haus" list.

[RESOLVED] Amazing, thank you (and portable version)

Thank you for sharing with us your work over the years. I have a potentially dumb question: is there a portable version of CWP-Utilities?

[RESOLVED] Default Hosts

Firstly, thanks for this neat tool. it works flawlessly and saves a lot of time.

The Malware domains list is seemingly no longer maintained and now just points to an empty file, it should probably be removed from the default list of hosts files.

Here are some active malware lists you could used instead:
https://raw.githubusercontent.com/davidonzo/Threat-Intel/master/lists/latestdomains.piHole.txt
https://curben.gitlab.io/malware-filter/urlhaus-filter-hosts-online.txt
https://raw.githubusercontent.com/mitchellkrogza/Badd-Boyz-Hosts/master/hosts
https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Risk/hosts

[RESOLVED] Malwarebytes flags 'elevate.exe' as malicious

VirusTotal results

Might be worth informing users that this is a false positive.