rfc-st/humble

A humble, and 𝗳𝗮𝘀𝘁, security-oriented HTTP headers analyzer.

humble's Issues

Finetune returned "Missing HTTP Security Headers".

I would like to have a way to finetune humble in a way that only headers are checked according to my input.
e.g. if I don't want "NEL" to be implemented, I would like to skip it in the "missing HTTP Security Headers" section.

AttributeError: module 'urllib3.util.ssl_' has no attribute 'DEFAULT_CIPHERS'

The Issue

After cloning and using the project in a virtual env and install requirements, i had this error message when launching the program :

└─$ python humble.py -u https://business-together-as-a-service.com/                                                                                                                                                               

 Analyzing URL, please wait ...

Traceback (most recent call last):
  File "/home/milan/Stuff/Tools/humble/humble.py", line 872, in <module>
    requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'urllib3.util.ssl_' has no attribute 'DEFAULT_CIPHERS'

OS and Python infos

Python 3.11.4 on kali linux
Distributor ID: Kali
Description: Kali GNU/Linux Rolling
Release: 2023.3
Codename: kali-rolling

How i fixed this :

I fixed the issue by uninstalling urllib3 ( urllib3-2.0.4 ) and adding urllib3<2 in requirements.txt

Avoiding HTTP 403 errors

Currently a single header (a well-formed 'User-Agent', associated with a real browser version) is sent in 'humble' requests. In most cases this is not a problem, and the HTTP response headers are retrieved correctly.

On other occasions, the domain responds with a "403 Forbidden". This may be due to the presence of a WAF or a GDPR IP blocking but I get the impression that this error is actually caused by the request being interpreted as being made by a bot (by not including certain headers that might be necessary.)

I have tried many combinations: including the default request headers sent by curl, Chrome, etc, without success.

I need your help to identify a pattern that allows 'humble' to retrieve the HTTP response headers in these specific cases. Any ideas, advices or suggestions will be welcome and, of course, I will mention them in the README of the project!!.

Thank you.

Feature request

Is your feature request related to a problem? Please describe.
when saving the output to a txt file, it is being saved with generic name, therefore if i scan 2 wpengine sites (ex: sxq.wpengine.com and 123.wpengine.com), the file name that is being saved is the same (wpengine_header_date). So if i run scan on 5 wpenging sites, it keeps on overwriting the name of the file since its the same as prior scan.

Describe the solution you'd like
Able to include a txt file that humble can use to scan a list of sites and save the output as txt/pdf/html file. Also, save the output as the full URL (123.wpengine.com_date) instead of generic (ex:wpendine_header_date)

Describe alternatives you've considered
manually doing it

Additional context
Add any other context or screenshots about the feature request here.

X-XSS-Protection issues

According to OWASP Secure Headers Project, the X-XSS-Protection header is deprecated.

Although I didn't find a formal RFC or document saying it's deprecated, The caniuse shows that almost all modern browsers are not supporting this HTTP header, and MDN also mentioned that this is non-standard and recommends using Content-Security-Policy instead.

Perhaps X-XSS-Protection should have the issue as Deprecated Header in addition to Unsafe Value and Duplicated Values

P.S. There might be a typo In insecure.txt, where v is lower case in Content-Security-Policy: Incorrect values and others are Values

Support JSON output including details

It would be nice if humble presents a better output format including more details like CWE numbers or a severity justification.
HSTS could have the following CWE number: https://cwe.mitre.org/data/definitions/319.html

This would be a benefit for e.g. https://defectdojo.github.io/django-DefectDojo/integrations/parsers/file/ as I would like to use humble and parse the information of your scanner.

Alot More to add!

hello there, my name is ArkAngeL43, Angel is prefered, im a cyber weapons developer currently studying 20+ programming languages, as of now focusing on ruby, i recreated this tool in ruby while not being as affective i think there could be alot more to do to it

in my script i added the following
WHOIS
Port
Server
Server OS
Scheme
request code
a port scanner

i originally came across your tool and i thought why not remake it in ruby, so i did im not going to say its better because there is a shit ton of bugs in it however there are somethings you can add, since this is just a http analyzer its not worth it going into port scanners, however i think it would be great to add

scheme, connection code, request sent and request given back, methods, server name, and since your doing python maybe consider a vulnerability tester for the current server. python is a great language for speed, ability, portability, and so much more. i think this tool really could be great if you add some more things like the suggestions above

if you want to know more about my thoughts please do message me here and if you want which would be easier you can direct message me on instagram

sec_re43p3r_scare

i hope my suggestions help you in the further future, and i hope i can help. thank you for taking your time to read this!

date ==> Tue 14 Sep 2021 10:56:17 PM
name ==> Angel, RE43P3R, Reaper, or arkangel

have a great night/day

Growing together!

What is 'humble' missing?; being reasonable :):

• What HTTP header checks do you miss?.
• What totals do you think should be reported?.
• What results export format do you need and why?.

Let's use this issue to talk about the shortcomings of 'humble' (without preventing specific issues to be created for specific topics), ideas to make it more interesting and accessible for everyone. With total freedom and regardless of your technical knowledge,

I have the healthy habit of thanking all the contributions that result in an improvement of this tool: https://github.com/rfc-st/humble/?tab=readme-ov-file#acknowledgements .

Thanks for your time!.

Distinguish between HTTP and HTTPS response

Is your feature request related to a problem? Please describe.
When checking headers for HTTP and HTTPS the output file has the same name, thus overwriting the result of another one.
Also, the result of the header should stick to the given URL. Currently, humble will follow the redirection or get the response headers using HTTPS even when HTTP schema is specified

Describe the solution you'd like
Provide an option for specifying the output file name.
I've seen the previous issue #5 and it's solution. However, I think it'll be more flexible to allow others to choose their own filename instead of adding the new attribute to the original filename.

Provide an option for the user to choose whether automatic redirection is necessary.

X-Permitted-Cross-Domain-Policies Header

According to OWASP Secure Headers Project , the HTTP Header X-Permitted-Cross-Domain-Policies Header should also be used. A check is mssing regarding this header.

Trailer checking

Describe the bug
The checking for disallowed directives of Trailer is conflicting

To Reproduce
Steps to reproduce the behavior:

1. For example, a header like Trailer: Authorization
2. The output will not contain the result where the Trailer is using Authorization which is a disallowed directive

Expected behavior
Should have the result, Trailer (Disallowed Directives)

Additional context

if 'Trailer' in headers:
    trailer_h = headers['Trailer'].lower()
    if any(elem in trailer_h for elem in l_trailer):
        print_detail_r('[itrailer_h]', is_red=True)
        if not args.brief:
            matches_trailer = [x for x in l_trailer if x in trailer_h]
            print_detail_l("[itrailer_d_s]")
            print(', '.join(matches_trailer))
            print_detail("[itrailer_d_r]")
        i_cnt[0] += 1

l_trailer = ['Authorization', 'Cache-Control', 'Content-Encoding',
             'Content-Length', 'Content-Type', 'Content-Range', 'Host',
             'Max-Forwards', 'Set-Cookie', 'TE', 'Trailer',
             'Transfer-Encoding']

The any check goes through l_trailer which contains capital letters in every element while it uses .lower() to convert the actual value of Trailer. Therefore, the condition was never met.

SyntaxError: invalid syntax File "humble.py", line 137 with open(analysis_h_file, 'a+', encoding='utf8') as a_history

Fixed with on line 137:

- with (open(analysis_h_file, 'a+', encoding='utf8') as a_history,
-       open(analysis_h_file, 'r', encoding='utf8') as c_history):
+ with open(analysis_h_file, 'a+', encoding='utf8') as a_history, \
+       open(analysis_h_file, 'r', encoding='utf8') as c_history:

Fix humble scanner output if target not available

I am using v 1.30 of Humble and tried out the JSON output.

In case the target is not reachable, the output of "Missing Security Headers" should be empty. Right now the output indicates that the target is available.

See at the following output example:

https://github.com/DefectDojo/django-DefectDojo/pull/8989/files#diff-ff6a401d1d37c4f253ab3df1e2c0963f254bb27a23d687542eaf105766f9608d

Add option to skip certificate validation

I couldn't find any switch to disable certificate validation. Would be nice to have. I.e. if used with proxy tools like burp or proxychains.