unapibageek/ctfr

Abusing Certificate Transparency logs for getting HTTPS websites subdomains.

CTFR

Do you miss AXFR technique? This tool allows to get the subdomains from a HTTPS website in a few seconds.
How it works? CTFR does not use neither dictionary attack nor brute-force, it just abuses of Certificate Transparency logs.
For more information about CT logs, check www.certificate-transparency.org and crt.sh.

Getting Started

Please, follow the instructions below for installing and run CTFR.

Pre-requisites

Make sure you have installed the following tools:

Python 3.0 or later.
pip3 (sudo apt-get install python3-pip).

Installing

$ git clone https://github.com/UnaPibaGeek/ctfr.git
$ cd ctfr
$ pip3 install -r requirements.txt

Running

$ python3 ctfr.py --help

Usage

Parameters and examples of use.

Parameters

-d --domain [target_domain] (required)
-o --output [output_file] (optional)

Examples

$ python3 ctfr.py -d starbucks.com

$ python3 ctfr.py -d facebook.com -o /home/shei/subdomains_fb.txt

With Docker

I think it's a little bit crazy to use Docker for running such a little python script, but if you want to do it anyway, you can use this Docker image.

The instructions are there.

Screenshots

Author

• Sheila A. Berta - (@UnaPibaGeek).

ctfr's Issues

Missleading description

Since the tool is making an HTTP request to CRT.sh and parsing the JSON, the description should state this instead of mentioning that the tool itself is abusing certificate transparency logs.

TypeError

I got an error. Please check the following outputs;

python3 ctfr.py -d starbucks.com

      ____ _____ _____ ____
     / ___|_   _|  ___|  _ \
    | |     | | | |_  | |_) |
    | |___  | | |  _| |  _ <
     \____| |_| |_|   |_| \_\

 Version 1.2 - Hey don't miss AXFR!
Made by Sheila A. Berta (UnaPibaGeek)

Traceback (most recent call last):
File "ctfr.py", line 80, in
main()
File "ctfr.py", line 65, in main
subdomains.append(value['name_value'])
TypeError: list indices must be integers or slices, not str

AttributeError: module 'simplejson' has no attribute 'loads'

SO: Ubuntu 18.04.4
Commands:
git clone https://github.com/UnaPibaGeek/ctfr.git
cd ctfr
pip3 install -r requirements.txt

which python3.6 python pip3 pip
/usr/bin/python3.6
/usr/bin/python
/home/user/.local/bin/pip3
/home/user/.local/bin/pip

pip3 --version
pip 19.2.3 from /home/user/.local/lib/python3.6/site-packages/pip (python 3.6)

python3 ctfr.py -d google.com -o outfile

Error:
Traceback (most recent call last):
File "ctfr.py", line 77, in <module>
main()
File "ctfr.py", line 61, in main
for (key,value) in enumerate(req.json()):
File "/home/user/.local/lib/python3.6/site-packages/requests/models.py", line 888, in json
return complexjson.loads(
AttributeError: module 'simplejson' has no attribute 'loads'

works with:

python ctfr.py -d google.com -o outfile

unicode error

Traceback (most recent call last):
File "ctfr.py", line 71, in
main()
File "ctfr.py", line 53, in main
json_data = json.loads('[{}]'.format(req.text.replace('}{', '},{')))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xdc' in position 51138: ordinal not in range(128)

Error con algunos dominios

Ejemplo:
python ctfr.py -d madrid.es

File "ctfr.py", line 80, in
main()
File "ctfr.py", line 62, in main
json_data = json.loads('[{}]'.format(req.text.replace('}{', '},{')))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 480: ordinal not in range(128)

Gracias por el programa.

[X] Information not available!

HI spend a lot of time without using this tool but now iam seeing an error
[X] Information not available!
on every scan attempt
thanks

Doesn't work with the .FISH TLD.

I tried with this site https://bread.fish but got an error.

[$:~/programs/ctfr] ctfr.py bread.fish

          ____ _____ _____ ____
         / ___|_   _|  ___|  _ \
        | |     | | | |_  | |_) |
        | |___  | | |  _| |  _ <
         \____| |_| |_|   |_| \_\

     Version 1.2 - Hey don't miss AXFR!
    Made by Sheila A. Berta (UnaPibaGeek)

[X] Error! Invalid domain or information not available!

Failed to parse

I cloned the latest and pip3 install -r requirements.txt - all was good no errors. Pretty much new kali-linux-headless install on raspi4

python3 ctfr.py -d google.com

      ____ _____ _____ ____
     / ___|_   _|  ___|  _ \
    | |     | | | |_  | |_) |
    | |___  | | |  _| |  _ <
     \____| |_| |_|   |_| \_\

 Version 1.2 - Hey don't miss AXFR!
Made by Sheila A. Berta (UnaPibaGeek)

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/models.py", line 382, in prepare_url
scheme, auth, host, port, path, query, fragment = parse_url(url)
File "/usr/lib/python3/dist-packages/urllib3/util/url.py", line 394, in parse_url
return six.raise_from(LocationParseError(source_url), None)
File "", line 3, in raise_from
urllib3.exceptions.LocationParseError: Failed to parse: https://crt.sh/?q=%.google.com&output=json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/ctfr/ctfr.py", line 77, in
main()
File "/opt/ctfr/ctfr.py", line 55, in main
req = requests.get("https://crt.sh/?q=%.{d}&output=json".format(d=target))
File "/usr/lib/python3/dist-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 528, in request
prep = self.prepare_request(req)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 456, in prepare_request
p.prepare(
File "/usr/lib/python3/dist-packages/requests/models.py", line 316, in prepare
self.prepare_url(url, params)
File "/usr/lib/python3/dist-packages/requests/models.py", line 384, in prepare_url
raise InvalidURL(*e.args)
requests.exceptions.InvalidURL: Failed to parse: https://crt.sh/?q=%.google.com&output=json

Feature Request: Read multiple domains from txt file

I am absolutely loving this repo and I feel if you added a feature to where it could read a list of domain names (example: "github.com" - no quotes) from a txt file with each domain being on it's own new line, this would make the script not only 10x better in my opinion, but also 100x more productive and less time-consuming to utilize.

With the above said, I understand this project is in its beginning stages, and that's why I am requesting this feature. Furthermore, if you do end up adding this feature, although this would in no way be required, an added bonus would be utilizing multithreads to dump the subdomains contained within any txt file specified on the command line at a much faster pace. However, as I said, this would only be an added bonus (maybe something to consider down the road). As far as I'm concerned, reading multiple domains from a txt file while using a single thread would be just fine! =)

Thanks for your time and consideration.

Information not available

python ctfr.py -d google.com -o google

and yahoo.com

[X] Information not available!

Thanks

compatibility

This works on Python 2.7 fine, don't see why requires python3 or later :P