aboul3la/sublist3r

Fast subdomains enumeration tool for penetration testers

sublist3r's Issues

Multiple Problems During Bruteforce

1. Tried running sublist3r using 20 threads on bruteforce and it freezes. Verbose is ON but it's not enumerating any more. I've waited for almost 6 hours but nothing happened.
2. During bruteforce (verbose is ON), multiple copies of the same subdomain are being enumerated. I think this causes the bruteforce method to take some considerable amount of time.
3. Sometimes, running sublist3r prints this error:

Process NetcraftEnum-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 461, in run
domain_list = self.enumerate()
File "sublist3r.py", line 506, in enumerate
cookies = self.create_cookies(resp.headers['set-cookie'])
File "/usr/lib/python2.7/dist-packages/requests/structures.py", line 56, in getitem
return self._store[key.lower()][1]
KeyError: 'set-cookie'

BUG: error parsing dnsdumpster

hi,

the error is

[-] Searching now in Baidu..
[-] Searching now in Yahoo..
[-] Searching now in Google..
[-] Searching now in Bing..
[-] Searching now in Ask..
[-] Searching now in Netcraft..
[-] Searching now in DNSdumpster..
Process DNSdumpster-7:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "sublist3r.py", line 549, in run
    domain_list = self.enumerate()
  File "sublist3r.py", line 604, in enumerate
    self.extract_domains(post_resp)
  File "sublist3r.py", line 611, in extract_domains
    results_tbl = tbl_regex.findall(resp)[0]
IndexError: list index out of range

thanks,

roy

Netcraft error - looks like Netcraft have changed something

Process NetcraftEnum-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 461, in run
domain_list = self.enumerate()
File "sublist3r.py", line 506, in enumerate
cookies = self.create_cookies(resp.headers['set-cookie'])
File "/usr/local/lib/python2.7/dist-packages/requests/structures.py", line 56, in getitem
return self._store[key.lower()][1]
KeyError: 'set-cookie'
[!] Error: Google probably now is blocking our requests
[~] Finished now the Google Enumeration ...

Using Ports

Hi, can you make ports scaning not comma separated but also ports range for example 80-2000

Baidu - Subdomain Enum issue

Hi,

Trying to use your tool I got this erro just seconds after I start the process:

"
Process BaiduEnum-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 264, in run
domain_list = self.enumerate()
File "sublist3r.py", line 237, in enumerate
links = self.extract_domains(resp)
File "sublist3r.py", line 489, in extract_domains
return links
UnboundLocalError: local variable 'links' referenced before assignment
"
Any idea how to fix this?

Cheers.

dns resolver/message error

Hi,
it was working fine, having this issue since two days.

error:

[Traceback (most recent call last):
  File "E:\Sublist3r\sublist3r.py", line 20, in <module>
    from subbrute import subbrute
  File "E:\Sublist3r\subbrute\subbrute.py", line 18, in <module>
    import dns.resolver
  File "C:\Users\paresh\AppData\Roaming\Python\Python27\site-packages\dns\resolver.py", line 35, in <module>
    import dns.message
  File "C:\Users\paresh\AppData\Roaming\Python\Python27\site-packages\dns\message.py", line 191
    print('id %d' % self.id, file=s)
                                 ^
SyntaxError: invalid syntax

my python version: 2.7
dnspython==1.15.0

same type of issue: https://github.com/byt3bl33d3r/MITMf/issues/245
any solution for this ?

Thanks

Why it is important?

Sorry my ignorance.

Avoid bare exceptions

See https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ for description and justification.

AttributeError: 'NoneType' object has no attribute 'content'

https://asciinema.org/a/7r6ye52ojj3eqc13wygx6xcor

Python Version: 2.7.6

two issues

1.the first one
Process AskEnum-5:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 196, in run
domain_list = self.enumerate()
File "sublist3r.py", line 164, in enumerate
resp = self.send_req(query, page_no)
File "sublist3r.py", line 110, in send_req
return resp.text
AttributeError: 'Response' object has no attribute 'text'

if the "return resp.text" is false and should replace into "return resp" ?

2.a second one
Process BingEnum-4:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 196, in run
domain_list = self.enumerate()
File "sublist3r.py", line 169, in enumerate
links = self.extract_domains(resp)
File "sublist3r.py", line 369, in extract_domains
return links_list
UnboundLocalError: local variable 'links_list' referenced before assignment

why dose the error occur?

Hey The scan stops after about 10 seconds, And I get a message like this ...

Traceback (most recent call last):
File "sublist3r.py", line 1025, in
res = main(domain, threads, savefile, ports, silent=False, verbose=verbose, enable_bruteforce=enable_bruteforce, engines=engines)
File "sublist3r.py", line 971, in main
enum.join()
File "C:\Python27.007\lib\threading.py", line 940, in join
self.__block.wait()
File "C:\Python27.007\lib\threading.py", line 340, in wait
waiter.acquire()
KeyboardInterrupt

Error on collecting requirements.txt when running on macbook

Hi!
I got the following errors while running: pip install requirements.txt.

bash-3.2$ pip install requirements.txt Collecting requirements.txt Could not find a version that satisfies the requirement requirements.txt (from versions: ) No matching distribution found for requirements.txt

Thanks

may be you can do more error check?

in the word not every country can use google and so on fluently. i change you code may be i can push it .

Error in Yahoo

Got the following error when using your latest copy of sublist3r:

Process YahooEnum-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "sublist3r.py", line 205, in run
    domain_list = self.enumerate()
  File "sublist3r.py", line 178, in enumerate
    links = self.extract_domains(resp)
  File "sublist3r.py", line 290, in extract_domains
    return links_list
UnboundLocalError: local variable 'links_list' referenced before assignment

Problem with DNSdumpster

Got the following error:

root@kali:~/Sublist3r# ./sublist3r.py -t 40 -d site.com

             ____        _     _ _     _   _____
            / ___| _   _| |__ | (_)___| |_|___ / _ __
            \___ \| | | | '_ \| | / __| __| |_ \| '__|
             ___) | |_| | |_) | | \__ \ |_ ___) | |
            |____/ \__,_|_.__/|_|_|___/\__|____/|_|

             # Coded By Ahmed Aboul-Ela - @aboul3la

[-] Enumerating subdomains now for site.com
[-] Searching now in Baidu..
[-] Searching now in Yahoo..
[-] Searching now in Google..
[-] Searching now in Bing..
[-] Searching now in Ask..
[-] Searching now in Netcraft..
[-] Searching now in DNSdumpster..
[-] Searching now in Virustotal..
[-] Searching now in ThreatCrowd..
[-] Searching now in SSL Certificates..
[-] Searching now in PassiveDNS..
Process DNSdumpster-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 565, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 625, in enumerate
token = self.get_csrftoken(resp)
File "./sublist3r.py", line 620, in get_csrftoken
token = csrf_regex.findall(resp)[0]
IndexError: list index out of range

not scan

root@kali:~/Sublist3r-master# ./sublist3r.py -d 1

             ____        _     _ _     _   _____
            / ___| _   _| |__ | (_)___| |_|___ / _ __
            \___ \| | | | '_ \| | / __| __| |_ \| '__|
             ___) | |_| | |_) | | \__ \ |_ ___) | |
            |____/ \__,_|_.__/|_|_|___/\__|____/|_|

            # Coded By Ahmed Aboul-Ela - @aboul3la

[-] Enumerating subdomains now for 1
[-] Searching now in Baidu..
[-] Searching now in Yahoo..
[-] Searching now in Google..
[-] Searching now in Bing..
[-] Searching now in Ask..
[-] Searching now in Netcraft..
[-] Searching now in DNSdumpster..
[-] Searching now in Virustotal..
[-] Searching now in ThreatCrowd..
[-] Searching now in SSL Certificates..
[-] Searching now in PassiveDNS..
HTTPSConnectionPool(host='searchdns.netcraft.com', port=443): Max retries exceeded with url: /?restriction=site+ends+with&host=example.com (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff2b757b610>: Failed to establish a new connection: [Errno -2] Name or service not known',))
Process GoogleEnum-4:
Process BaiduEnum-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 251, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 251, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 224, in enumerate
links = self.extract_domains(resp)
File "./sublist3r.py", line 476, in extract_domains
return links
UnboundLocalError: local variable 'links' referenced before assignment
File "./sublist3r.py", line 222, in enumerate
if not self.check_response_errors(resp):
File "./sublist3r.py", line 285, in check_response_errors
if 'Our systems have detected unusual traffic' in resp:
TypeError: argument of type 'int' is not iterable
Process BingEnum-5:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 251, in run
domain_list = self.enumerate()
Process NetcraftEnum-7:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 224, in enumerate
links = self.extract_domains(resp)
File "./sublist3r.py", line 251, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 430, in extract_domains
return links_list
UnboundLocalError: local variable 'links_list' referenced before assignment
File "./sublist3r.py", line 551, in enumerate
cookies = self.get_cookies(resp.headers)
AttributeError: 'NoneType' object has no attribute 'headers'
HTTPSConnectionPool(host='dnsdumpster.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff2b757b610>: Failed to establish a new connection: [Errno -2] Name or service not known',))
Process AskEnum-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 251, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 224, in enumerate
links = self.extract_domains(resp)
File "./sublist3r.py", line 382, in extract_domains
return links_list
UnboundLocalError: local variable 'links_list' referenced before assignment
Process DNSdumpster-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 251, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 634, in enumerate
token = self.get_csrftoken(resp)
File "./sublist3r.py", line 629, in get_csrftoken
token = csrf_regex.findall(resp)[0]
TypeError: expected string or buffer
HTTPConnectionPool(host='ptrarchive.com', port=80): Max retries exceeded with url: /tools/search.htm?label=1.com (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7ff2b757b710>: Failed to establish a new connection: [Errno -2] Name or service not known',))
HTTPSConnectionPool(host='www.virustotal.com', port=443): Max retries exceeded with url: /en/domain/1.com/information/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff2b757b610>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Need some changes in yahoo enemuration class

Subdomains are not scanned by yahoo, so on line 330 need to change to
:
link_regx2 = re.compile('(.*?)')

thanks

Additional options

1.How to add own search engine such as ya.ru, mail.ru, censys.io, shodan.io
2. How to search by IP address?
3. Can you add robtex.com results?

Warning: No nameservers found, trying fallback list.

Warning: No nameservers found, trying fallback list.

error and programm crashes

After bruteforce module starts, am getting the errors

Hi, am getting these errors on arch after the bruteforce module starts. i tried with both python 2.7 and 3.6. Please assist

 [-] Starting bruteforce module now using subbrute..
Process lookup-16:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/dns/inet.py", line 104, in is_multicast
    first = ord(dns.ipv4.inet_aton(text)[0])
TypeError: ord() expected string of length 1, but int found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/dns/inet.py", line 108, in is_multicast
    first = ord(dns.ipv6.inet_aton(text)[0])
  File "/usr/lib/python3.6/site-packages/dns/ipv6.py", line 153, in inet_aton
    raise dns.exception.SyntaxError
dns.exception.SyntaxError: Text input is malformed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/home/iqzer0/tools/sublist3r/subbrute/subbrute.py", line 334, in run
    response = self.check(hostname, record_type)
  File "/home/iqzer0/tools/sublist3r/subbrute/subbrute.py", line 299, in check
    raise e
  File "/home/iqzer0/tools/sublist3r/subbrute/subbrute.py", line 227, in check
    resp = self.resolver.query(host)
  File "/usr/lib/python3.6/site-packages/dns/resolver.py", line 962, in query
    source_port=source_port)
  File "/usr/lib/python3.6/site-packages/dns/query.py", line 243, in udp
    (dns.inet.is_multicast(where) and
  File "/usr/lib/python3.6/site-packages/dns/inet.py", line 111, in is_multicast
    raise ValueError
ValueError
Process lookup-38:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/dns/inet.py", line 104, in is_multicast
    first = ord(dns.ipv4.inet_aton(text)[0])
TypeError: ord() expected string of length 1, but int found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/dns/inet.py", line 108, in is_multicast
    first = ord(dns.ipv6.inet_aton(text)[0])
  File "/usr/lib/python3.6/site-packages/dns/ipv6.py", line 153, in inet_aton
    raise dns.exception.SyntaxError
dns.exception.SyntaxError: Text input is malformed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/home/iqzer0/tools/sublist3r/subbrute/subbrute.py", line 334, in run
    response = self.check(hostname, record_type)
  File "/home/iqzer0/tools/sublist3r/subbrute/subbrute.py", line 299, in check
    raise e
  File "/home/iqzer0/tools/sublist3r/subbrute/subbrute.py", line 227, in check
    resp = self.resolver.query(host)
  File "/usr/lib/python3.6/site-packages/dns/resolver.py", line 962, in query
    source_port=source_port)
  File "/usr/lib/python3.6/site-packages/dns/query.py", line 243, in udp
    (dns.inet.is_multicast(where) and
  File "/usr/lib/python3.6/site-packages/dns/inet.py", line 111, in is_multicast
    raise ValueError
ValueError

Incorrect characters in the domain name can abort writing results to the file

Hello, i noticed one issue - in cases, when parsed domain has unsupported characters, and sublist3r runs on Windows 7, it can lead to the exception during file writing (for example, when enumerating shopify.com subdomains):

f.write(subdomain + "\r\n")
UnicodeEncodeError: 'ascii' codec can't encode character u'\uf0fc' in position 0
: ordinal not in range(128)

I fixed it on the line 108 using workaround, by changing the
f.write(subdomain + "\r\n")
to
f.write(subdomain.encode('utf-8') + "\r\n")
but i think it can be fixed by updating the regular expression also.

Better error handling for timeouts

Hi,

I'm sometimes getting unhandled exceptions due to timeouts:

Process NetcraftEnum-7:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "sublist3r.py", line 465, in run
    domain_list = self.enumerate()
  File "sublist3r.py", line 524, in enumerate
    self.extract_domains(resp)
  File "sublist3r.py", line 544, in extract_domains
    return links_list
UnboundLocalError: local variable 'links_list' referenced before assignment
HTTPConnectionPool(host='toolbar.netcraft.com', port=80): Read timed out. (read timeout=10)

Maybe you want to fix that and/or increase the timeouts?

Bruteforce Method Taking Too Long

It says that subbrute is integrated in this tool, and subbrute claims to be the fastest subdomain enum tool. When sublist3r is running the bruteforce method using the subbrute, it takes really long time to finish. Sometimes, the bruteforce is running for around 3-4 hrs and it still not yet finished. Is it normal?

More verbose outputs

Hi
Thanks for the tool.
I think it would be great to tell how many subdomains have been found from each search engine
eg.

[-] Searching now in Baidu (2 found records )
[-] Searching now in Yahoo  (3 found records )
[-] Searching now in Google  (12 found records )
[-] Searching now in Bing (10 found records )
[-] Searching now in Ask (1 found record )
[-] Searching now in Netcraft (5 found records )
[-] Searching now in DNSdumpster (0 found records )

and/or more statistical information like

• Search Engines: Found 10 subdomains
• Bruteforce: Found 10 subdomains

Thank!

Multiple Errors with python 2.7.11 on OSX 10.10

`tyr:~/Code/github/Sublist3r (master) benc$ python2.7 sublist3r.py -d domainname.us -o ~/tmp/sublist3r.out | tee ~/tmp/sublist3r.tee.txt

             ____        _     _ _     _   _____
            / ___| _   _| |__ | (_)___| |_|___ / _ __
            \___ \| | | | '_ \| | / __| __| |_ \| '__|
             ___) | |_| | |_) | | \__ \ |_ ___) | |
            |____/ \__,_|_.__/|_|_|___/\__|____/|_|

# Fast Subdomains Enumeration tool using Search Engines and BruteForce
# Coded By Ahmed Aboul-Ela - @aboul3la
# Special Thanks to Ibrahim Mosaad - @ibrahim_mosaad for his contributions

[-] Enumerating subdomains now for reticulum.us
[-] Searching now in Baidu..
[-] Searching now in Yahoo..
[-] Searching now in Google..
[-] Searching now in Bing..
[-] Searching now in Ask..
[-] Searching now in Netcraft..
[-] Searching now in DNSdumpster..
[!] Error: Google probably now is blocking our requests
[~] Finished now the Google Enumeration ...
Process NetcraftEnum-6:
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 461, in run
domain_list = self.enumerate()
File "sublist3r.py", line 505, in enumerate
resp = self.req(start_url)
File "sublist3r.py", line 477, in req
resp = self.session.get(url, headers=headers, timeout=self.timeout,cookies=cookies)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 487, in get
return self.request('GET', url, *_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, *_send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 585, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 459, in send
raise ConnectTimeout(e, request=request)
ConnectTimeout: HTTPConnectionPool(host='searchdns.netcraft.com', port=80): Max retries exceeded with url: /?restriction=site+ends+with&host=example.com (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x10ca6e290>, 'Connection to searchdns.netcraft.com timed out. (connect timeout=10)'))
HTTPConnectionPool(host='searchdns.netcraft.com', port=80): Max retries exceeded with url: /?restriction=site+ends+with&host=example.com (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x10ca6e290>, 'Connection to searchdns.netcraft.com timed out. (connect timeout=10)'))`

No log file was written at all! Here is the output of the tee:
`tyr:~/Code/github/Sublist3r (master) benc$ cat ~/tmp/sublist3r.tee.txt

             ____        _     _ _     _   _____
            / ___| _   _| |__ | (_)___| |_|___ / _ __
            \___ \| | | | '_ \| | / __| __| |_ \| '__|
             ___) | |_| | |_) | | \__ \ |_ ___) | |
            |____/ \__,_|_.__/|_|_|___/\__|____/|_|

# Fast Subdomains Enumeration tool using Search Engines and BruteForce
# Coded By Ahmed Aboul-Ela - @aboul3la
# Special Thanks to Ibrahim Mosaad - @ibrahim_mosaad for his contributions

[-] Enumerating subdomains now for reticulum.us
[-] Searching now in Baidu..
[-] Searching now in Yahoo..
[-] Searching now in Google..
[-] Searching now in Bing..
[-] Searching now in Ask..
[-] Searching now in Netcraft..
[-] Searching now in DNSdumpster..
[!] Error: Google probably now is blocking our requests
[~] Finished now the Google Enumeration ...
HTTPConnectionPool(host='searchdns.netcraft.com', port=80): Max retries exceeded with url: /?restriction=site+ends+with&host=example.com (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x10ca6e290>, 'Connection to searchdns.netcraft.com timed out. (connect timeout=10)'))`

I even tried:
sudo pip2.7 freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U

This did not change the errors

New version broken in windows

I guess new version is broken in windows while on linux it seems to be working fine. Please view screenshot given below.

Output to file does not save ports scanned

While the port scan is a useful feature, the output to file flag does not append the scanned ports to the file. Maybe add a --save-ports flag so the user can choose whether to do it?

handle ports & e-mail-adresses better

If you get results like b.example.com:1234 and b.example.com , they should be unified in the summary.
Same goes for [email protected]. We care about the subdomain, not the localuser part of the e-mail adress

Move engines classes to separate files

Hi, can you please move classes to separate files and then import them to main script?
I trying to do that but nothing is working then. as i not a python programmer. Please help.

parsing issue that adds an unrelated domain

If i want subdomains listed for gu.com, the result contains

Bing: d.gu.com
Netcraft: d.gu.com
PassiveDNS: staticmail.sendprovider-mta-q4rkgu.com
ThreatCrowd: t.gu.com

wwwspbocom.gu.com
wwwtst.gu.com
staticmail.sendprovider-mta-q4rkgu.com

The last one is imho not related to the domain i was looking for

Incorporate dnsdb.org

This is a another great site to add.

Is this project active?

It's been a while since the last commit and my PR hasn't had any activity. Are you still maintaining this project?

Readme should state which Python versions are supported

The argparsemodule is built into Python 3.2 and later.

everytime it prints error when run

everytime it prints error when run

[-] Searching now in DNSdumpster..
[Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "C:\tools\Sublist3r-master\sublist3r.py", line 196, in run
    domain_list = self.enumerate()
  File "C:\tools\Sublist3r-master\sublist3r.py", line 164, in enumerate
    resp = self.send_req(query, page_no)
  File "C:\tools\Sublist3r-master\sublist3r.py", line 106, in send_req
    resp = self.session.get(url, headers=headers, timeout=self.timeout)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 447, in send
    raise SSLError(e, request=request)
SSLError: [Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

[Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "C:\tools\Sublist3r-master\sublist3r.py", line 196, in run
    domain_list = self.enumerate()
  File "C:\tools\Sublist3r-master\sublist3r.py", line 164, in enumerate
    resp = self.send_req(query, page_no)
  File "C:\tools\Sublist3r-master\sublist3r.py", line 106, in send_req
    resp = self.session.get(url, headers=headers, timeout=self.timeout)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 447, in send
    raise SSLError(e, request=request)
SSLError: [Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Scan domains from newline delimited file

It would be great to have the ability to scan multiple domains from one program.

Right now you can do a work around by looping through the file with domains and running a sublist3r process against each domain.

Subbrute also supports this with the '-t' option.

No module named engines.engine

getting this error

File "./sublist3r.py", line 26, in
from subscann3r import SubScann3r
File "/root/Sublist3r/subscann3r.py", line 13, in
from engines.engine import Engines
ImportError: No module named engines.engine

Problems when running

Any idea why I am getting the following problem?

✘ maranhao@crystal  ~/test/Sublist3r   master  python sublist3r.py -d www.example.com

             ____        _     _ _     _   _____
            / ___| _   _| |__ | (_)___| |_|___ / _ __
            \___ \| | | | '_ \| | / __| __| |_ \| '__|
             ___) | |_| | |_) | | \__ \ |_ ___) | |
            |____/ \__,_|_.__/|_|_|___/\__|____/|_|

             # Coded By Ahmed Aboul-Ela - @aboul3la

[-] Enumerating subdomains now for www.example.com
[-] Searching now in Baidu..
[-] Searching now in Yahoo..
[-] Searching now in Google..
[-] Searching now in Bing..
[-] Searching now in Ask..
[-] Searching now in Netcraft..
[-] Searching now in DNSdumpster..
[-] Searching now in Virustotal..
[-] Searching now in SSL Certificates..
[-] Searching now in PassiveDNS..
Process GoogleEnum-4:
Traceback (most recent call last):
File "/Users/maranhao/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 202, in run
domain_list = self.enumerate()
File "sublist3r.py", line 170, in enumerate
resp = self.send_req(query, page_no)
File "sublist3r.py", line 110, in send_req
return self.get_response(resp)
UnboundLocalError: local variable 'resp' referenced before assignment
Process BaiduEnum-2:
Traceback (most recent call last):
File "/Users/maranhao/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 202, in run
domain_list = self.enumerate()
File "sublist3r.py", line 170, in enumerate
resp = self.send_req(query, page_no)
File "sublist3r.py", line 110, in send_req
return self.get_response(resp)
UnboundLocalError: local variable 'resp' referenced before assignment

Would you support Python3 ?

false positive subdomains

Got lots of false positive subdomains on the output, still tryna find a way to update the resolver but i dont have any clue with it haha

Type Error

I was using this with the bruteforece option and 4 threads (not sure yet how many my connection can handle) and after a while i got this:

Exception in thread Thread-18:
Traceback (most recent call last):
File "C:\Program Files\Python35-32\lib\site-packages\dns\inet.py", line 104, i
n is_multicast
first = ord(dns.ipv4.inet_aton(text)[0])
TypeError: ord() expected string of length 1, but int found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python35-32\lib\site-packages\dns\inet.py", line 108, i
n is_multicast
first = ord(dns.ipv6.inet_aton(text)[0])
File "C:\Program Files\Python35-32\lib\site-packages\dns\ipv6.py", line 153, i
n inet_aton
raise dns.exception.SyntaxError
dns.exception.SyntaxError: Text input is malformed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python35-32\lib\threading.py", line 914, in bootstrap
inner
self.run()
File "C:\Users\Sam\Downloads\Sublist3r\subbrute\subbrute.py", line 334, in run

response = self.check(hostname, record_type)

File "C:\Users\Sam\Downloads\Sublist3r\subbrute\subbrute.py", line 299, in che
ck
raise e
File "C:\Users\Sam\Downloads\Sublist3r\subbrute\subbrute.py", line 227, in che
ck
resp = self.resolver.query(host)
File "C:\Program Files\Python35-32\lib\site-packages\dns\resolver.py", line 96
2, in query
source_port=source_port)
File "C:\Program Files\Python35-32\lib\site-packages\dns\query.py", line 243,
in udp
(dns.inet.is_multicast(where) and
File "C:\Program Files\Python35-32\lib\site-packages\dns\inet.py", line 111, i
n is_multicast
raise ValueError
ValueError

What does this want to tell me?

subbrute was integrated with Sublist3r

As you wrote subbrute was integrated with Sublist3r

How to use Sublist3r for:
Tests multiple domains
delimited list of domains
save each domain result in separate txt file

Tor and Proxy use

Can your program use Tor or Proxy? Can you make it working? take allready working script from sqlmap )))

Bruteforcing option not working in windows

How can I make it work. Is it a known issue?

Add the option to get Domains from the SAN part of the TLS Cert

Given the fact that some website operators are lazy and use their TLS Certificates for more than one servera nd more than one domain, i would like to have Sublis3r get additional domains from the SAN of the certificate for the domain we were looking at originally.
If we are checking example.com and the certificate also is valid for *.example-cdn.com
we should recursively crawl for this as well to find a.example-cdn.com and b.example-cdn.com
If we are bruteforcing we might have to loop though cdn1.example-cdn.com to get
www.cdn1.example-cdn.com or ns1.cdn1.example-cdn.com

‎Modularize

It would be awesome if sublist3r could be modularized, so that we could easily use it in other python programs.

PassiveDNS Blocking sublist3r

It looks like the only domain returned by passivedns is
older.sublist3r.versions.work.better.for.<domain>
Should this source maybe just be removed?

AttributeError: portscan instance has no attribute 'print_'

nice tool,thanks for sharing it. i got this:

[-] Total Unique Subdomains Found: 18
[-] Start port scan now for the following ports: 80,443,8080,81,82,8081,8082
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "sublist3r.py", line 880, in port_scan
    self.print_("%s%s%s - %sFound open ports:%s %s%s%s"%(G,host,W,R,W,Y,', '.join(openports),W))
AttributeError: portscan instance has no attribute 'print_'

Exception in thread Thread-18:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "sublist3r.py", line 880, in port_scan
    self.print_("%s%s%s - %sFound open ports:%s %s%s%s"%(G,host,W,R,W,Y,', '.join(openports),W))
AttributeError: portscan instance has no attribute 'print_'
[-] Total Unique Subdomains Found: 18
[-] Start port scan now for the following ports: 80,443,8080,81,82,8081,8082
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "sublist3r.py", line 880, in port_scan
    self.print_("%s%s%s - %sFound open ports:%s %s%s%s"%(G,host,W,R,W,Y,', '.join(openports),W))
AttributeError: portscan instance has no attribute 'print_'

Exception in thread Thread-18:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "sublist3r.py", line 880, in port_scan
    self.print_("%s%s%s - %sFound open ports:%s %s%s%s"%(G,host,W,R,W,Y,', '.join(openports),W))
AttributeError: portscan instance has no attribute 'print_'

Add please this

Hi! Can you add please count to verbose? I need to see how much domains to brute left. In example

bruteforcing 0/10000
bruteforcing 100/10000
bruteforcing 3500/10000
bruteforcing 10000/10000 (complete)

File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap

Process AskEnum-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 264, in run
domain_list = self.enumerate()
File "sublist3r.py", line 237, in enumerate
links = self.extract_domains(resp)
File "sublist3r.py", line 395, in extract_domains
return links_list
UnboundLocalError: local variable 'links_list' referenced before assignment
Process GoogleEnum-4:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "sublist3r.py", line 264, in run
domain_list = self.enumerate()
File "sublist3r.py", line 235, in enumerate
if not self.check_response_errors(resp):
File "sublist3r.py", line 298, in check_response_errors
if 'Our systems have detected unusual traffic' in resp:
TypeError: argument of type 'int' is not iterable

Support punycode/IDN domains