Problem with DNSdumpster

Problem with DNSdumpster

[hoseta]

If any1 will have the same problem just follow my instructions

1. Install beautifulsoup4

pip install beautifulsoup4

2. open sublist3r.py file in any editor.

and import bs4

add this line on the top of file
from bs4 import BeautifulSoup

go to line 640 and replace function get_csrftoken

def get_csrftoken(self, resp)

to following

def get_csrftoken(self, resp):

        soup = BeautifulSoup(resp,features="html.parser")

        token = soup.find('input', {'name': 'csrfmiddlewaretoken'}).get('value')

        return token.strip()

[aboul3la]

Hi @hoseta @AdriiiPRodri, thanks for reporting this. This issue was caused due to DNSDumpster was making some modifications in their website HTML structure.

I just pushed a fix for this issue in the latest commit 5d33d9f, but I will try to apply a better fix using HTML parser better than relying on constant regex values.

[aboul3la]

@D4master, I'm gonna add a check for that in the code, so if the website doesn't respond with "200 OK" then it will ignore it and temporary stop the enumeration from that website.

@aboul3la

i get this too in DNSdumpster

Process DNSdumpster-8:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/share/Sublist3r/sublist3r.py", line 565, in run
    domain_list = self.enumerate()
  File "/usr/share/Sublist3r/sublist3r.py", line 625, in enumerate
    token = self.get_csrftoken(resp)
  File "/usr/share/Sublist3r/sublist3r.py", line 620, in get_csrftoken
    token = csrf_regex.findall(resp)[0]
IndexError: list index out of range

i suggest add module that handle connections error

[aboul3la]

Looks like the problem with the DNSdumpster website itself. As the website "https://dnsdumpster.com/" is down right now and it's not working properly. So the issue should be fixed when the website back to the service again.

[acaetano]

I'm getting a different error and dnsdumpster is online.

[-] Searching now in PassiveDNS..
HTTPSConnectionPool(host='dnsdumpster.com', port=443): Read timed out. (read timeout=25)
Process DNSdumpster-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "./sublist3r.py", line 565, in run
domain_list = self.enumerate()
File "./sublist3r.py", line 628, in enumerate
self.extract_domains(post_resp)
File "./sublist3r.py", line 641, in extract_domains
results_tbl = tbl_regex.findall(resp)[0]
TypeError: expected string or buffer

Maybe they changed the output format/syntax?

[gloomy-ghost]

HTTPSConnectionPool(host='dnsdumpster.com', port=443): Read timed out. (read timeout=10)
Exception in thread Thread-7:
Traceback (most recent call last):
  File "%Python%\Python35\lib\threading.py", line 914, in _bootstrap_inner
    self.run()
  File "%Sublist3r%\sublist3r.py", line 258, in run
    domain_list = self.enumerate()
  File "%Sublist3r%\sublist3r.py", line 635, in enumerate
    self.extract_domains(post_resp)
  File "%Sublist3r%\sublist3r.py", line 647, in extract_domains
    results_tbl = tbl_regex.findall(resp)[0]
TypeError: expected string or bytes-like object

[aboul3la]

This bug should be fixed now.

[AdriiiPRodri]

Hi everyone,

This problem is still present, the @hoseta fix should be merge in the main branch. It improves the results obtained and does not break the compatibility with Python2.

Before

[-] Total Unique Subdomains Found: 9

After

[-] Total Unique Subdomains Found: 10

[fitzpr]

I'm still getting the error from DNSDumpster. I changed the CSRF code with @aboul3la fix but nothing changed? Maybe I missed something.

Process DNSdumpster-80: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/home/Sublist3r/sublist3r.py", line 271, in run domain_list = self.enumerate() File "/home/Sublist3r/sublist3r.py", line 654, in enumerate self.extract_domains(post_resp) File "/home/Sublist3r/sublist3r.py", line 666, in extract_domains results_tbl = tbl_regex.findall(resp)[0] TypeError: expected string or buffer