ring04h/wydomain

to discover subdomains of your target domain

wydomain

To Discover Subdomains Of Your Target Domain

使用帮助

提示

记得每次运行前git pull一下，有空的话都会修bug.

chaxun.la 对请求频率过高的ip，要人工输入验证码，代码嵌入了[云速打码]自动识别验证码的功能，但是多人公用账号，导致我的账号被封禁了，所以临时关闭人机绕过功能，如果需要开启，你们可以注册使用自己账号。

https://github.com/ring04h/wydomain/blob/wydomain2/captcha.py
https://github.com/ring04h/wydomain/blob/wydomain2/utils/chaxunla.py#L41

库依赖安装

$ pip install -r requirements.txt	

1. 先使用字典穷举目标的子域名

$ python dnsburte.py -h
usage: dnsburte.py [-h] [-t] [-d] [-f] [-o]

wydomian v 2.0 to bruteforce subdomains of your target domain.

optional arguments:
  -h, --help      show this help message and exit
  -t , --thread   thread count
  -d , --domain   domain name
  -f , --file     subdomains dict file name
  -o , --out      result out file

字典名称	说明
default.csv	top 200 子域名字典
dnstop.csv	dnspod.com 官方提供的top 2000条子域名字典
wydomain.csv	wyodmian 1.0 的top 3000子域名字典 (非常高效)

wydomian 1.0 大字典
https://github.com/ring04h/wydomain/blob/master/domain_larger.csv

1.1 实际使用演示

子域名字典穷举结果保存在 result/aliyun.com 目录下的 dnsburte.json 文件。

$ python dnsburte.py -d aliyun.com -f dnspod.csv -o aliyun.log
2016-11-01 13:01:02,327 [INFO] starting bruteforce threading(16) : aliyun.com
2016-11-01 13:02:15,985 [INFO] dns bruteforce subdomains(51) successfully...
2016-11-01 15:03:43,367 [INFO] result save in : aliyun.log

2. 使用API查询目标的子域名

各个API查询的结果保存在 result/aliyun.com 目录下 对应的json文件中。

$ python wydomain.py -h
usage: wydomain.py [-h] [-d] [-o]

wydomain v 2.0 to discover subdomains of your target domain.

optional arguments:
  -h, --help      show this help message and exit
  -d , --domain   domain name
  -o , --out      result out file

3. 查看结果

domains.log 为最终的子域名结果集合。

阿里云 aliyun.com 子域名结果
https://github.com/ring04h/wydomain/tree/wydomain2/result/aliyun.com
https://github.com/ring04h/wydomain/blob/wydomain2/domains.log

微博 weibo.com 子域名结果
https://github.com/ring04h/wydomain/tree/wydomain2/result/weibo.com https://github.com/ring04h/wydomain/blob/wydomain2/weibo_domains.log

wydomain's Issues

kail 2.0 执行不成功

root@kali:~/wydomain# python wydomain.py baidu.com\

---

* Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/root/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/root/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

mysubdomain links_get代码中缺个while(True)

mysubdomain links_get代码中缺个while(True)
会导致在获取失败后，返回值为NoneType

sth error

环境：

Python 2.7.3
Linux pentest 3.14-kali1-686-pae #1 SMP Debian 3.14.5-1kali1 (2014-06-07) i686 GNU/Linux

报错信息：

--------------------------------------------------
* Starting fofa plugin search
--------------------------------------------------
--------------------------------------------------
* Starting process [****.com] partner domain
--------------------------------------------------
* Starting get ns record
--------------------------------------------------
* ns1.dnsv2.com in whitelist
* ns2.dnsv2.com in whitelist
--------------------------------------------------
--------------------------------------------------
* Starting bruteforce subdomain task
--------------------------------------------------
* running: dnsdict6 -4 -t 32 ****.com domain_default.csv
--------------------------------------------------
* Starting subdomain search task
--------------------------------------------------
--------------------------------------------------
* Starting process [602.com] partner domain
--------------------------------------------------
* Starting get ns record
--------------------------------------------------
* ns1.dnsv2.com in whitelist
* ns2.dnsv2.com in whitelist
--------------------------------------------------
--------------------------------------------------
* Starting bruteforce subdomain task
--------------------------------------------------
* running: dnsdict6 -4 -t 32 602.com domain_default.csv
--------------------------------------------------
* Starting subdomain search task
--------------------------------------------------
Traceback (most recent call last):
  File "wydomain.py", line 346, in <module>
    print start_wydomain(sys.argv[1])
  File "wydomain.py", line 178, in start_wydomain
    subdomains_result = wy_subdomain_run(pdomain)
  File "/root/wydomain/wysubdomain.py", line 56, in wy_subdomain_run
    my_subdomains = get_subdomain_run(domain)
  File "/root/wydomain/mysubdomain.py", line 200, in get_subdomain_run
    mydomains.extend(links_get(domain))
TypeError: 'NoneType' object is not iterable

bug:dns ns和zone数据获取不全

bug在dnsfunc.py的get_zone_record函数和get_ns_record函数。
get_zone_record 的54行，在for内循环中声明了变量a_record和cname_record，导致只能记录最后一次的循环解析，在循环超过两次的情况下会出现遗漏。
get_ns_record函数的204行，同样在for循环中声明了ns_record变量，导致只能记录一次。
例如get_ns_record("ce.cn")。

安装好了报错 TypeError: 'NoneType' object is not iterable

TypeError: 'NoneType' object is not iterable

安装phantomjs里少了一步进入文件夹.

安装phantomjs-2
在cp ./bin/phantomjs /usr/bin/前, 应cd phantomjs-1.9.8-linux-x86_64

ValueError: No JSON object could be decoded

---

* Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/root/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/root/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/lib/python2.7/json/init.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 383, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
几天前运行时非常正常，今天突然报错，求解

安装完成后运行报错 ValueError: No JSON object could be decoded

[root@VM_15_187_centos wydomain]# python wydomain.py 163.com 
--------------------------------------------------
* Starting fofa plugin search
--------------------------------------------------
Traceback (most recent call last):
  File "wydomain.py", line 346, in <module>
    print start_wydomain(sys.argv[1])
  File "wydomain.py", line 53, in start_wydomain
    fofa_result = start_fofa_plugin(domain)
  File "/usr/local/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
    get_partner_domain(domain)
  File "/usr/local/wydomain/fofaplugin.py", line 181, in get_partner_domain
    jobId = json.loads(taskinfo)['jobId']
  File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.6/json/decoder.py", line 338, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

几个月前运行正常，今天再次安装后运行报如上错误。

#12 和这个报错一样吗？
https://fofa.so/lab/addtask/?taskaction=alldomains&domain=
这个接口不可用了？脚本就不可用了？

首次运行报错

➜ wydomain git:(master) python wydomain.py xxx.com

* Starting fofa plugin search

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/Users/user/git/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/Users/user/git/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

http_request_get出错

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 243, in start_fofa_plugin
get_partner_domain(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 186, in get_partner_domain
partner_result = json.loads(http_request_get(domian_jobInfo_url)['html'])
TypeError: 'Response' object has no attribute 'getitem'

api查询和暴力破解整合到一起是不是更方便

结果整合问题

最近使用猪哥的工具 发现返回结果没有域名的标题啥的
自己写了个小脚本 希望猪哥参考添加进去
`import requests
import json
from bs4 import BeautifulSoup

domain = set()
jsonname = ['alexa.json', 'chaxunla.json', 'dnsburte.json', 'ilinks.json', 'netcraft.json', 'sitedossier.json'
, 'threatminer.json']
def do_json(name):
with open(name) as f:
data = json.load(f)
return data

def get_title(url):
try:
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 UBrowser/6.0.1471.914 Safari/537.36'}
url = 'http://'+url
print(url)
response = requests.get(url=url, headers=headers,timeout=5,verify=False)
response.encoding = response.apparent_encoding
response = BeautifulSoup(response.text,'lxml')
title = response.html.title.get_text()
except:
title = '空'
return title

for i in jsonname:
for url in do_json(name=i):
domain.add(url)

with open('result.txt','w',errors='ignore') as f:
for i in domain:
title = get_title(url=i)
print(title)
f.write(i+' '+title+'\n')

`

fofa模块貌似跪了

$ python wydomain.py baidu.com

• Starting fofa plugin search

---

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/X/X/X/wydomain/fofaplugin.py", line 244, in start_fofa_plugin
get_partner_domain(domain)
File "/X/X/X/wydomain/fofaplugin.py", line 181, in get_partner_domain
jobId = json.loads(taskinfo)['jobId']
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

新版本依旧报错

`
Traceback (most recent call last):
File "wydomain.py", line 149, in
run(args)
File "wydomain.py", line 89, in run
logging.info("chaxunla fetcher subdomains({0}) successfully...".format(len(result)))
TypeError: object of type 'NoneType' has no len()

`

获取 NS服务器记录 可能不完整

import dns.query
import dns.resolver

_res = dns.resolver.Resolver(filename='/etc/resolv.conf', configure=True)
ns = _res.query('zju.edu.cn', 'NS')
for i in ns.response.answer:
  print i

通过多次获取可以获取到不同的NS记录。

域名穷举结果为空

我用2个线程 domain_larger.csv字典去尝试穷举 youku.com的域名，发现穷举结果为空。麻烦帮忙看一下什么原因。

使用中的小问题

1 使用chaxun的api，有时候好有时候不好，没有超过额度，也是一天之内使用的cookie应该不会过期，('Connection aborted.', error(104, 'Connection reset by peer'))

2 我有一个列表批量查询的时候怎么做，或者我想要即使用字典又使用API能一个命令搞定么

3 看到 netcraft貌似最多只有500条记录，如果我想抓所有的.edu.cn的域名这500个明显是不够的，或者这种情况下怎么使用这个工具。或者是这个工具只是针对二级域名，对根域名不是很好。

4 使用过程中还提示
[INFO] Starting new HTTPS connection (1): www.threatminer.org
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning

python wydomain.py wooyun.org 第一次使用时有点问题 ubuntu 32位系统

root@VM-131-71-ubuntu:~/wydomain# python wydomain.py wooyun.org

* Starting fofa plugin search

* No parent domain

* Starting get ns record

• ns2.dnsv2.com in whitelist

* ns1.dnsv2.com in whitelist

---

* Starting bruteforce subdomain task

* running: dnsdict6 -4 -t 32 wooyun.org domain_default.csv

* Starting subdomain search task

Traceback (most recent call last):
File "wydomain.py", line 337, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 298, in start_wydomain
subdomains_result = wy_subdomain_run(domain)
File "/root/wydomain/wysubdomain.py", line 56, in wy_subdomain_run
my_subdomains = get_subdomain_run(domain)
File "/root/wydomain/mysubdomain.py", line 200, in get_subdomain_run
mydomains.extend(links_get(domain))
TypeError: 'NoneType' object is not iterabl

README.md中的wydomian.csv拼写错误

应该为wydomain.csv

出错。

---

Traceback (most recent call last):
File "wydomain.py", line 346, in
print start_wydomain(sys.argv[1])
File "wydomain.py", line 53, in start_wydomain
fofa_result = start_fofa_plugin(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 242, in start_fofa_plugin
get_partner_domain(domain)
File "/home/huang.jinbao/wydomain/fofaplugin.py", line 180, in get_partner_domain
domian_jobInfo_url = 'http://fofa.so/lab/gettask?jobId=%s&t=%s' % (jobId, int(time.time()))
NameError: global name 'time' is not defined

api.ysdm.net get verify_code failed

[INFO] Starting new HTTP connection (1): api.ysdm.net
('get verify_code():', None)

运行有bug

File "wydomain/wydomain.py", line 41
os.makedirs(_cache_path, 0777)
^
SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
(venv) (base)

泛域名解析不稳定，会有解析不到的状况

方便调试，加了两句打印代码：
前后发起多次请求，发现有时泛域名没有解析到IP（泛域名确实存在）

# 能解析泛域名
2016-12-29 01:31:11,904 [INFO] starting bruteforce threading(16) : tjut.edu.cn
-- ehost ['wyspider0.tjut.edu.cn', 'wyspider1.tjut.edu.cn', 'wyspider2.tjut.edu.cn']
-- esets: [u'202.113.64.2', u'202.113.64.2']
-- extensive [u'202.113.64.2', u'202.113.64.2']

# 未解析
2016-12-29 01:32:16,314 [INFO] starting bruteforce threading(16) : tjut.edu.cn
-- ehost ['wyspider0.tjut.edu.cn', 'wyspider1.tjut.edu.cn', 'wyspider2.tjut.edu.cn']
-- esets: []
-- extensive []