ctf-missfeng / bayonet Goto Github PK
View Code? Open in Web Editor NEWbayonet是一款src资产管理系统,从子域名、端口服务、漏洞、爬虫等一体化的资产管理系统
bayonet是一款src资产管理系统,从子域名、端口服务、漏洞、爬虫等一体化的资产管理系统
执行docker run -itd --name bayonet -p 5000:80 missfeng/bayonet:v1.1
后继报错如下
docker: Error response from daemon: driver failed programming external connectivity on endpoint bayonet (0d2c5853b4bc91e0e2fae7b5c4e6c493915be166a0b421c308c4517c80cb2bb5): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 5000 -j DNAT --to-destination 172.17.0.2:80 ! -i docker0: iptables: No chain/target/match by that name.
(exit status 1)).
环境:使用docker环境,已经替换了最新的代码文件。
添加主域名以后,扫描完毕以后。tools.log就不再滚动了,端口扫描等动作也都不执行。
已经添加了shodan 的api,fofa的,zoomeye的都添加了。
tools.log最后的日志就是
02:13:19,629 [INFOR] resolve:170 - 完成异步查询子域的A记录
02:13:19,651 [INFOR] dbexport:65 - 开始进行子域名入库
02:13:20,232 [INFOR] dbexport:77 - 子域名入库完成
02:13:20,242 [INFOR] oneforall:206 - 结束运行OneForAll
w13scan.log中的日志一直在提示
不存在results目录
后续所有的动作都未执行。
请问这个应该怎么处理。
用两台机器分别拉取了容器下来,配置分别为阿里云轻量服务器ubuntu18.04.4和腾讯云服务器ubuntu18.04.1,
docker run -itd --name bayonet -p 5000:5000 missfeng/bayonet:v1.2
我是讲端口映射放到5000的。
在容器内部执行curl http://127.0.0.1
能够正常返回页面内容,但是在宿主机执行curl http://127.0.0.1:5000
返回的则是Empty reply from server
在创建任务的时候,增加一栏用用于填写黑名单来排除不在奖励范围内的域名。
如:*.test.qq.com ,排除test.qq.com下的子域名。对于一些资产比较多src还是可以减少挺多不在范围内的域名,不用再去扫描这些域名的端口和目录等,也有助于提升速度
尝试国内、国外的pip源安装后扔不能解决问题。
使用iipip-ipdb 1.3.2模块即可,下载地址:https://pypi.org/project/ipip-ipdb/#files
填坑很久...
XD
1.添加域名-->2.获取子域名(添加企查查、备案信息等)[支持自行导入域名list]-->3.子域名解析ip的c段所有ip1-65535端口-->4.实时监控子域名列表,定期获取子域名。
结果支持导出功能
web日志:
[2020-03-25 14:10:54,477] ERROR in app: Exception on /api/src/subdomain [GET]
Traceback (most recent call last):
File "/usr/local/bin/python3/lib/python3.8/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/bin/python3/lib/python3.8/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/bin/python3/lib/python3.8/site-packages/flask_restful/__init__.py", line 468, in wrapper
resp = resource(*args, **kwargs)
File "/usr/local/bin/python3/lib/python3.8/site-packages/flask/views.py", line 89, in view
return self.dispatch_request(*args, **kwargs)
File "/usr/local/bin/python3/lib/python3.8/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
resp = meth(*args, **kwargs)
File "/root/bayonet/web/route/src/api.py", line 271, in get
data1['loudong_count'] = len(i.src_vulnerabilitie)
AttributeError: 'SrcSubDomain' object has no attribute 'src_vulnerabilitie'
222.209.84.125 - - [25/Mar/2020 14:10:54] "GET /api/src/subdomain?page=1&limit=20&searchParams=%7B%22subdomain%22%3A%22%22%2C%22subdomain_ip%22%3A%22%22%7D HTTP/1.1" 500 -
师傅, 请问下关于数据库连接字符:
SQLALCHEMY_DATABASE_URI = 'postgresql://postgres:qazxsw@[email protected]/bayonet'
其中postgresql postgres qazxsw@123 bayonet这几个参数各自代表什么?
新手查了半天资料无获,谢谢~
一段时间使用之后发现那几个python进程偶尔会一些原因退出,或者假死
建议加一个心跳检测之类的模块,减少一些命令行运维的操作
您好,我使用docker的方式搭建起来环境,添加域名扫描。
添加之后,过一会可以看到域名的扫描结果,但是端口和漏洞数据没有。
请问,添加任务之后是会进行自动的端口 & 漏洞扫描吗?
个人感觉端口数据应该是可以出来的。
另外,是否能给个任务进度提示,我也不知道是不是任务没有跑完?反正等了蛮久的
实际使用中,有时候我会手动想让OneForALL 某些域名,urlscan对某些端口再跑一次。我会直接修改数据库的flag, 这个时候需要手动重启Run.py,因为缓存问题。所以,建议在扫描之前的读取数据库的时候刷新一下缓存。比如下面的地方,加一行代码commit一下。
比如:
def ReadPort():
# 刷新缓存
DB.session.commit()
在w13scan上挂着xray的上游代理。这样给数据给w13scan时候,xray也同时能够接受到。
urlscan/Run.py
ipdata.ipdb IP数据库不存在
请问能否手动批量加入子域名列表
crawlergo有一个可以设置代理的选项--push-to-proxy
建议可以直接用这个代理上xray,不然爬取的流量还要用python再放一遍
OneForAll的结果可以直接取one.datas,不用写得这么复杂的哦。
bayonet/tools/oneforall/Run.py
Lines 32 to 43 in cd191cd
[{"id": null, "url": "http://freebuf.com:80", "subdomain": "freebuf.com", "port": 80, "ips": "'182.254.150.199'", "status": 200, "reason": "OK", "valid": 1, "new": 0, "title": "FreeBuf互联网安全新媒体平台", "banner": "'Server': 'Apache/2.2.21', 'Via': None, 'X-Powered-By': None"},{...}]
另外现在OneForAll支持传path参数了,对OneForAll有其他问题欢迎交流:P
We noticed that you used the API of one of Spyse's products most likely it was Findsubdomains, Cert DB or DNS table. We want to notify you about the release of a new and improved API key which you can find in your personal account at spyse.com.
Unfortunately, we should stop supporting the old API a few weeks earlier, starting from Monday. Thanks for being with us.
我已经本地搭建成功,但是不能设置外网访问。
对5000端口已经防火墙开放成功。
想问一下大佬,如何设置。
不能准确查询到域名对应的正确的ip,和dig出来的ip不一样,我也对同一个域名用oneforall跑,得到的域名也不一致
项目里面的crawlergo是二进制文件,你应该下错了crawlergo
https://github.com/0Kee-Team/crawlergo/releases
需要在这个项目里面下载linux版的crawlergo。
tools/portscan/socket_scan.py", line 154, in socket_main
asyncio.run(action(ip, threads, PortScan.async_scan_timeout))
AttributeError: module 'asyncio' has no attribute 'run'
17:18:40,606 [ALERT] Run:63 - 端口入库SQL错误:(psycopg2.errors.ForeignKeyViolation) insert or update on table "src_ports" violates foreign key constraint "src_ports_subdomain_fkey"
DETAIL: Key (subdomain)=(xxx.com) is not present in table "src_subdomain".
[SQL: INSERT INTO src_ports (subdomain_ip, subdomain, port, service, product, version, flag, brute, port_time) VALUES (%(subdomain_ip)s, %(subdomain)s, %(port)s, %(service)s, %(product)s, %(version)s, %(flag)s, %(brute)s, %(port_time)s) RETURNING src_ports.id]
[parameters: {'subdomain_ip': '110.110.110.110', 'subdomain': 'xxx.com', 'port': 80, 'service': 'http', 'product': '', 'version': '', 'flag': False, 'brute': False, 'port_time': '2020-03-24 17:18:40'}]
(Background on this error at: http://sqlalche.me/e/gkpj)
Traceback (most recent call last):
File "run_portscan.py", line 4, in
tools.portscan.Run.port_main()
File "/root/bayonet/tools/portscan/Run.py", line 91, in port_main
WriteSubDomain(results)
File "/root/bayonet/tools/portscan/Run.py", line 40, in WriteSubDomain
sql_subdomain = SrcSubDomain.query.filter(SrcSubDomain.subdomain_ip == results.subdomain_ip).all()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 282, in get
return self.impl.get(instance_state(instance), dict_)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 705, in get
value = state._load_expired(state, passive)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/state.py", line 660, in _load_expired
self.manager.deferred_scalar_loader(self, toload)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 987, in load_scalar_attributes
raise orm_exc.ObjectDeletedError(state)
sqlalchemy.orm.exc.ObjectDeletedError: Instance '<SrcSubDomain at 0x7fd6ae270370>' has been deleted, or its row is otherwise not present.
一键搭建,会方便一些,而且不用担心环境报错
SQLALCHEMY_DATABASE_URI = 'postgresql://bayonet:aCymcibCp8RN@localhost:5432/bayonet'
我的数据库配置连接没问题呀!但是就是在插入数据库的时候报错!我数据库版本为11版本
可以增加扫ip段,敏感信息搜集模块,git、bing、google、百度等
新版本有些什么变化呢??
root@e41801b31c12:/etc/init.d# ./postgresql start
第一版:
将子域名扫描、爬虫、漏洞扫描,分布布置到不同服务器进行工作,然后把数据统一返回web界面。
第二版:
然后在考虑,多节点形式,比如同时用3个w13scan扫描漏洞,然后在推送到web界面。
开源聚合大卡车啊!已经推荐到到我的项目!
感谢开发,感谢开源!
开发不易,还没有阅读源码的,想问一下是不是有统一的proxy/proxy池支持、设置或管理,还是需要单独去每个工具里面设置,或者是在主机上设置所有的出口流量.因为现在遍地WAF...估计没怎么扫描IP就挂了.
已经安装了PortScan,依然显示这个。
把自己的shodan apikey输入进入了,但是显示错误
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.