evilsocket/xray

XRay is a tool for recon, mapping and OSINT gathering from public networks.

XRAY

XRay is a tool for network OSINT gathering, its goal is to make some of the initial tasks of information gathering and network mapping automatic.

How Does it Work?

XRay is a very simple tool, it works this way:

1. It'll bruteforce subdomains using a wordlist and DNS requests.
2. For every subdomain/ip found, it'll use Shodan to gather open ports and other intel.
3. If a ViewDNS API key is provided, for every subdomain historical data will be collected.
4. For every unique ip address, and for every open port, it'll launch specific banner grabbers and info collectors.
5. Eventually the data is presented to the user on the web ui.

Grabbers and Collectors

• HTTP Server, X-Powered-By and Location headers.
• HTTP and HTTPS robots.txt disallowed entries.
• HTTPS certificates chain ( with recursive subdomain grabbing from CN and Alt Names ).
• HTML title tag.
• DNS version.bind. and hostname.bind. records.
• MySQL, SMTP, FTP, SSH, POP and IRC banners.

Notes

Shodan API Key

The shodan.io API key parameter ( -shodan-key KEY ) is optional, however if not specified, no service fingerprinting will be performed and a lot less information will be shown (basically it just gonna be DNS subdomain enumeration).

ViewDNS API Key

If a ViewDNS API key parameter ( -viewdns-key KEY ) is passed, domain historical data will also be retrieved.

Anonymity and Legal Issues

The software will rely on your main DNS resolver in order to enumerate subdomains, also, several connections might be directly established from your host to the computers of the network you're scanning in order to grab banners from open ports. Technically, you're just connecting to public addresses with open ports (and there's no port scanning involved, as such information is grabbed indirectly using Shodan API), but you know, someone might not like such behaviour.

If I were you, I'd find a way to proxify the whole process ... #justsaying

Building a Docker image

To build a Docker image with the latest version of XRay:

git clone https://github.com/evilsocket/xray.git
cd xray
docker build -t xraydocker .

Once built, XRay can be started within a Docker container using the following:

docker run --rm -it -p 8080:8080 xraydocker xray -address 0.0.0.0 -shodan-key shodan_key_here -domain example.com 

Manual Compilation

Make sure you are using Go >= 1.7, that your installation is working properly, that you have set the $GOPATH variable and you have appended $GOPATH/bin to your $PATH.

Then:

go get github.com/evilsocket/xray
cd $GOPATH/src/github.com/evilsocket/xray/
make

You'll find the executable in the build folder.

Usage

Usage: xray -shodan-key YOUR_SHODAN_API_KEY -domain TARGET_DOMAIN
Options:
  -address string
        IP address to bind the web ui server to. (default "127.0.0.1")
  -consumers int
        Number of concurrent consumers to use for subdomain enumeration. (default 16)
  -domain string
        Base domain to start enumeration from.
  -port int
        TCP port to bind the web ui server to. (default 8080)
  -preserve-domain
        Do not remove subdomain from the provided domain name.
  -session string
        Session file name. (default "<domain-name>-xray-session.json")
  -shodan-key string
        Shodan API key.
  -viewdns-key string
        ViewDNS API key.
  -wordlist string
        Wordlist file to use for enumeration. (default "wordlists/default.lst")

Example:

# xray -shodan-key yadayadayadapicaboo... -viewdns-key foobarsomethingsomething... -domain fbi.gov

____  ___
\   \/  /
 \     RAY v 1.0.0b
 /    by Simone 'evilsocket' Margaritelli
/___/\  \
      \_/

@ Saving session to fbi.gov-xray-session.json
@ Web UI running on http://127.0.0.1:8080/

License

XRay was made with ♥ by Simone Margaritelli and it's released under the GPL 3 license.

The files in the wordlists folder have been taken from various open source tools accross several weeks and I don't remember all of them. If you find the wordlist of your project here and want to be mentioned, feel free to open an issue or send a pull request.

xray's Issues

.co.za not supported?

When using .co.za domain, it thinks za is the TLD and co is the domain...

how to set up VPN on a router?

where to get data?
1 Server
2 username
3 Password

Contribute Biggest subdomain Wordlist

Hey, I am very much fascinated by your tool. As mentioned by you I would like to contribute the largest wordlist for sub domains.With 881462 common subdomains
400_blacklist.txt

[Feature Requests] Screenshot & S3 Buckets

Hey Dude,

Can you add a screenshot system for when it hits pages that are responding with http so it can help work out what the pages are?

also on s3 buckets like this one..

can you have it so it grabs the bucketname?

view-source:http://js.mapbox.com/

shows

which would be very handy to know it's on a s3 bucket and then i can check permissions of the bucket :)

crawling subject alternative names

Nice tool! What about reading the hostnames in the X509v3 Subject Alternative Name(s) for additional in-domain discoveries?

Installation issue

Hello @evilsocket

I'm trying to set up the Xray and getting errors like below error.

via Docker image

➜  xray git:(master) docker build -t xraydocker .

Sending build context to Docker daemon   8.92MB
Step 1/7 : FROM golang:alpine
 ---> 3024b4e742b0
Step 2/7 : RUN apk add --no-cache git make
 ---> Using cache
 ---> 30023051ed15
Step 3/7 : RUN go get github.com/evilsocket/xray &&     cd $GOPATH/src/github.com/evilsocket/xray/ &&     make get_glide &&     make install_dependencies &&     go get -u github.com/jteeuwen/go-bindata/... &&     make build
 ---> Running in 1104275da16e
package github.com/moul/http2curl: code in directory /go/src/github.com/moul/http2curl expects import "moul.io/http2curl"
The command '/bin/sh -c go get github.com/evilsocket/xray &&     cd $GOPATH/src/github.com/evilsocket/xray/ &&     make get_glide &&     make install_dependencies &&     go get -u github.com/jteeuwen/go-bindata/... &&     make build' returned a non-zero code: 1

via manual

➜  pentest go get github.com/evilsocket/xray

package github.com/moul/http2curl: code in directory /Users/REDACTED/go/src/github.com/moul/http2curl expects import "moul.io/http2curl"

undefined: AssetDir / AssetInfo

# make build
go-bindata -o cmd/xray/ui.go -pkg main ui
# command-line-arguments
cmd/xray/bfs.go:54: undefined: AssetDir
cmd/xray/bfs.go:54: undefined: AssetInfo
Makefile:35: recipe for target 'build' failed
make: *** [build] Error 2

I've set $GOPATH and added $GOPATH/bin to $PATH

any idea?

Issue in Installation guide

Part: Builiding a Docker images.

Current for clonning repository following command is listed.

git clone github.com/evilsocket/xray

Above command should be changed to.

git clone https://github.com/evilsocket/xray.git

xray docker build installation issue.

Hi Team,

I am trying to setting up the xray in my mac and getting below error, please help me with way out.

docker build -t xraydocker .
Sending build context to Docker daemon 8.893MB
Step 1/7 : FROM golang:alpine
---> 05fe62871090
Step 2/7 : RUN apk add --no-cache git make
---> Using cache
---> 160439d01c8d
Step 3/7 : RUN go get github.com/evilsocket/xray && cd $GOPATH/src/github.com/evilsocket/xray/ && make get_glide && make install_dependencies && go get -u github.com/jteeuwen/go-bindata/... && make build
---> Running in 640dad4d4888

github.com/evilsocket/xray

src/github.com/evilsocket/xray/target.go:116:47: not enough arguments in call to t.ctx.Shodan.GetServicesForHost
have (string, *shodan.HostServicesOptions)
want (context.Context, string, *shodan.HostServicesOptions)
The command '/bin/sh -c go get github.com/evilsocket/xray && cd $GOPATH/src/github.com/evilsocket/xray/ && make get_glide && make install_dependencies && go get -u github.com/jteeuwen/go-bindata/... && make build' returned a non-zero code: 2

installation issue

First i had some problem to installing glide and go-bindata , i think you should add apt-get install go-bindata on dependencies .

and

curl -L $(curl -s https://api.github.com/repos/Masterminds/glide/releases/latest | grep 'browser_' | cut -d\" -f4 | grep 'linux-amd64.tar.gz') > glide.tar.gz &&
tar --strip-components=1 -C /usr/bin -xzf glide.tar.gz linux-amd64/glide &&
rm glide.tar.gz

Helped to fix glide issue ,

and now

root@xray:~/work/src/github.com/evilsocket/xray# make build
go-bindata -o cmd/xray/ui.go -pkg main ui
# command-line-arguments
cmd/xray/bfs.go:54: undefined: AssetDir
cmd/xray/bfs.go:54: undefined: AssetInfo
Makefile:35: recipe for target 'build' failed
make: *** [build] Error 2

Makefile:9: recipe for target 'get_glide' failed

Hey thanks for making and sharing it, while installing and following installation step i encountered with this error

root@localhost:~/xray# make get_glide
go get -u -v github.com/Masterminds/glide
github.com/Masterminds/glide (download)
github.com/Masterminds/glide/path
github.com/Masterminds/glide/cache
github.com/Masterminds/glide/mirrors
github.com/Masterminds/glide/cfg
github.com/Masterminds/glide/godep
github.com/Masterminds/glide/dependency
github.com/Masterminds/glide/gb
github.com/Masterminds/glide/gpm
github.com/Masterminds/glide/gom
github.com/Masterminds/glide/importer
github.com/Masterminds/glide/repo
github.com/Masterminds/glide/tree
github.com/Masterminds/glide/action
github.com/Masterminds/glide
/bin/glide install
make: /bin/glide: Command not found
Makefile:9: recipe for target 'get_glide' failed
make: *** [get_glide] Error 127

panic: open wordlists/default.lst: no such file or directory

Runnning from /build:

./xray -shodan-key XXXXXXXX -viewdns-key XXXXXXXX -domain xxxxxxxx.com

---

\ / /
\ RAY v 1.0.0b
/ by Simone 'evilsocket' Margaritelli
/___/\
_/

Warning: You have 0 query credits.
@ Saving session to hackerone.com-xray-session.json
panic: open wordlists/default.lst: no such file or directory

/src/github.com/evilsocket/xray/wordlists# ls
all.lst default.lst top1mil-20000.lst top1mil-5000.lst top1mil.ls

wordlists/default.lst: no such file or directory

____  ___
\   \/  /
 \     RAY v 1.0.0b
 /    by Simone 'evilsocket' Margaritelli
/___/\  \
      \_/

@ Saving session to mydomain.co-xray-session.json
panic: open wordlists/default.lst: no such file or directory

goroutine 1 [running]:
main.main()
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:173 +0xc9e
root@xray:~/work/src/github.com/evilsocket/xray/build# cd ..
root@xray:~/work/src/github.com/evilsocket/xray# ls wordlists/
all.lst  default.lst  top1mil-20000.lst  top1mil-5000.lst  top1mil.lst
root@xray:~/work/src/github.com/evilsocket/xray# 

Go >= 1.7 required....

If you got a package context: unrecognized import path "context", maybe you have an older Go version. Check with "go version": you need almost 1.7...

i got warning from my VPS

We've detected an outgoing Denial of Service attack (http://do.co/21Y1Gc1) originating from your Droplet. Specifically, we have detected inbound traffic exceeding 0.01 Mb/s and outbound traffic exceeding 781.34 Mb/s. Due to the traffic’s harmful nature, your Droplet was taken offline; this means it is not connected to the internet and all hosted sites and services are unreachable. We know that this action is disruptive, but it’s necessary to protect you, our network, and the target of your Droplet’s attack.

am i shouldn't run it on vps ?

Invalid memory

$ ./xray -shodan-key GxoWeVf2CYZuiqEbaeoW6K4hfvIfxR9M -domain facebook.com -wordlist /home/kali/Programs/xray/wordlists/default.lst

---

\ / /
\ RAY v 1.0.0b
/ by Simone 'evilsocket' Margaritelli
/___/\
_/

@ Restoring session from facebook.com-xray-session.json ...
@ Loaded 23 entries from session file.
Warning: You have 0 query credits.
@ Restoring DNS bruteforcing from 3.59%
@ Web UI running on http://127.0.0.1:8080/

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x6ae290]

goroutine 360 [running]:
github.com/evilsocket/xray.(*Target).scanDomainAsync.func1(0xc0000b5b90, {0xc0005a00a0, 0x10})
/home/kali/Programs/xray/target.go:79 +0x90
created by github.com/evilsocket/xray.(*Target).scanDomainAsync
/home/kali/Programs/xray/target.go:76 +0x9a

PassiveTotal Support for Subdomain/alternate domain Enumeration

Leveraging an API key for the PassiveTotal (passivetotal.org) service would be super neat for grabbing more subdomains and related domains.

[FEATURE REQUEST]

Can you obtain more domain info via the following
Google transparency report
Censys
virtualtotal
netcraft
passive total

like aquatone does :) https://github.com/michenriksen/aquatone/tree/master/lib/aquatone/collectors

Error while installing, docker and manual installation

System:

Linux test-08 4.19.0-kali4-amd64 #1 SMP Debian 4.19.28-2kali1 (2019-03-18) x86_64 GNU/Linux
go version go1.12.7 linux/amd64

Error on docker build:

docker build -t xraydocker .
Sending build context to Docker daemon   8.92MB
Step 1/7 : FROM golang:alpine
 ---> 6b21b4c6e7a3
Step 2/7 : RUN apk add --no-cache git make
 ---> Using cache
 ---> dfcfbc9b6dea
Step 3/7 : RUN go get github.com/evilsocket/xray &&     cd $GOPATH/src/github.com/evilsocket/xray/ &&     make get_glide &&     make install_dependencies &&     go get -u github.com/jteeuwen/go-bindata/... &&     make build
 ---> Running in 7669f48b71f4
make: *** No rule to make target 'get_glide'.  Stop.
The command '/bin/sh -c go get github.com/evilsocket/xray &&     cd $GOPATH/src/github.com/evilsocket/xray/ &&     make get_glide &&     make install_dependencies &&     go get -u github.com/jteeuwen/go-bindata/... &&     make build' returned a non-zero code: 2

Error on manual installation:

~/go/src/github.com/evilsocket/xray# make
dep ensure
make: dep: Command not found
make: *** [Makefile:15: deps] Error 127

Executable issue

[INFO] Replacing existing vendor dependencies
root@r00t:/n00b/go/src/github.com/evilsocket/xray# make build
go-bindata -o cmd/xray/ui.go -pkg main ui
make: go-bindata: Command not found
Makefile:30: recipe for target 'static' failed
make: *** [static] Error 127
root@r00t:/n00b/go/src/github.com/evilsocket/xray# go get -u github.com/jteeuwen/go-bindata/...
root@r00t:~/n00b/go/src/github.com/evilsocket/xray# ls
cmd glide.yaml line_reader.go pool.go ui
dns_grabber.go grabber.go machine.go README.md vendor
Dockerfile http_grabber.go Makefile session.go viewdns.go
glide.lock LICENSE mysql_grabber.go target.go wordlists

i don't see any build folder :/

Can not build docker

When trying to build docker container, the build process fails with

docker build -t xraydocker .

Sending build context to Docker daemon  8.898MB
Step 1/7 : FROM golang:alpine
alpine: Pulling from library/golang
ff3a5c916c92: Already exists 
f32d2ea73378: Pull complete 
c6678747892c: Pull complete 
16b5f22d8b23: Pull complete 
Digest: sha256:356aea725be911d52e0f2f0344a17ac3d97c54c74d50b8561f58eae6cc0871bf
Status: Downloaded newer image for golang:alpine
 ---> 52d894fca6d4
Step 2/7 : RUN apk add --no-cache git make
 ---> Running in 1b2b08f249a0
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/6) Installing libssh2 (1.8.0-r2)
(2/6) Installing libcurl (7.59.0-r0)
(3/6) Installing expat (2.2.5-r0)
(4/6) Installing pcre2 (10.30-r0)
(5/6) Installing git (2.15.0-r1)
(6/6) Installing make (4.2.1-r0)
Executing busybox-1.27.2-r7.trigger
OK: 19 MiB in 18 packages
Removing intermediate container 1b2b08f249a0
 ---> 78904c9a833a
Step 3/7 : RUN go get github.com/evilsocket/xray &&     cd $GOPATH/src/github.com/evilsocket/xray/ &&     make get_glide &&     make install_dependencies &&     go get -u github.com/jteeuwen/go-bindata/... &&     make build
 ---> Running in 20eba2238519
# github.com/evilsocket/xray
src/github.com/evilsocket/xray/target.go:116:47: not enough arguments in call to t.ctx.Shodan.GetServicesForHost
	have (string, *shodan.HostServicesOptions)
	want (context.Context, string, *shodan.HostServicesOptions)
The command '/bin/sh -c go get github.com/evilsocket/xray &&     cd $GOPATH/src/github.com/evilsocket/xray/ &&     make get_glide &&     make install_dependencies &&     go get -u github.com/jteeuwen/go-bindata/... &&     make build' returned a non-zero 

Im guessing there were some API changes

please make output human readable

please make human readable for result ... thanks ...
in this picture i just replace ',' with newline '\n'

Error on install

.git/.svn/.hg... etc... enumeration

Enumerating .git/.svn/.hg and other cruft occasionally left on servers is often useful. We have a partial list of files here: https://github.com/HarryR/httphose/blob/master/httphose/common.txt that could be fun to enumerate to find more out about the server.

Feature request: export options?

Hey evilsocket,

Finally got to use xray today and it's GREAT! Thanks much for your efforts in making this tool.

It would be nice to be able to have some auto-export functionality to dump results into html, csv, txt, pdf, doc, etc. Just a thought.

Keep up the great work!

Brian

Make Shodan key static

Hey, nice work, and it's just enhancement suggestion as we know Shodan key is imp part of the tool, so is it good to make it static and save it in some file and use from it each time, instead of providing each time with argument.

im getting this error after xray finished running and saving session file

so i wanted to reload session and continue analysis of the target but i get this error

All tasks completed, press Ctrl-C to quit.
panic: json: unsupported value: +Inf

goroutine 43 [running]:
github.com/evilsocket/xray.(*Session).Flush(0xc00000e240, 0xc0002c8000)
/root/go/src/github.com/evilsocket/xray/session.go:78 +0xc7
github.com/evilsocket/xray.(*Pool).FlushSession(0xc0002c0840, 0xc0002c8000)
/root/go/src/github.com/evilsocket/xray/pool.go:55 +0x73
main.OnResult(0xaf4300, 0xc000331590)
/root/go/src/github.com/evilsocket/xray/cmd/xray/main.go:80 +0x238
github.com/evilsocket/xray.(*Machine).outputConsumer(0xc0002c8000)
/root/go/src/github.com/evilsocket/xray/machine.go:125 +0x61
created by github.com/evilsocket/xray.(*Machine).Start
/root/go/src/github.com/evilsocket/xray/machine.go:142 +0x88

install error

after running
make get_glide
make install_dependencies
i run into the following error on make build

gofmt -s -w .
vendor/golang.org/x/net/ipv4/batch.go:59:14: expected type, found '='
vendor/golang.org/x/net/ipv4/batch.go:78:2: expected declaration, found 'if'
vendor/golang.org/x/net/ipv6/batch.go:56:14: expected type, found '='
vendor/golang.org/x/net/ipv6/batch.go:69:2: expected declaration, found 'if'
Makefile:8: recipe for target 'format' failed
make: *** [format] Error 2

Can't bind to public IP

All ports are open on the machine, but only localhost is working, when try to bind public ip using -address i see this, i tried to change ports as well, for all port get same error!

panic: listen tcp public_ip:8080: bind: cannot assign requested address

goroutine 18 [running]:
main.main.func2()
        /root/go/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x23a
created by main.main
        /root/go/src/github.com/evilsocket/xray/cmd/xray/main.go:156 +0xc53

Web ui/output file

Hi evilsocket,

Again thanks for this awesome tool.

I have some problem regarding the web ui, i cannot access it even it say that it is running on xxxxxxx:8080 when access it i got site cannot be reached

And when i cancel the process, can i still view the json file ?

Thank you !

make issue

When I run to build the docker way it says there is no install candidate for get_glide. There is no get_glide objective in the make file so when i get to step three when building it says failed. I know you need to pull glide from github but I'm not familiar enough with go to fix myself. I'm also not familiar enough with Docker but i just seems like somewhere you need to make sure the Dockerfile or Makefile drags down glide because it appears to be missing.

go-shodan fix

Fetch go-shodan.v1 or v2 instead of go-shodan
go get gopkg.in/ns3777k/go-shodan.v2/shodan

Change imports in target.go and context.go to
import ( "gopkg.in/ns3777k/go-shodan.v2" "strings" "sync" )

virustotal

Hi!

I've just tried it and it seems very powerful! I usually check a domain name on virustotal.com/domain information as an integration of a subdomains enumeration activity (my little bash script: theMiddleBlue/DNSenum).

I'm not sure if virustotal accepts this... but usually, I get a list of subdomains from https://www.virustotal.com/it/domain/${DOMAIN_NAME}/information/ with something like this:

curl -s \
-c vtcookie.txt \
-b vtcookie.txt \
-A "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" \
"https://www.virustotal.com/en-gb/domain/example.com/information/"

curl -s \
-c vtcookie.txt \
-b vtcookie.txt \
-A "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" \
"https://www.virustotal.com/en-gb/domain/example.com/information/" | egrep "\<a target\=.\_blank. href\=..en\-gb.domain" | awk 'BEGIN{FS="/"}{print $4}'

the output is:

ns1.example.com
fake6.example.com
fake5.example.com
fake4.example.com
fake3.example.com
fake2.example.com
fake1.example.com
fake7.example.com
mail2.example.com
www.example.com

Hope this can be useful 😄

Xray on OpenBSD

I am creating the port of xray for OpenBSD and so far I am getting this:

# xray -wordlist /home/gonzalo/xray/wordlists -shodan-key LpJVd71839173914JhQ6 -domain google.com  
____  ___
\   \/  /
 \     RAY v 1.0.0b
 /    by Simone 'evilsocket' Margaritelli
/___/\  \
      \_/

Warning: You have 0 query credits.
@ Saving session to google.com-xray-session.json
@ Web UI running on http://127.0.0.1:8080/


All tasks completed, press Ctrl-C to quit.
panic: json: unsupported value: NaN

goroutine 37 [running]:
github.com/evilsocket/xray.(*Session).Flush(0xc0003cb800, 0xc00002a6e0?)
        github.com/evilsocket/xray/session.go:52 +0x9e
github.com/evilsocket/xray.(*Pool).FlushSession(0xc0003fe400, 0xc000576000?)
        github.com/evilsocket/xray/pool.go:29 +0x73
main.OnResult({0x8a9be0?, 0xc00041fdd0?})
        github.com/evilsocket/xray/cmd/xray/main.go:51 +0x1e7
github.com/evilsocket/xray.(*Machine).outputConsumer(0xc0000beb00)
        github.com/evilsocket/xray/machine.go:99 +0x4b
created by github.com/evilsocket/xray.(*Machine).Start
        github.com/evilsocket/xray/machine.go:116 +0xb6

web interface issue

it seems every is okey but when i browse that ip address i see bad log and its empty . also i didn't have this issue on first Released .

____  ___
\   \/  /
 \     RAY v 1.0.0b
 /    by Simone 'evilsocket' Margaritelli
/___/\  \
      \_/

@ Saving session to mehran.co-xray-session.json
@ Web UI running on http://xxxxxxx:8080/

0.28 % completed, 35.80 req/s, 7 unique targets found so far ...
0.69 % completed, 44.50 req/s, 7 unique targets found so far ...
0.94 % completed, 40.63 req/s, 7 unique targets found so far ...
http: panic serving xxxxxxx:51691: [map]: [optional]: [xray.Target]: [optional]: [xray.Context]: [optional]: [xray.Machine]: unsupported type: chan interface {}
goroutine 3893 [running]:
net/http.(*conn).serve.func1(0xc420059300)
        /usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0xae96a0, 0xc42039f6f0)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.JSON.Render(0xb0cb80, 0xc420330c00, 0x7f70d8237100, 0xc420405130, 0xc420405100, 0x7f70d8237100)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render/json.go:35 +0x96
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.(*JSON).Render(0xc4203ca4f0, 0x7f70d8237100, 0xc420405130, 0x7f70d8237100, 0xc420405130)
        <autogenerated>:10 +0x6f
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Render(0xc420405130, 0xc8, 0xf0b580, 0xc4203ca4f0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:598 +0xb7
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).JSON(0xc420405130, 0xc8, 0xb0cb80, 0xc420330c00)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:629 +0x96
main.main.func1(0xc420405130)
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x1fa
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc420405130)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:110 +0x5a
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc420290360, 0xc420405130)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:337 +0x6bb
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc420290360, 0xf0ce80, 0xc4201aa1a0, 0xc42048e2d0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:301 +0x130
net/http.serverHandler.ServeHTTP(0xc420238b80, 0xf0ce80, 0xc4201aa1a0, 0xc42048e2d0)
        /usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420059300, 0xf0dac0, 0xc420358640)
        /usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2293 +0x44d
http: panic serving xxxxx:51692: [map]: [optional]: [xray.Target]: [optional]: [xray.Context]: [optional]: [xray.Machine]: unsupported type: chan interface {}
goroutine 3894 [running]:
net/http.(*conn).serve.func1(0xc420059380)
        /usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0xae96a0, 0xc42055d420)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.JSON.Render(0xb0cb80, 0xc4205532f0, 0x7f70d8237100, 0xc4209fb6b0, 0xc4209fb600, 0x7f70d8237100)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render/json.go:35 +0x96
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.(*JSON).Render(0xc42055cb80, 0x7f70d8237100, 0xc4209fb6b0, 0x7f70d8237100, 0xc4209fb6b0)
        <autogenerated>:10 +0x6f
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Render(0xc4209fb6b0, 0xc8, 0xf0b580, 0xc42055cb80)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:598 +0xb7
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).JSON(0xc4209fb6b0, 0xc8, 0xb0cb80, 0xc4205532f0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:629 +0x96
main.main.func1(0xc4209fb6b0)
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x1fa
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4209fb6b0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:110 +0x5a
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc420290360, 0xc4209fb6b0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:337 +0x6bb
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc420290360, 0xf0ce80, 0xc42024edd0, 0xc42048e870)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:301 +0x130
net/http.serverHandler.ServeHTTP(0xc420238b80, 0xf0ce80, 0xc42024edd0, 0xc42048e870)
        /usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420059380, 0xf0dac0, 0xc420358700)
        /usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2293 +0x44d
http: panic serving xxxxxxx:51690: [map]: [optional]: [xray.Target]: [optional]: [xray.Context]: [optional]: [xray.Machine]: unsupported type: chan interface {}
goroutine 3892 [running]:
net/http.(*conn).serve.func1(0xc420059280)
        /usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0xae96a0, 0xc42028c250)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.JSON.Render(0xb0cb80, 0xc4202888d0, 0x7f70d8237100, 0xc420280840, 0xc420280800, 0x7f70d8237100)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render/json.go:35 +0x96
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.(*JSON).Render(0xc4202fd960, 0x7f70d8237100, 0xc420280840, 0x7f70d8237100, 0xc420280840)
        <autogenerated>:10 +0x6f
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Render(0xc420280840, 0xc8, 0xf0b580, 0xc4202fd960)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:598 +0xb7
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).JSON(0xc420280840, 0xc8, 0xb0cb80, 0xc4202888d0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:629 +0x96
main.main.func1(0xc420280840)
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x1fa
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc420280840)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:110 +0x5a
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc420290360, 0xc420280840)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:337 +0x6bb
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc420290360, 0xf0ce80, 0xc4201c7ee0, 0xc42048e0f0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:301 +0x130
net/http.serverHandler.ServeHTTP(0xc420238b80, 0xf0ce80, 0xc4201c7ee0, 0xc42048e0f0)
        /usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420059280, 0xf0dac0, 0xc420358540)
        /usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2293 +0x44d
http: panic serving xxxxx:51689: [map]: [optional]: [xray.Target]: [optional]: [xray.Context]: [optional]: [xray.Machine]: unsupported type: chan interface {}
goroutine 3881 [running]:
net/http.(*conn).serve.func1(0xc420058700)
        /usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0xae96a0, 0xc42041e010)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.JSON.Render(0xb0cb80, 0xc420403800, 0x7f70d8237100, 0xc420281ad0, 0xc420281a00, 0x7f70d8237100)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render/json.go:35 +0x96
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.(*JSON).Render(0xc420429380, 0x7f70d8237100, 0xc420281ad0, 0x7f70d8237100, 0xc420281ad0)
        <autogenerated>:10 +0x6f
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Render(0xc420281ad0, 0xc8, 0xf0b580, 0xc420429380)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:598 +0xb7
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).JSON(0xc420281ad0, 0xc8, 0xb0cb80, 0xc420403800)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:629 +0x96
main.main.func1(0xc420281ad0)
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x1fa
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc420281ad0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:110 +0x5a
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc420290360, 0xc420281ad0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:337 +0x6bb
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc420290360, 0xf0ce80, 0xc4201bedd0, 0xc42048e960)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:301 +0x130
net/http.serverHandler.ServeHTTP(0xc420238b80, 0xf0ce80, 0xc4201bedd0, 0xc42048e960)
        /usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420058700, 0xf0dac0, 0xc4203f5b40)
        /usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2293 +0x44d
http: panic serving xxxxxxx:51693: [map]: [optional]: [xray.Target]: [optional]: [xray.Context]: [optional]: [xray.Machine]: unsupported type: chan interface {}
goroutine 3880 [running]:
net/http.(*conn).serve.func1(0xc420058680)
        /usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0xae96a0, 0xc420574140)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.JSON.Render(0xb0cb80, 0xc420380c00, 0x7f70d8237100, 0xc4200e84d0, 0xc4200e8400, 0x7f70d8237100)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render/json.go:35 +0x96
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.(*JSON).Render(0xc4203f75c0, 0x7f70d8237100, 0xc4200e84d0, 0x7f70d8237100, 0xc4200e84d0)
        <autogenerated>:10 +0x6f
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Render(0xc4200e84d0, 0xc8, 0xf0b580, 0xc4203f75c0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:598 +0xb7
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).JSON(0xc4200e84d0, 0xc8, 0xb0cb80, 0xc420380c00)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:629 +0x96
main.main.func1(0xc4200e84d0)
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x1fa
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4200e84d0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:110 +0x5a
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc420290360, 0xc4200e84d0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:337 +0x6bb
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc420290360, 0xf0ce80, 0xc42005a4e0, 0xc42048e000)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:301 +0x130
net/http.serverHandler.ServeHTTP(0xc420238b80, 0xf0ce80, 0xc42005a4e0, 0xc42048e000)
        /usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420058680, 0xf0dac0, 0xc4203f5c40)
        /usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2293 +0x44d
http: panic serving xxxxx:51694: [map]: [optional]: [xray.Target]: [optional]: [xray.Context]: [optional]: [xray.Machine]: unsupported type: chan interface {}
goroutine 3895 [running]:
net/http.(*conn).serve.func1(0xc420059400)
        /usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0xae96a0, 0xc420478020)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.JSON.Render(0xb0cb80, 0xc420534ba0, 0x7f70d8237100, 0xc4209fa210, 0xc4209fa200, 0x7f70d8237100)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render/json.go:35 +0x96
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/render.(*JSON).Render(0xc42055d6c0, 0x7f70d8237100, 0xc4209fa210, 0x7f70d8237100, 0xc4209fa210)
        <autogenerated>:10 +0x6f
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Render(0xc4209fa210, 0xc8, 0xf0b580, 0xc42055d6c0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:598 +0xb7
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).JSON(0xc4209fa210, 0xc8, 0xb0cb80, 0xc420534ba0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:629 +0x96
main.main.func1(0xc4209fa210)
        /root/work/src/github.com/evilsocket/xray/cmd/xray/main.go:154 +0x1fa
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Context).Next(0xc4209fa210)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/context.go:110 +0x5a
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc420290360, 0xc4209fa210)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:337 +0x6bb
github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc420290360, 0xf0ce80, 0xc42024e1a0, 0xc42048e1e0)
        /root/work/src/github.com/evilsocket/xray/vendor/github.com/gin-gonic/gin/gin.go:301 +0x130
net/http.serverHandler.ServeHTTP(0xc420238b80, 0xf0ce80, 0xc42024e1a0, 0xc42048e1e0)
        /usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420059400, 0xf0dac0, 0xc4203584c0)
        /usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2293 +0x44d
^C

Install Fails with make install_dependencies

Heya,

Interesting project!

Here's the error I'm getting while running make install_dependencies

cw@WolfDen:~/go/src/github.com/evilsocket/xray$ make install_dependencies
glide install
make: glide: Command not found
Makefile:15: recipe for target 'install_dependencies' failed
make: *** [install_dependencies] Error 127

Go Version:

cw@WolfDen:~/go/src/github.com/evilsocket/xray$ go version
go version go1.7.5 linux/amd64

Output of ls -la $GOPATH/bin

cw@WolfDen:~/go/src/github.com/evilsocket/xray$ ls -la $GOPATH/bin
total 10320
drwxrwxr-x 2 cw cw 4096 Jul 9 20:04 .
drwxrwxr-x 5 cw cw 4096 Jul 9 20:04 ..
-rwxrwxr-x 1 cw cw 10557277 Jul 9 20:04 glide

Appreciate guidance in getting xray up and running :)

Thanks

Integrate with Nmap and HTTPScreenshot

Would be possible to integrate with Nmap and Eyewitness to gather http/s screenshot ?

Error checking

I see a lot of if err == nil checks, but I don't see you handling errors at all. It might be a good idea to handle and log the errors. It seems like many of the errors can be logged as informational, but some errors like this may want to be logged as a warning, or even this as an error.

build the docker error

=> ERROR [3/4] RUN go get github.com/evilsocket/xray && cd /go/s 0.3s

[3/4] RUN go get github.com/evilsocket/xray && cd /go/src/github.com/evilsocket/xray/ && make:
#5 0.298 go: go.mod file not found in current directory or any parent directory.
#5 0.298 'go get' is no longer supported outside a module.
#5 0.298 To build and install a command, use 'go install' with a version,
#5 0.298 like 'go install example.com/cmd@latest'
#5 0.298 For more information, see https://golang.org/doc/go-get-install-deprecation
#5 0.298 or run 'go help get' or 'go help install'.

---

executor failed running [/bin/sh -c go get github.com/evilsocket/xray && cd $GOPATH/src/github.com/evilsocket/xray/ && make]: exit code: 1