ga0 / netgraph Goto Github PK

View Code? Open in Web Editor NEW

938.0 37.0 96.0 5.34 MB

A cross platform http sniffer with a web UI

License: MIT License

HTML 1.36% CSS 0.29% JavaScript 2.97% Go 94.81% Python 0.49% Shell 0.09%

network-analysis http pcap-analyzer network-monitoring sniffer

netgraph's Introduction

Netgraph

Netgraph is a packet sniffer tool that captures all HTTP requests/responses, and display them in web page.

You can run Netgraph in your linux server without desktop environment installed, and monitor http requests/responses in your laptop's browser.

Compile, Install, Run

  1. go get github.com/ga0/netgraph
  2. run $GOPATH/bin/netgraph -i INTERFACE -p PORT
  3. open the netgraph web page in your browser (for example: http://localhost:9000, 9000 is the PORT set in step 2)

  Windows user need to install winpcap library first.

Options

  -bpf string
        Set berkeley packet filter (default "tcp port 80")
  -i string
        Listen on interface, auto select one if no interface is provided
  -input-pcap string
        Open a pcap file
  -o string
        Write HTTP requests/responses to file, set value "stdout" to print to console
  -output-pcap string
        Write captured packet to a pcap file
  -output-request-only
	      Write only HTTP request to file, drop response. Only used when option "-o" is present. (default true)
  -p int
        Web server port. If the port is set to '0', the server will not run.  (default 9000)
  -s	Save HTTP event in server
  -v	Show verbose message (default true)

Example: print captured requests to stdout:

  $ ./netgraph -i en0 -o=stdout
  2018/07/26 10:33:24 open live on device "en0", bpf "tcp port 80"
  [2018-07-26 10:33:34.873] #0 Request 192.168.1.50:60448->93.184.216.34:80
  GET / HTTP/1.1
  Host: www.example.com
  Connection: keep-alive
  Pragma: no-cache
  Cache-Control: no-cache
  Upgrade-Insecure-Requests: 1
  User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
  Accept-Encoding: gzip, deflate
  Accept-Language: zh-CN,zh;q=0.9,en;q=0.8,zh-TW;q=0.7

  content(0)

License

MIT

netgraph's People

Contributors

Stargazers

Watchers

Forkers

ver007 oblank 4lyingli0n image72 xxoxx funkyyang iitsoft yushen777 gamehacker hinike gooops hexl21 tanyingyu xi4mi huakaii pgyggi 0x554simon adnaan flying0er juneman alexmiomio yuanfeng0905 srinivasgowda097 genghaihua xiazemin swat521 awesome-golang hh9net zxm679 deleskie gotoolkits lorock zhanglei ruhip xiaozhenliang funstuff zjuchenyuan queniao thattimc grantpullen kaelanb 5up3rc fishline yuanddqiao jokaye dikang123 sahilgoel1610 cancheung wangbokun xuanjiazhen priestd09 chennqqi kakiezhang weijunlee mfrigillana grospy kissinger31 brooksbellinc iceber zuiwanting orochi4268 go-stream apple0407 debugspy 76428778fada ninefive xuxx09 zhangyangisme sanen killvxk fsfish zero-yy yuanli3303 zhangpuhan typercode whczs nh-live bluemoon2008 leetesla cyjdlxd foxhack yl365 wjt2015 douglasslaviero rodolfocdsantos cnmars billmetangmo libin8412 go-archive nrbackback iq-scm mrevilliver yjq635 yhtsnda

netgraph's Issues

请求无法侦的问题

我的8888 端口是一个http 代理，使用的ip 是 192.168.33.10
root@stretch:~# netstat -ano|grep 8888
tcp6 0 0 :::8888 :::* LISTEN off (0.00/0/0)
tcp6 0 0 192.168.33.10:8888 192.168.33.1:52950 ESTABLISHED keepalive (14.74/0/0)
tcp6 0 0 192.168.33.10:8888 192.168.33.1:53477 FIN_WAIT2 timewait (48.60/0/0)
tcp6 0 0 192.168.33.10:8888 192.168.33.1:53018 ESTABLISHED keepalive (2.46/0/0)

该ip 在 eth1 网卡上面
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.33.10 netmask 255.255.255.0 broadcast 192.168.33.255
inet6 fe80::a00:27ff:fec3:4005 prefixlen 64 scopeid 0x20
ether 08:00:27:c3:40:05 txqueuelen 1000 (Ethernet)
RX packets 6808 bytes 753436 (735.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 9798 bytes 8891562 (8.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

我通过 go/bin/netgraph -bpf "tcp port 8888" -i eth1
侦听eth1网卡上面的8888 端口，然后并没有请求该代理上的请求包信息
请问是我那个步骤是错误的吗？

there`s some problem when go build

system platform: macos 10.14
go version: go1.11.2 darwin/amd64
when run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o netgrap
problem:
./ng.go:71:14: undefined: pcap.FindAllDevs
./ng.go:101:18: undefined: pcap.OpenOffline
./ng.go:116:17: undefined: pcap.OpenLive
./ng.go:116:57: undefined: pcap.BlockForever

As seen in this screenshot, the body of DELETE request is not correct:

To make sure it's not a bad pcap file, I looked at packets using http://www.tastycocoabytes.com/cpa/.

This is how body of the request should look like:

I used netgraph -input-pcap trace_indexes_from_client_go.pcap

Race condition in netgraph

When I build netgraph with -race enabled:

go build -race github.com/ga0/netgraph

And then run on a .pcap file e.g.:

./netgraph -input-pcap trace_indexes_from_client_go.pcap

I get the following race condition report:

WARNING: DATA RACE
Read at 0x00c4200ae508 by goroutine 20:
  github.com/ga0/netgraph/ngnet.(*HTTPStreamFactory).runStreamPair()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ngnet/httpstreamfactory.go:44 +0x84

Previous write at 0x00c4200ae508 by goroutine 10:
  github.com/ga0/netgraph/ngnet.(*HTTPStreamFactory).runStreamPair()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ngnet/httpstreamfactory.go:44 +0x9a

Goroutine 20 (running) created at:
  github.com/ga0/netgraph/ngnet.HTTPStreamFactory.New()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ngnet/httpstreamfactory.go:71 +0x108c
  github.com/ga0/netgraph/ngnet.(*HTTPStreamFactory).New()
      <autogenerated>:1 +0xf9
  github.com/google/gopacket/tcpassembly.(*StreamPool).getConnection()
      /Users/kjk/src/go/src/github.com/google/gopacket/tcpassembly/assembly.go:502 +0x1f6
  github.com/google/gopacket/tcpassembly.(*Assembler).AssembleWithTimestamp()
      /Users/kjk/src/go/src/github.com/google/gopacket/tcpassembly/assembly.go:550 +0x23f
  main.runNGNet()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ng.go:175 +0x757

Goroutine 10 (running) created at:
  github.com/ga0/netgraph/ngnet.HTTPStreamFactory.New()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ngnet/httpstreamfactory.go:71 +0x108c
  github.com/ga0/netgraph/ngnet.(*HTTPStreamFactory).New()
      <autogenerated>:1 +0xf9
  github.com/google/gopacket/tcpassembly.(*StreamPool).getConnection()
      /Users/kjk/src/go/src/github.com/google/gopacket/tcpassembly/assembly.go:502 +0x1f6
  github.com/google/gopacket/tcpassembly.(*Assembler).AssembleWithTimestamp()
      /Users/kjk/src/go/src/github.com/google/gopacket/tcpassembly/assembly.go:550 +0x23f
  main.runNGNet()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ng.go:175 +0x757

This probably could be fixed by using atomic.AddInt32 for HTTPStreamFactory.seq

and:

WARNING: DATA RACE
Write at 0x00c4200c2230 by main goroutine:
  internal/race.Write()
      /usr/local/Cellar/go/1.10.3/libexec/src/internal/race/race.go:41 +0x38
  sync.(*WaitGroup).Wait()
      /usr/local/Cellar/go/1.10.3/libexec/src/sync/waitgroup.go:127 +0xf3
  main.(*NGServer).Wait()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ngserver.go:98 +0x3e
  main.runEventHandler()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ng.go:273 +0x14a
  main.main()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ng.go:286 +0x94

Previous read at 0x00c4200c2230 by goroutine 6:
  internal/race.Read()
      /usr/local/Cellar/go/1.10.3/libexec/src/internal/race/race.go:37 +0x38
  sync.(*WaitGroup).Add()
      /usr/local/Cellar/go/1.10.3/libexec/src/sync/waitgroup.go:70 +0x16e
  main.(*NGServer).Serve()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ngserver.go:144 +0x199

Goroutine 6 (running) created at:
  main.initEventHandlers()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ng.go:60 +0x404
  main.main()
      /Users/kjk/src/go/src/github.com/ga0/netgraph/ng.go:283 +0x33

This probably would require stopping listening for new requests before calling NGServer.Wait.

go build时报错

$ go build
# github.com/google/gopacket/pcap
C:\Users\ADMINI~1\AppData\Local\Temp\go-build504898774\github.com\google\gopacket\pcap\_obj\pcap.cgo2.o: In function `_cgo_c09307494e16_Cfunc_pcap_free_datalinks':
H:/mygopath/src/github.com/google/gopacket/pcap/pcap.go:322: undefined reference to `pcap_free_datalinks'
collect2.exe: error: ld returned 1 exit status

Readme.md typo

2. 执行 $GOPATH/bin/netgraph -e 网卡名称(比如eth0) -p 服务器端口(默认9000)

似乎应该是：

2. 执行 $GOPATH/bin/netgraph -i 网卡名称(比如eth0) -p 服务器端口(默认9000)

about: cannot allocate memory

I can captrue pacage use tcpdump ; but I get into trouble when use netgraph ,like this:

2016/01/12 13:01:26 lo: can't create rx ring on packet socket: Cannot allocate memory

What's wrong with me？
thanks for your help！

公司需要抓取网卡的镜像数据，能不能给个联系方式，需要支持，可付费。

请问是否支持HTTPS？

您好，请问是否支持HTTPS？根据 Pre-Master-Secret log 文件解码 TLS 层的加密数据，然后按照HTTP流量的方式解析？