kristian-lange/net-glimpse

1) Real-time visualization of network traffic (Ethernet and Internet), and 2) Streaming of header data from your network interfaces via WebSockets

I wanted a little tool that gives me a quick overview on what's going in my network right now, displaying it in the browser so I can access it easily whenever I want. Tools like tcpdump or Wireshark are great but don't give you this 'glimpse'.

net-glimpse has two parts: 1) Visualization of network traffic (Ethernet and/or Internet) in real-time, and 2) Streaming of header data from your network interfaces via WebSockets.

Have a look at this video.

Used technologies

• Scala, JavaScript
• Pcap4J (https://github.com/kaitoy/pcap4j) to access network interfaces
• Play Framework and sbt
• Akka to distribute network interface data to multiple WebSockets
• Graphics with p5js and physics with toxiclibs

Contents

• How to run
  • Prerequisites
  • Install and run
• Visualization of network traffic
  • Endpoints
  • Visualization Details
  • Visualization Configuration
• Streaming of header data from your network interfaces via WebSockets
  • Usage in JavaScript
  • Backend configuration
• Build yourself and modify the source code

How to run

net-glimpse works on Linux/Unix (including Mac OS X) and Windows.

Prerequisites

• net-glimpse needs Java 8 (JRE is enough) to run.
• For the visualizations you need a modern browser (one that supports WebSockets and WebGL).
• On Windows it is necessary to install Npcap (https://nmap.org/npcap/) OR WinPcap (https://www.winpcap.org/). On Linux/Unix libpcap is needed.

Install and run

1. Download the net-glimpse-x.x.zip

2. Unzip and change into the unzipped folder

3. On Linux or Unix to access network interfaces you have to start the net-glimpse either as root or give Java special capabilities, e.g. with sudo setcap cap_net_raw,cap_net_admin=eip /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java (but exchange the path to your Java installation). On Windows you will be ask after starting net-glimpse's batch file if you want to grant access to the network interfaces.

4. On Unix it might be necessary to make the run script executable: chmod u+x ./bin/net-glimpse

5. Run on Linux or Unix in the terminal ./bin/net-glimpse - on Windows double-click the ./bin/net-glimpse.bat

   You can specify IP and port with the parameters -Dhttp.address and -Dhttp.port. E.g. ./bin/net-glimpse -Dhttp.address=172.23.1.81 -Dhttp.port=8080 binds net-glimpse to IP 172.23.1.81 and port 8080. If you don't specify these parameters the defaults 0.0.0.0 (listens all addresses) and 9000 are used.

   If net-glimpse does not want to start have a look in its installation folder whether you find a file RUNNING_PID. Delete it and start again.

6. Try in a browser, e.g. with http://localhost:9000/?nif=myNetworkInterface (instead of myNetworkInterface use the name of the network interface you want to intercept)

   net-glimpse prints out potential network interface names (NIF) in it's log. You can copy-paste one from there. This is especially handy on Windows where they have names like, e.g. \Device\NPF_{998BB72F-3468-413E-813C-7E3A2E7B591B} which would lead to the URL http://localhost:9000/?nif=\Device\NPF_{998BB72F-3468-413E-813C-7E3A2E7B591B}.

   The resulting webpage shows a list of raw packet header data in JSON format.

   

7. If you are done with net-glimpse you can stop it with Ctrl+C.

Visualization of network traffic

Endpoints

1. http://localhost:9000/glimpse?nif=myNetworkInterface - shows both, Ethernet and Internet  

2. http://localhost:9000/ipglimpse?nif=myNetworkInterface - shows only Internet

3. http://localhost:9000/etherglimpse?nif=myNetworkInterface - shows only Ethernet

4. http://localhost:9000/?nif=myNetworkInterface - shows raw packet header data in JSON

E.g. http://localhost:9000/glimpse?nif=wlp3s0 shows a visualization of the Ethernet layer and the Internet layer of the network interface wlp3s0.

You can open multiple pages of the same or different network interface(s) at the same time.

Visualization Details

• You can press 'p' to pause the drawing at any time
• It's actually a force-directed graph
• Nodes represent MAC or IP addresses
• Node colors are determined by the MAC or IP address (means the same MAC or IP address leads always to the same color)
• Nodes with broadcast or multicast IP addresses are white.
• Nodes and edges blink when a new packet is sent
• Edges represent sent packets
• The arrow shows the direction of the sent packet
• The edges get thicker the more packets are sent
• The EtherType (in the Ethernet visualization) is annotated at the edge (scroll down - under the graphic is a glossary)
• In the Internet visualization if it is a TCP or UDP packet and the port is one of the well known or registered ones (port 0 to 49151) the port is annotated at the edge. The most common port numbers are exchanged with their names, eg. port 22 is exchanged with SSH (scroll down - under the graphic is a glossary).
• If it's the Internet and not a TCP or UDP packet then the protocol name is annotated at the edge.
• Edges of unknown EtherTypes or Internet packets are black/gray by default (but you can add new types in the config)
• Nodes and edges get removed after a while if no packets are sent (default is 10 s)
• In fullscreen mode the whole screen is used for the graph(s) - otherwise they have a squared canvas

Visualization Configuration

Many parameters of the visualizations can be changed, e.g.

• Edge colors and annotation
• Node size and node repulsion
• Cleaning interval and max age of nodes
• Blacklist and whitelist for IP and MAC addresses

The configuration file is in ./config/glimpse.conf. More details are in the comments of the config file.

Streaming of header data from your network interfaces via WebSockets (backend)

Usually it's not possible to access network interfaces from within a browser. net-glimpse uses pcap4j to access the interfaces and then streams the header data via WebSockets into the browser. You can use this part of net-glimpse independent of the visualization.

Usage in JavaScript

If you just want to get the header data without the visualization you have to open a WebSocket with the URL /netdata and the network interface you want to intercept has to be specified in the query string with the parameter 'nif'.

E.g. in JavaScript (browser) to get traffic from the network interface wlp3s0 one could write

var socket = new WebSocket("ws://myhost/netdata/?nif=wlp3s0");

or more generally with secure WebSockets and assuming net-glimpse runs on the same host as your JavaScript is served from.

var socket = new WebSocket(
      ((window.location.protocol === "https:") ? "wss://" : "ws://") +
      window.location.host + "/netdata/?nif=wlp3s0");

The streamed packet header data are in JSON format.

• It is possible to stream different network interfaces in parallel.
• It is also possible to stream the same network interface to multiple destinations.
• Only header data are captured and streamed via WebSockets - the actual payload is not read.

Backend configuration

Via -D run parameters

net-glimpse takes a couple of parameters:

• -Dnif - Specifies the default network interface. If you specify it here you can leave it out in the URL query. It has no default.
• -DskipOwnTraffic - If true net-glimpse's own network traffic (via WebSockets) is not streamed. Default is true.
• -Dsnaplen - Sets the snap length (see wiki.wireshark.org/SnapLen for more info). Default is 128 byte.
• -Dhttp.address - Specifies the IP address net-glimpse runs on. Default is 0.0.0.0 (listens on all IPs).
• -Dhttp.port - Specifies the port net-glimpse runs on. Default is 9000.

e.g. ./bin/net-glimpse -Dhttp.address=192.168.178.160 -Dhttp.port=80 -Dnif=wlp3s0 -DskipOwnTraffic=false

Via conf/application.conf

All parameters that can be specified via -D run parameters can be set in ./conf/application.conf too.

Build yourself and modify the source code

If you don't trust net-glimpse' pre-build releases you can build it yourself. It uses sbt as build tool. Just download the source code and run sbt dist. In ./target/universal/ will be the built .zip file. More information can be found in https://www.playframework.com/documentation/2.6.x/Deploying or https://www.playframework.com/documentation/2.6.x/BuildOverview.

If you want to modify the source code (e.g. you need more data from the network packets) the following files are probably where you want to start:

• /app/services/PacketToJsonTransfer.scala - serialises the packet data into JSON (backend-side)
• /public/netDataReceiver.js - deserialising JSON in the browser
• /public/p5visu.js - visalisation with p5.js
• /public/graph.js - graph data structure and physics

net-glimpse's Issues

Unexpected exception[RuntimeException: Couldn't open network interface empty

Reproduction steps:

1. Download zip file
2. Extract
3. Fix permissions ./bin/net-glimpse
4. run ./bin/net-glimpse -Dhttp.port=9001
5. Go to http://localhost:9001 and you will see a blank screen, logs:

[info] play.api.Play - Application started (Prod)
[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9001
[info] o.p.core.Pcaps - 13 NIF(s) found.
[error] application - 

! @74jld8p16 - Internal server error, for (GET) [/netdata] ->
 
play.api.UnexpectedException: Unexpected exception[RuntimeException: Couldn't open network interface empty]
	at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:289)
	at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:220)
	at play.api.GlobalSettings$class.onError(GlobalSettings.scala:160)
	at play.api.DefaultGlobal$.onError(GlobalSettings.scala:188)
	at play.api.http.GlobalSettingsHttpErrorHandler.onServerError(HttpErrorHandler.scala:100)
	at play.core.server.netty.PlayRequestHandler$$anonfun$handle$1.applyOrElse(PlayRequestHandler.scala:129)
	at play.core.server.netty.PlayRequestHandler$$anonfun$handle$1.applyOrElse(PlayRequestHandler.scala:127)
	at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:346)
	at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:345)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
Caused by: java.lang.RuntimeException: Couldn't open network interface empty
	at services.PcapInitializer.openPcap(PcapInitializer.java:97)
	at services.PcapInitializer.getPcapDispatcherActorRef(PcapInitializer.java:80)
	at controllers.HomeController.netdata(HomeController.java:30)
	at router.Routes$$anonfun$routes$1$$anonfun$applyOrElse$5$$anonfun$apply$5.apply(Routes.scala:213)
	at router.Routes$$anonfun$routes$1$$anonfun$applyOrElse$5$$anonfun$apply$5.apply(Routes.scala:213)
	at play.core.j.JavaWebSocket$$anonfun$ofJson$1.apply(JavaWebSocket.scala:107)
	at play.core.j.JavaWebSocket$$anonfun$ofJson$1.apply(JavaWebSocket.scala:107)
	at play.core.j.JavaWebSocket$$anonfun$webSocketWrapper$1.apply(JavaWebSocket.scala:34)
	at play.core.j.JavaWebSocket$$anonfun$webSocketWrapper$1.apply(JavaWebSocket.scala:28)
	at play.api.mvc.WebSocket$$anon$1.apply(WebSocket.scala:43)
[info] s.NifDispatcherActor - registered WebSocket actor to network interface empty

What am I doing wrong?

Not working after a few seconds

For some reason, the web socket stops sending messages after a few seconds... I can restart the application so it starts working again, but stops after a few seconds...

I use Fedora Linux

Ignoring Port 9000 traffic

Firstly, this is a great idea!!

However, I'm finding that if I expose the service then all of the output is net-glimpse websockets itself.

Is there a way to ignore traffic on port 9000?