skydive-project/skydive

An open source real-time network topology and protocols analyzer

skydive's Issues

Add ES as topology backend

Add a reference on node to capture UUID

Currently we only know when there is capture on a node thanks to the State.FlowCapture metadata entry. It could be useful to know the UUID of the capture attached.

Improve traversal step reduce

At the moment, we iterate through all the step and check if a step can be reduced with the next one.

Let's consider we have the steps : A -> B -> C
If B -> C can be reduced to C, we'll have : A -> C

With the current code, we don't check that A -> C can be reduced.

Timeout with 'G.V().Flows()' when no agent

The gremlin request 'G.V().Flows()' return an empty list after the timeout expiration without reporting any error. When there is no agent connected to the analyzer, for example due to bad etcd configuration.

Improve WebUI to show layer2 link between host

Graph need to be locked when doing Gremlin request

There are races on Graph access on Exec function of traversal_parser.go

http://softwarefactory-project.io/jenkins/job/skydive-functional-tests-backend-gremlin-ws/3320/console

Add functional tests for storage backends

Support versioning of metrics in ES

Trying to lookup flow only on supported nodes

Currently when we do a lookup for flow we lookup for all the graph nodes but only some of them can have link to flows. For example host node or netns node can't have link to flows. It makes sense to not send UUID of these nodes when doing lookups.

Add configuration for ES Bulk parameters

Secure infos sent by the agent, add TLS layer over UDP

Add limit step to gremlin

Handle ovsdb disconnection/reconnection

Use snaplen in pcap probe

Need to adapt the snaplen according the flow process, no need to get the whole packet

Switch from lebauce/dockerclient to docker/engine-api

At this time, we use a fork of samalba/dockerclient which is now deprecated. We should use docker/engine-api which are the official Go bindings for Docker

report ofport to the topology

Add filter to WebUI to only show nodes and edges that match a Gremlin expression

Add support for multiple analyzers

Add a way to define external resources

http://softwarefactory-project.io/r/#/c/4458/

Add eBPF Flow probe

http://softwarefactory-project.io/r/#/c/4063/

gzip websocket message

This should improve the performances especially for the flow requests

This extension could help
gorilla/websocket@a87eae1

Adapt the HeaderSize in ovssflow according the flow process

Need to use a better size according to the size of the headers we need to parse.

Add information to all interfaces managed by neutron

Currently only showing interface on one interface

Add IPV6 support

Skydive analyzer crashes if elasticsearch is not available

When I start an analyzer and the elastic search is not ready, the analyzer crashes without waiting

Add a way to export live capture to a file

Add a Gremlin 'Sort' step

Right now, the flows are sorted on the agent when they are queried. This is time consuming and should be optional.

Fix regular etcd failure on CI

Generate a nightly tests from rpm package

Generate nightly rpm packages binaries
Do some test by installing on Fedora/Centos image and launching multiple agents, ...

etcd initialization error

I'm getting errors while starting th eanalyzer

$ skydive analyzer --conf /etc/skydive/skydive.yml
2016-03-03T15:54:43.768Z devstack:skydive analyzer.go:46 analyzer glob.func1 > NOTI 005 Skydive Analyzer starting...
2016-03-03 15:54:43.768837 I | etcdserver: name = skydive
2016-03-03 15:54:43.768900 I | etcdserver: data dir = /tmp/skydive-etcd
2016-03-03 15:54:43.768994 I | etcdserver: member dir = /tmp/skydive-etcd/member
2016-03-03 15:54:43.769056 I | etcdserver: heartbeat = 100ms
2016-03-03 15:54:43.769142 I | etcdserver: election = 1000ms
2016-03-03 15:54:43.769202 I | etcdserver: snapshot count = 0
2016-03-03 15:54:43.769263 I | etcdserver: advertise client URLs = http://192.168.122.17:2379
2016-03-03 15:54:43.786243 I | etcdserver: restarting member cc743f7e0a51972 in cluster 1ddb10353625184f at commit index 4
2016-03-03 15:54:43.786392 I | raft: cc743f7e0a51972 became follower at term 2
2016-03-03 15:54:43.786503 I | raft: newRaft cc743f7e0a51972 [peers: [], term: 2, commit: 4, applied: 0, lastindex: 4, lastterm: 2]
2016-03-03 15:54:43.786974 I | etcdserver: set snapshot count to default 10000
2016-03-03 15:54:43.787072 I | etcdserver: starting server... [version: 2.3.0-alpha.0+git, cluster version: to_be_decided]
2016-03-03 15:54:43.797405 N | etcdserver: added local member cc743f7e0a51972 [http://localhost:2379] to cluster 1ddb10353625184f
2016-03-03T15:54:44.799Z devstack:skydive analyzer.go:52 analyzer glob.func1 > CRIT 006 Can't start Analyzer : client: etcd cluster is unavailable or misconfigured

It seems a note on etcd configuration is missing on the "install guide", or am I doing something wrong?

Flatten Flow structure

Currently flows are made of embed structure for statistics, keeping a statistics per layer. The goal here is to flatten the flow structure to keep only one level of statistics. The will help to improve the performance. Moreover one packet will possibly generate more than one flow depending on the encapsulation. The flows generated could be linked.

race is http/wsserver.go with clients struct member

There is a race when accessing to the clients struct member as this member can be read/written from different goroutines like in SendWSMessageTo and in listenAndServe.

Use analyzers address when no etcd parameter provided

Currently we have to set the etcd servers parameter is the config file even if we are using the embed etcd. It would be better to use the analyzers list as etcd servers list when this list is not specified is the config file.

Add layer2 link between host

http://softwarefactory-project.io/r/#/c/4458/

Add unique topology ID on graph element not changing over the time

API to inject packets

Add netlink Statistics to nodes

Add Statistics returned by netlink to interface nodes so that we can get dropped packet number, etc...
This need to add an async mechanism to get metrics in order to have them updated.

titangraph: no unique index for edges

nodes have unique ids but not edges, but should have the same _ID definition

Add support for nested environment like VM hosting VMs

Use case:

Skydive agent on the hypervisor and within a VM

Add Openshift connector

Switch back to protobuf 2.x

As we didn't use protobuf 3.x features, like map
and protobuf 2.x mostly supported natively by all distro

Update documentation for flow filtering

Add G.Flows() gremlin step

Add Flows().Has validator

Currently the "Has" expression is sent directly to the agents or the storage. It could improve the performances to check that the expression matches the Flow definition before sending the request. So that the expressions not matching the flow definition or using a wrong type for the value will not be executed on each flow.

Add Both gremlin step for flows

Add Count gremlin step for topology and flows

Reenable devstack job

update rpm spec file

update contrib/packaging/rpm/skydive.spec file to work with govendor and github source : https://github.com/skydive-project/skydive/archive/v0.3.0.tar.gz

Reduce calls to the neutron API

Add Layer3 link to topology

Use routing and layer2 information to create Layer3 links to the topology.

Partially :
http://softwarefactory-project.io/r/#/c/4458/