Giter Site home page Giter Site logo

tcpkit's Introduction

tcpkit Build Status

The tcpkit was designed to analyze network packets with lua script, can also be used to observe the request latency of the service with simple protocol, like redis/memcached.

Install

$ git clone https://github.com/git-hulk/tcpkit.git tcpkit
$ cd tcpkit
$ sudo make && make install

Usage

the tcpkit was designed to make network packets programable with LUA by @git-hulk
   -h, Print the tcpkit version strings, print a usage message, and exit
   -i interface, Listen on network card interface
   -r file, Read packets from file (which was created with the -w option or by other tools that write pcap)
   -A Print each packet (minus its link level header) in ASCII.  Handy for capturing web pages
   -B buffer_size, Set the operating system capture buffer size to buffer_size, in units of KiB (1024 bytes)
   -s snaplen, Snarf snaplen bytes of data from each packet rather than the default of 1500 bytes
   -S file, Push packets to lua state if the script was specified
   -t threshold, Print the request lantecy which slower than the threshold, in units of Millisecond
   -w file, Write the raw packets to file
   -p protocol, Parse the packet if the protocol was specified (supports: redis, memcached, http, raw)
   -P stats port, Listen port to fetch the latency stats, default is 33333


For example:

   `tcpkit -i eth0 tcp port 6379 -p redis` was used to monitor the redis reqeust latency

   `tcpkit -i eth0 tcp port 6379 -p redis -w 6379.pcap` would also dump the packets to `6379.pcap`

   `tcpkit -i eth0 tcp port 6379 -p redis -t 10` would only print the request latency slower than 10ms

How To Observe The Latency Of Redis/Memcached

$ tcpkit -i eth0 tcp port 6379 -p redis

tcpkit would listen on NIC eth0 and caputure the tcp port 6379, then parse network packets with Redis protocol. The output was like below:

2020-03-08 19:23:06.253384 127.0.0.1:51137 => 127.0.0.1:6379 | 0.615 ms | COMMAND
2020-03-08 19:23:06.258761 127.0.0.1:51137 => 127.0.0.1:6379 | 0.059 ms | get a

Use the option -t would only show the request which the request latency was slower than threshold(in units of millisecond).

How to Use Lua Script

$ tcpkit -i eth0 tcp port 6379 -p redis -S scripts/example.lua

the callback function function process(packet) in scripts/example.lua would be triggered if new packets reached.

Predefine Scripts

  1. exmaple.lua - example for user defined script
  2. dns.lua - print the dns latency
  3. tcp-connnect.lua - print connection with syn packet retransmit

Request Latency

the stats port listen on 33333 by default, use the -P to change the listen port.

➜  ~ telnet 127.0.0.1 33333
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

{
	"127.0.0.1:6379":	{
		"requests":	1700,
		"request_bytes":	184100,
		"responses":	1700,
		"response_bytes":	1413764,
		"latency":	[{
				"<0.1ms":	326
			}, {
				"0.1ms~0.2ms":	371
			}, {
				"0.2~0.5ms":	589
			}, {
				"0.5ms~1ms":	291
			}, {
				"1ms~5ms":	123
			}]
	}
}

License

tcpkit is under the MIT license. See the LICENSE file for details.

tcpkit's People

Contributors

dezren39 avatar git-hulk avatar postwait avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tcpkit's Issues

make failed: collect2: error: ld returned 1 exit status

I try to install the project but failed.

system info:

$ uname -a
Linux archlinux 4.15.2-2-ARCH #1 SMP PREEMPT Thu Feb 8 18:54:52 UTC 2018 x86_64 GNU/Linux

make outputs:

make-error.txt

......
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:238: undefined reference to `dbus_error_free'
../deps/libpcap/libpcap.a(pcap-dbus.o): In function `dbus_cleanup':
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:143: undefined reference to `dbus_connection_unref'
../deps/libpcap/libpcap.a(pcap-dbus.o): In function `dbus_activate':
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:169: undefined reference to `dbus_bus_get'
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:178: undefined reference to `dbus_error_free'
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:227: undefined reference to `dbus_connection_set_max_received_size'
../deps/libpcap/libpcap.a(pcap-dbus.o): In function `dbus_cleanup':
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:143: undefined reference to `dbus_connection_unref'
../deps/libpcap/libpcap.a(pcap-dbus.o): In function `dbus_activate':
/home/guyskk/opt/tcpkit/deps/libpcap/./pcap-dbus.c:193: undefined reference to `dbus_error_free'
collect2: error: ld returned 1 exit status
make: *** [Makefile:23: tcpkit] Error 1

libpcap build error

using arch linux here. core/libpcap 1.9.0-1 [installed] is installed and I'm run into the following build error

gcc -g -Wall -o tcpkit util.o array.o hashtable.o logger.o packet.o redis.o server.o sniffer.o stats.o tcpkit.o vm.o -I../deps/libpcap/ -I../deps/lua/src ../deps/libpcap/libpcap.a ../deps/lua/src/liblua.a -lm -ldl -lpthread
/usr/bin/ld: ../deps/lua/src/liblua.a(loslib.o): in function `os_tmpname':
/tmp/tcpkit/deps/lua/src/loslib.c:107: warning: the use of `tmpnam' is dangerous, better use `mkstemp'
/usr/bin/ld: ../deps/libpcap/libpcap.a(pcap-linux.o): in function `nl80211_init':
/tmp/tcpkit/deps/libpcap/./pcap-linux.c:674: undefined reference to `nl_socket_alloc'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:681: undefined reference to `genl_connect'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:687: undefined reference to `genl_ctrl_alloc_cache'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:695: undefined reference to `genl_ctrl_search_by_name'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:707: undefined reference to `nl_socket_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:689: undefined reference to `nl_geterror'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:705: undefined reference to `nl_cache_free'
/usr/bin/ld: ../deps/libpcap/libpcap.a(pcap-linux.o): in function `nl80211_cleanup':
/tmp/tcpkit/deps/libpcap/./pcap-linux.c:714: undefined reference to `genl_family_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:715: undefined reference to `nl_cache_free'
/usr/bin/ld: ../deps/libpcap/libpcap.a(pcap-linux.o): in function `del_mon_if':
/tmp/tcpkit/deps/libpcap/./pcap-linux.c:846: undefined reference to `nlmsg_alloc'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:853: undefined reference to `genl_family_get_id'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:853: undefined reference to `genlmsg_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:855: undefined reference to `nla_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:857: undefined reference to `nl_send_auto_complete'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:865: undefined reference to `nl_wait_for_ack'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:877: undefined reference to `nlmsg_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:884: undefined reference to `nlmsg_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:859: undefined reference to `nl_geterror'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:870: undefined reference to `nlmsg_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:867: undefined reference to `nl_geterror'
/usr/bin/ld: ../deps/libpcap/libpcap.a(pcap-linux.o): in function `add_mon_if':
/tmp/tcpkit/deps/libpcap/./pcap-linux.c:736: undefined reference to `nlmsg_alloc'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:743: undefined reference to `genl_family_get_id'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:743: undefined reference to `genlmsg_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:745: undefined reference to `nla_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:746: undefined reference to `nla_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:747: undefined reference to `nla_put'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:749: undefined reference to `nl_send_auto_complete'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:770: undefined reference to `nl_geterror'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:801: undefined reference to `nlmsg_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:830: undefined reference to `nlmsg_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:777: undefined reference to `nl_wait_for_ack'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:798: undefined reference to `nl_geterror'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:791: undefined reference to `nlmsg_free'
/usr/bin/ld: /tmp/tcpkit/deps/libpcap/./pcap-linux.c:809: undefined reference to `nlmsg_free'
/usr/bin/ld: ../deps/libpcap/libpcap.a(pcap-linux.o): in function `nl80211_cleanup':
/tmp/tcpkit/deps/libpcap/./pcap-linux.c:716: undefined reference to `nl_socket_free'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:24: tcpkit] Error 1
make[1]: Leaving directory '/tmp/tcpkit/src'
make: *** [Makefile:4: all] Error 2

udp packet

  1. support udp packet
  2. support dns lantency

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.