session-replay-tools/tcpcopy

An online request replication and TCP stream replay tool, ideal for real testing, performance testing, stability testing, stress testing, load testing, smoke testing, and more.

tcpcopy's Issues

如果tcpcopy命令中没有‘-’，需要有命令错误提示

intercept报nl recvfrom (No buffer space available），怎么办

加大设置intercept所在的机器的内核buffer，比如：

sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216

have any method for copying multi request?

I use tcpcopy for some test take the place of ab.
it have any method for copying multi requests?

if online machine req = 10req/s, but on the target machine i want 10*n req/s.

在configure.ac文件中针对某些头文件设置AC_CHECK_HEADERS后，有warning

例如：
linux/netfilter_ipv4/ip_queue.h

在有些操作系统有如下warning：

configure: WARNING: linux/netfilter_ipv4/ip_queue.h: present but cannot be compiled
configure: WARNING: linux/netfilter_ipv4/ip_queue.h: check for missing prerequisite headers?
configure: WARNING: linux/netfilter_ipv4/ip_queue.h: see the Autoconf documentation
configure: WARNING: linux/netfilter_ipv4/ip_queue.h: section "Present But Cannot Be Compiled"
configure: WARNING: linux/netfilter_ipv4/ip_queue.h: proceeding with the preprocessor's result
configure: WARNING: linux/netfilter_ipv4/ip_queue.h: in the future, the compiler will take precedence
configure: WARNING: ## ----------------------------------- ##
configure: WARNING: ## Report this to [email protected] ##
configure: WARNING: ## ----------------------------------- ##

如何解决intercept先关闭后又快速启动，系统报can not bind:: Address already in use的问题

tcpcopy和intercept产生的日志文件应该不同，方便用户提交日志文件

An example: migrating memcached

http://globaldev.co.uk/2013/01/migrating-memcached/

tcpcopy将在如下方面进行性能优化

采用内存池，降低系统调用，快速抓包，快速截获响应包，减少不必要计算等

syscall 疑问

void tcp_copy_over(const int sig)
{
long int pid = (long int)syscall(224);
log_info(LOG_WARN, "sig %d received, pid=%ld", sig, pid);
sync();
exit(EXIT_SUCCESS);
}

代码manager.c中，调用函数syscall直接传入整数224。我想这里的意思是syscall(SYS_gettid)，SYS_gettid在不同的平台值不一样，比如32位的和64位的不一样， 如下
unistd_32.h:232:#define __NR_gettid 224
unistd_64.h:436:#define __NR_gettid 186

集成udpcopy到tcpcopy中去

离线回放所需要的pcap数据，可不可以从ip层抓，而不是数据链路层抓

sh autogen.sh运行遇到障碍

[wangbin@bj250-217 tcpcopy]$ sh autogen.sh
autogen.sh: line 1: autoreconf: command not found

需要安装autoconf
yum install autoconf

继续执行：
[wangbin@bj250-217 tcpcopy]$ sh autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --output=aclocal.m4t
Can't exec "aclocal": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 326.
autoreconf: failed to run aclocal: No such file or directory

需要安装automake
yum install automake

继续执行：
[wangbin@bj250-217 tcpcopy]$ sh autogen.sh
autoreconf: Entering directory .' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal --force configure.ac:20: warning: macroAM_PROG_LIBTOOL' not found in library
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
configure.ac:19: error: possibly undefined macro: AC_PROG_LIBTOOL
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:20: error: possibly undefined macro: AM_PROG_LIBTOOL
autoreconf: /usr/bin/autoconf failed with exit status: 1

需要安装libtool
yum install libtool

继续执行，就ok了
[wangbin@bj250-217 tcpcopy]$ sh autogen.sh
autoreconf: Entering directory .' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal --force autoreconf: configure.ac: tracing autoreconf: running: libtoolize --copy --force libtoolize: putting auxiliary files in.'.
libtoolize: copying file ./ltmain.sh' libtoolize: Consider addingAC_CONFIG_MACRO_DIR([m4])' to configure.ac and
libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
libtoolize: Consider adding -I m4' to ACLOCAL_AMFLAGS in Makefile.am. autoreconf: running: /usr/bin/autoconf --force autoreconf: running: /usr/bin/autoheader --force autoreconf: running: automake --add-missing --copy --force-missing configure.ac:19: installing./config.guess'
configure.ac:19: installing ./config.sub' configure.ac:10: installing./install-sh'
configure.ac:10: installing ./missing' src/communication/Makefile.am: installing./depcomp'
Makefile.am: installing ./INSTALL' autoreconf: Leaving directory.'

tcpcopy 同步几秒钟后就没反应了 是什么问题呢

请教一下 刚在线上测试了一下tcpcopy ，每次程序启动以后 数据传输接近1分钟左右就停止了，很奇怪连续测试了20次都是同样的问题。
测试环境 Cent os 6.3 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
两台机器都一样，分别部署在不同机房。
使用的都是nginx 服务.
tcpcopy 服务器IP 182.118.23.5
client 服务器IP 125.39.31.146
以下是tcpcopy 的一部分 error.log
2012/12/11 22:22:57 +554 [notice] active:283,rel reqs:52,obs del:18
2012/12/11 22:22:57 +554 [notice] conns:184,resp packs:42205,c-resp packs:41832
2012/12/11 22:22:57 +554 [notice] send Packets:42676,send content packets:242
2012/12/11 22:22:57 +554 [notice] reconnect for closed :0,for no syn:112
2012/12/11 22:22:57 +554 [notice] retransmit:0
2012/12/11 22:22:57 +554 [notice] successful retransmit:0
2012/12/11 22:22:57 +554 [notice] syn cnt:224,all clt packs:344025,clt cont:427
2012/12/11 22:22:57 +554 [warn] many connections can't be established
2012/12/11 22:22:57 +554 [notice] session size:283
2012/12/11 22:22:57 +554 [warn] unsend:too many packets:292,p:1382
2012/12/11 22:22:57 +554 [warn] unsend:candidate erased:270,p:7605
2012/12/11 22:22:57 +554 [warn] unsend:too many packets:283,p:59829
2012/12/11 22:22:57 +554 [warn] unsend:too many packets:306,p:1213
2012/12/11 22:22:57 +554 [warn] unsend:candidate erased:281,p:52157
2012/12/11 22:22:57 +554 [notice] activate_dead_sessions
2012/12/11 22:23:03 +054 [notice] active:272,rel reqs:75,obs del:41
2012/12/11 22:23:03 +054 [notice] conns:184,resp packs:42205,c-resp packs:41832
2012/12/11 22:23:03 +054 [notice] send Packets:42713,send content packets:242
2012/12/11 22:23:03 +054 [notice] reconnect for closed :0,for no syn:121
2012/12/11 22:23:03 +054 [notice] retransmit:0
2012/12/11 22:23:03 +054 [notice] successful retransmit:0
2012/12/11 22:23:03 +054 [notice] syn cnt:227,all clt packs:354132,clt cont:439
2012/12/11 22:23:03 +054 [warn] many connections can't be established
2012/12/11 22:23:03 +054 [notice] session size:272
2012/12/11 22:23:03 +054 [warn] unsend:candidate erased:260,p:10777
2012/12/11 22:23:03 +054 [warn] unsend:candidate erased:285,p:2445
2012/12/11 22:23:03 +054 [warn] unsend:too many packets:293,p:7605
2012/12/11 22:23:03 +054 [warn] unsend:too many packets:304,p:52157
2012/12/11 22:23:03 +054 [warn] unsend:candidate erased:257,p:1988
2012/12/11 22:23:03 +054 [warn] unsend:candidate erased:260,p:25054
2012/12/11 22:23:03 +054 [notice] activate_dead_sessions
2012/12/11 22:23:08 +354 [notice] active:260,rel reqs:100,obs del:65
2012/12/11 22:23:08 +354 [notice] conns:184,resp packs:42205,c-resp packs:41832
2012/12/11 22:23:08 +354 [notice] send Packets:42745,send content packets:242
2012/12/11 22:23:08 +354 [notice] reconnect for closed :0,for no syn:127
2012/12/11 22:23:08 +354 [notice] retransmit:0
2012/12/11 22:23:08 +354 [notice] successful retransmit:0
2012/12/11 22:23:08 +354 [notice] syn cnt:234,all clt packs:362966,clt cont:450
2012/12/11 22:23:08 +354 [warn] many connections can't be established
2012/12/11 22:23:08 +354 [notice] session size:260
2012/12/11 22:23:08 +354 [warn] unsend:too many packets:270,p:10777
2012/12/11 22:23:08 +354 [warn] unsend:too many packets:290,p:2445
2012/12/11 22:23:08 +354 [warn] unsend:candidate erased:274,p:27047
2012/12/11 22:23:08 +354 [warn] unsend:too many packets:268,p:1988
2012/12/11 22:23:08 +354 [warn] unsend:too many packets:275,p:25054
2012/12/11 22:23:08 +354 [notice] activate_dead_sessions
2012/12/11 22:23:13 +534 [notice] active:253,rel reqs:121,obs del:86
2012/12/11 22:23:13 +534 [notice] conns:184,resp packs:42205,c-resp packs:41832
2012/12/11 22:23:13 +534 [notice] send Packets:42780,send content packets:242
2012/12/11 22:23:13 +534 [notice] reconnect for closed :0,for no syn:134
2012/12/11 22:23:13 +534 [notice] retransmit:0
2012/12/11 22:23:13 +534 [notice] successful retransmit:0
2012/12/11 22:23:13 +534 [notice] syn cnt:241,all clt packs:375002,clt cont:464
2012/12/11 22:23:13 +534 [warn] many connections can't be established
2012/12/11 22:23:13 +534 [notice] session size:253
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:268,p:59426
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:277,p:4901
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:265,p:58154
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:263,p:3943
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:261,p:1654
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:278,p:1924
2012/12/11 22:23:13 +534 [warn] unsend:too many packets:296,p:27047
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:273,p:2491
2012/12/11 22:23:13 +534 [warn] unsend:candidate erased:276,p:35551
2012/12/11 22:23:13 +534 [notice] activate_dead_sessions
2012/12/11 22:23:19 +554 [notice] active:173,rel reqs:202,obs del:167
2012/12/11 22:23:19 +554 [notice] conns:184,resp packs:42205,c-resp packs:41832
2012/12/11 22:23:19 +554 [notice] send Packets:42862,send content packets:242
2012/12/11 22:23:19 +554 [notice] reconnect for closed :0,for no syn:134
2012/12/11 22:23:19 +554 [notice] retransmit:0
2012/12/11 22:23:19 +554 [notice] successful retransmit:0
2012/12/11 22:23:19 +554 [notice] syn cnt:242,all clt packs:377003,clt cont:464
2012/12/11 22:23:19 +554 [notice] session size:173
2012/12/11 22:23:19 +554 [warn] unsend:too many packets:271,p:59426
2012/12/11 22:23:19 +554 [warn] unsend:too many packets:278,p:4901
2012/12/11 22:23:19 +554 [warn] unsend:too many packets:267,p:58154
2012/12/11 22:23:19 +554 [warn] unsend:too many packets:267,p:3943
2012/12/11 22:23:19 +554 [warn] unsend:too many packets:283,p:1924
2012/12/11 22:23:19 +554 [warn] unsend:too many packets:280,p:35551
2012/12/11 22:23:19 +554 [notice] activate_dead_sessions
2012/12/11 22:23:25 +554 [notice] active:160,rel reqs:220,obs del:185
2012/12/11 22:23:25 +554 [notice] conns:184,resp packs:42205,c-resp packs:41832
2012/12/11 22:23:25 +554 [notice] send Packets:42885,send content packets:242
2012/12/11 22:23:25 +554 [notice] reconnect for closed :0,for no syn:139
2012/12/11 22:23:25 +554 [notice] retransmit:0
2012/12/11 22:23:25 +554 [notice] successful retransmit:0
2012/12/11 22:23:25 +554 [notice] syn cnt:242,all clt packs:379676,clt cont:470
2012/12/11 22:23:25 +554 [notice] session size:160
2012/12/11 22:23:25 +554 [notice] activate_dead_sessions

下面是 client server 的 intercept 的error.log
2012/12/11 22:22:30 +305 [notice] total response packets:469664
2012/12/11 22:22:30 +305 [notice] router size:215
2012/12/11 22:22:30 +305 [notice] router delete obsolete:1
2012/12/11 22:22:30 +305 [notice] delay total:77
2012/12/11 22:22:30 +305 [notice] delay delete obsolete :31
2012/12/11 22:22:36 +305 [notice] fd is null after session is created
2012/12/11 22:22:36 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2467,len 40,seq=3462026735,ack=1269695713,win:29440
2012/12/11 22:22:36 +305 [notice] fd is null after session is created
2012/12/11 22:22:36 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2466,len 40,seq=2428322404,ack=2240552728,win:29440
2012/12/11 22:22:36 +305 [notice] fd is null after session is created
2012/12/11 22:22:36 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2465,len 40,seq=4029497908,ack=2883825381,win:29440
2012/12/11 22:22:36 +305 [notice] fd is null after session is created
2012/12/11 22:22:36 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2464,len 40,seq=2161412811,ack=3876925621,win:29440
2012/12/11 22:22:46 +305 [notice] total response packets:470002
2012/12/11 22:22:46 +305 [notice] router size:259
2012/12/11 22:22:46 +305 [notice] router delete obsolete:1
2012/12/11 22:22:46 +305 [notice] delay total:54
2012/12/11 22:22:46 +305 [notice] delay delete obsolete :24
2012/12/11 22:22:52 +305 [notice] fd is null after session is created
2012/12/11 22:22:52 +305 [notice] from bak:125.39.31.146:80-->125.44.225.145:3085,len 40,seq=2038616683,ack=1744867950,win:29440
2012/12/11 22:22:52 +305 [notice] fd is null after session is created
2012/12/11 22:22:52 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2467,len 40,seq=3462026735,ack=1269695713,win:29440
2012/12/11 22:22:52 +305 [notice] fd is null after session is created
2012/12/11 22:22:52 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2466,len 40,seq=2428322404,ack=2240552728,win:29440
2012/12/11 22:22:52 +305 [notice] fd is null after session is created
2012/12/11 22:22:52 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2465,len 40,seq=4029497908,ack=2883825381,win:29440
2012/12/11 22:22:52 +305 [notice] fd is null after session is created
2012/12/11 22:22:52 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2464,len 40,seq=2161412811,ack=3876925621,win:29440
2012/12/11 22:22:53 +305 [notice] fd is null after session is created
2012/12/11 22:22:53 +305 [notice] from bak:125.39.31.146:80-->125.44.225.145:3085,len 40,seq=2038616683,ack=1744867950,win:29440
2012/12/11 22:22:55 +305 [notice] fd is null after session is created
2012/12/11 22:22:55 +305 [notice] from bak:125.39.31.146:80-->125.44.225.145:3085,len 40,seq=2038616683,ack=1744867950,win:29440
2012/12/11 22:23:02 +305 [notice] total response packets:470241
2012/12/11 22:23:02 +305 [notice] router size:266
2012/12/11 22:23:02 +305 [notice] router delete obsolete:0
2012/12/11 22:23:02 +305 [notice] delay total:55
2012/12/11 22:23:02 +305 [notice] delay delete obsolete :19
2012/12/11 22:23:02 +305 [notice] fd is null after session is created
2012/12/11 22:23:02 +305 [notice] from bak:125.39.31.146:80-->115.60.41.244:51456,len 40,seq=3900251137,ack=1196114717,win:31488
2012/12/11 22:23:07 +305 [notice] fd is null after session is created
2012/12/11 22:23:07 +305 [notice] from bak:125.39.31.146:80-->219.157.188.8:1140,len 40,seq=1409076880,ack=1129932192,win:59456
2012/12/11 22:23:19 +305 [notice] total response packets:470378
2012/12/11 22:23:19 +305 [notice] router size:178
2012/12/11 22:23:19 +305 [notice] router delete obsolete:0
2012/12/11 22:23:19 +305 [notice] delay total:51
2012/12/11 22:23:19 +305 [notice] delay delete obsolete :5
2012/12/11 22:23:21 +305 [notice] fd is null after session is created
2012/12/11 22:23:21 +305 [notice] from bak:125.39.31.146:80-->182.119.119.66:62775,len 40,seq=2701393524,ack=841231623,win:47164
2012/12/11 22:23:26 +305 [notice] fd is null after session is created
2012/12/11 22:23:26 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2467,len 40,seq=3462026735,ack=1269695713,win:29440
2012/12/11 22:23:26 +305 [notice] fd is null after session is created
2012/12/11 22:23:26 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2466,len 40,seq=2428322404,ack=2240552728,win:29440
2012/12/11 22:23:26 +305 [notice] fd is null after session is created
2012/12/11 22:23:26 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2465,len 40,seq=4029497908,ack=2883825381,win:29440
2012/12/11 22:23:26 +305 [notice] fd is null after session is created
2012/12/11 22:23:26 +305 [notice] from bak:125.39.31.146:80-->125.40.234.255:2464,len 40,seq=2161412811,ack=3876925621,win:29440
2012/12/11 22:23:39 +305 [notice] total response packets:470399
2012/12/11 22:23:39 +305 [notice] router size:148
2012/12/11 22:23:39 +305 [notice] router delete obsolete:0
2012/12/11 22:23:39 +305 [notice] delay total:54
2012/12/11 22:23:39 +305 [notice] delay delete obsolete :37
2012/12/11 22:23:51 +305 [notice] fd is null after session is created
2012/12/11 22:23:51 +305 [notice] from bak:125.39.31.146:80-->219.157.188.8:1140,len 40,seq=1409076880,ack=1129932192,win:59456
2012/12/11 22:23:55 +305 [notice] total response packets:470420
2012/12/11 22:23:55 +305 [notice] router size:148
2012/12/11 22:23:55 +305 [notice] router delete obsolete:0
2012/12/11 22:23:55 +305 [notice] delay total:21
2012/12/11 22:23:55 +305 [notice] delay delete obsolete :12
2012/12/11 22:24:39 +305 [notice] total response packets:470426
2012/12/11 22:24:39 +305 [notice] router size:148
2012/12/11 22:24:39 +305 [notice] router delete obsolete:44
2012/12/11 22:24:39 +305 [notice] delay total:10
2012/12/11 22:24:39 +305 [notice] delay delete obsolete :9
2012/12/11 22:24:55 +305 [notice] total response packets:470458
2012/12/11 22:24:55 +305 [notice] router size:104
2012/12/11 22:24:55 +305 [notice] router delete obsolete:40
2012/12/11 22:24:55 +305 [notice] delay total:6
2012/12/11 22:24:55 +305 [notice] delay delete obsolete :0
2012/12/11 22:25:39 +305 [notice] total response packets:470464
2012/12/11 22:25:39 +305 [notice] router size:64
2012/12/11 22:25:39 +305 [notice] router delete obsolete:64
2012/12/11 22:25:39 +305 [notice] delay total:7
2012/12/11 22:25:39 +305 [notice] delay delete obsolete :6
2012/12/11 22:25:55 +305 [notice] total response packets:470487
2012/12/11 22:25:55 +305 [notice] router size:0
2012/12/11 22:25:55 +305 [notice] router delete obsolete:0
2012/12/11 22:25:55 +305 [notice] delay total:6
2012/12/11 22:25:55 +305 [notice] delay delete obsolete :0
2012/12/11 22:26:39 +305 [notice] total response packets:470504
2012/12/11 22:26:39 +305 [notice] router size:0
2012/12/11 22:26:39 +305 [notice] router delete obsolete:0
2012/12/11 22:26:39 +305 [notice] delay total:7
2012/12/11 22:26:39 +305 [notice] delay delete obsolete :6
2012/12/11 22:26:55 +305 [notice] total response packets:470528
2012/12/11 22:26:55 +305 [notice] router size:0
2012/12/11 22:26:55 +305 [notice] router delete obsolete:0
2012/12/11 22:26:55 +305 [notice] delay total:6
2012/12/11 22:26:55 +305 [notice] delay delete obsolete :0
2012/12/11 22:27:39 +305 [notice] total response packets:470546
2012/12/11 22:27:39 +305 [notice] router size:0
2012/12/11 22:27:39 +305 [notice] router delete obsolete:0
2012/12/11 22:27:39 +305 [notice] delay total:7
2012/12/11 22:27:39 +305 [notice] delay delete obsolete :6
2012/12/11 22:27:55 +305 [notice] total response packets:470570
2012/12/11 22:27:55 +305 [notice] router size:0
2012/12/11 22:27:55 +305 [notice] router delete obsolete:0
2012/12/11 22:27:55 +305 [notice] delay total:6
2012/12/11 22:27:55 +305 [notice] delay delete obsolete :0
2012/12/11 22:28:39 +305 [notice] total response packets:470588
2012/12/11 22:28:39 +305 [notice] router size:0
2012/12/11 22:28:39 +305 [notice] router delete obsolete:0
2012/12/11 22:28:39 +305 [notice] delay total:7
2012/12/11 22:28:39 +305 [notice] delay delete obsolete :6
2012/12/11 22:28:55 +305 [notice] total response packets:470612
2012/12/11 22:28:55 +305 [notice] router size:0
2012/12/11 22:28:55 +305 [notice] router delete obsolete:0
2012/12/11 22:28:55 +305 [notice] delay total:6
2012/12/11 22:28:55 +305 [notice] delay delete obsolete :0
2012/12/11 22:29:39 +305 [notice] total response packets:470630
2012/12/11 22:29:39 +305 [notice] router size:0
2012/12/11 22:29:39 +305 [notice] router delete obsolete:0
2012/12/11 22:29:39 +305 [notice] delay total:7
2012/12/11 22:29:39 +305 [notice] delay delete obsolete :6
2012/12/11 22:30:39 +305 [notice] total response packets:470652
2012/12/11 22:30:39 +305 [notice] router size:0
2012/12/11 22:30:39 +305 [notice] router delete obsolete:0
2012/12/11 22:30:39 +305 [notice] delay total:5
2012/12/11 22:30:39 +305 [notice] delay delete obsolete :4
2012/12/11 22:30:55 +305 [notice] total response packets:470676
2012/12/11 22:30:55 +305 [notice] router size:0
2012/12/11 22:30:55 +305 [notice] router delete obsolete:0
2012/12/11 22:30:55 +305 [notice] delay total:6
2012/12/11 22:30:55 +305 [notice] delay delete obsolete :0
2012/12/11 22:31:39 +305 [notice] total response packets:470694
2012/12/11 22:31:39 +305 [notice] router size:0
2012/12/11 22:31:39 +305 [notice] router delete obsolete:0
2012/12/11 22:31:39 +305 [notice] delay total:7
2012/12/11 22:31:39 +305 [notice] delay delete obsolete :6
2012/12/11 22:31:55 +305 [notice] total response packets:470726
2012/12/11 22:31:55 +305 [notice] router size:0
2012/12/11 22:31:55 +305 [notice] router delete obsolete:0
2012/12/11 22:31:55 +305 [notice] delay total:6
2012/12/11 22:31:55 +305 [notice] delay delete obsolete :0

还请wangbin 帮忙看看是啥问题导致的。
另外机房上层初步分析有强制cache 设备不知道是不是由于上层cache 设备导致的 情况。

x参数无效

wangbin你好，请教个问题：

场景大概是，我的目标机子是B，从A引入流量。此时我又希望B能被C访问，
于是intercept形如：intercept -x C。
启动后，C可以访问B了，但从A的copy却失败了，
修改为：intercept -x B, C也一样无效。

机器间无法copy

现有两台机器192.168.1.101 192.168.1.102都运行memcached

configure
make
没有make install

running steps:
192.168.1.102
modprobe ip_queue
iptables -I OUTPUT -p tcp --sport 11211 -j QUEUE
./interception
192.168.1.101
./tcpcopy 192.168.1.101 11211 192.168.1.102 11211

telnet 192.168.1.101 11211
192.168.1.102上无反映，没有收到数据

如果使用一台机器的不同端口启动memcached就可以copy成功数据。例如./tcpcopy 192.168.1.101 11212 192.168.1.101 11211

请教您的经验会有什么可能？

pcap库的问题

pcap貌似是支持离线回放功能提供的一个库，现在似乎是强制依赖的，即使我不使用这个功能。这需要做成当用户编译此功能的时候才去依赖这个库。

最新内核3.2系列，netlink出现不能与ip queue进行通信的问题，需要进一步确认

TODO,需要安装最新的内核,试验一下

未来需要对ipv6的支持，以支持ipv4网络迁移到ipv6的需求

实现ipv6的好处有如下：
1）未来ipv6的应用
2）ipv4网络迁移到ipv6网络，已解决server迁移过程中可能会出现的问题
Over the coming years, there will probably be a gradual transition of the Internet from IPv4 to IPv6. During this transition phase, it is important that existing IPv4 applications continue to work with newer IPv6 applications.

用 centos 5.x 的系统，autoconf 那个环节有问题，无法生成 configure

mark一下

Red Hat Enterprise Linux Server release 6.3 (Santiago)运行intercept,出现复制过去的in数据包不能通过ip 层的问题（rpfilter的问题）

需要安装环境测试一下

如何找到netlink的替代方案

需要更加通用，更加高效的机制

tcpcopy相关博客地址

http://blog.csdn.net/wangbin579/article/category/926096

内核同时有nfqueue和ip queue的情况下，所有tcp状态都处于SYN_RECV，怎么回事？

是因为nfqueue干扰了ip queue模块的运行，使intercept接受不到响应包，导致都处于SYN_RECV状态。

先关闭tcpcopy和intercept程序，然后，采用如下方案：

1）方案1
卸载nfqueue相关内核模块
重启ip queue

比如：

modprobe -r nfnetlink_queue
modprobe -r ip_queue
modprobe ip_queue

2）方案2

采用nfqueue方式，不要采用ip queue

请问有关于mysql的文档么？

0.6.5的pdf关于mysql的使用实在是太少了。请问又更详细的文档么?

tcpcopy在freebsd，windows平台下的支持

见下面方法

seq回绕的问题，需要特殊处理一下

intercept写成内核模块的可行性分析

tcpcopy暂时用用户态程序，写成内核模块是有很大风险的，但intercept一般运行在测试服务器上面，因此可以先尝试这方面的探索

需要兼容nfqueue和netfilter_queue，在ip queue不能使用的情况下

部分虚拟机器，数据包已经达到数据链路层，但请求不能顺利达到上层应用的问题

代码风格

更随nginx风格确实不错，个人认为tcpcopy其实可以在某些方面保持自己的风格。。。当然这不是问题。

建议命名有一个全局的前缀，（比如tc，或者其他），这将有一个非常的好处，避免一些基础代码的文件和标准库冲突，比如：time.h, string.h这样的，加上自己的前缀，可能就是tc_time.h, tc_string.h。。。这样就可以实现

include <tc_time.h>而不是采用#include "../core/time.h"。

我觉得tc好的原因是，做好tcpcopy就非常不错了，在tcpcopy的基础上进行模块化开发出mysql, http等基于tcpcopy的应用协议，如果tcpcopy的模块做得好，完全可以由用户自己去开发自己的应用协议模块。 xcopy这个名字感觉还是没有tcpcopy拉风。以后真需要udp的话，大不了根据tcpcopy延伸出一个子项目udpcopy。毕竟还是tcp用得更多。

如何找到老的tcpcopy版本（0.7.0以下版本）

老的tcpcopy版本（0.7.0以下版本），在https://github.com/wangbin579/tcpcopy/downloads 目录下，但不推荐使用

如何降低抓包丢失率

可以试验一下pf_ring或者netmap，查看抓包的丢失率

如果系统报ip_conntrack: table full, dropping packet，那么如何解决？

在在线服务器内，由于ip_conntrack是在ip层中做的，而tcpcopy转发数据包，会走部分ip函数，导致了ip_conntrack瞎推测连接状态，为了解决此问题，可以采用如下方案：
1）去除ip_conntrack内核模块
modprobe -r ip_conntrack
如果报FATAL错误，那么可以这样：
service iptables stop
再执行此即可

2）设置对tcpcopy转发的包不进行track

如果需要对在线机器进行操作：
iptables -t raw -A OUTPUT -p tcp --dport 转发目的端口 -j NOTRACK

如果需要针对测试服务器，那么可以设置如下：
iptables -t raw -A PREROUTING -p tcp --dport 转发目的端口 -j NOTRACK
iptables -t raw -A OUTPUT -p tcp --sport 转发目的端口 -j NOTRACK

3)加大 ip_conntrack_max 值
echo "262144" > /proc/sys/net/ipv4/ip_conntrack_max

4)采用--enable-dlinject从数据链路层发包，避开ip_conntrack的检验

目前tcpcopy的测试方法，欢迎提出改进意见

由于tcpcopy是针对server进行测试的，而互联网的server种类繁多，有数据库，应用服务器，web服务器等等，所以要想达到充分测试是非常有难度的。

目前回归测试所用到的项目列表如下：
1）geoserver
2）jackrabbit-webapp-2.2.5
3）jetspeed
4）resin-doc
5）struts2-showcase

这些项目用于功能回归测试，效果比较理想，覆盖了大部分web应用场合。

目前采用tcpcopy的离线功能进行回归测试，所有访问都在pcap文件中（pcap文件总共包含6702次请求访问）。

在开发机上进行回归测试中没有出现请求丢失，则回归测试通过，否则不通过。

经过上述功能测试，基本可以确保tcpcopy没有大的问题，就可以git push操作。

进行功能测试以后，进行实际项目的回归测试和mysql回归测试，由于tcpcopy可以提供的configure选项特别多，这些测试过程会经历比较长的时间，目的是确保运行稳定性和性能没有出现大的下降，请求丢失率是否保持在较低的水准。

目前很少对传统压力测试工具发出的请求进行回归测试，因为这些压力测试工具发出的请求与在线的情况非常不一样，很多时候需要很好的优化系统参数，才能达到好的效果，否则丢失率会很高。

hash表实现问题

看到hash表中有不少地方出错后直接调用了exit来退出程序，我觉得这样写程序可能不是特别好，我们有理由认为hash表是我们的基础代码，类似库，库提供的接口最好只反应成功和失败等，而不能去决定是否退出程序。。。我们最好做到所有的退出情况都得返回到main函数后执行，当然coredump除外。只要做到这样子后，其实根本不需要注册退出处理的回调函数了。

架构大调整

要对tcpcopy进行架构大调整了，未来的架构将更加具有可扩展性，模拟在线的情况将更加逼真，而且将绕过tcp/ip协议栈，使其具有巨大的优化空间，测试服务器将不再受到之前的ip queue或者nfqueue的干扰，将和在线一模一样，测试server 10万级别的qps将不再是梦想。

这可能是tcpcopy的终极版本

需要对raw socket和tcp连接（tcpcopy和intercept之间的连接）进行必要的close操作

感谢郑明杭的bug提交

感谢分享

感谢mr.王无私分享

message too long

hi wangbin
在使用的过程中出现了很多 message too long，测试的服务器接收到的数据就不完整。

[warn] Thu Jun 14 16:45:12 2012 usec=107522 tot_len is wrong:14532

请问现在怎么能解决这个问题呢？

client 端不能正常启用

git clone 最新版安装后,

1. 服务端正常起动 ./intercept
   2.client 端
   ./tcpcopy 192.168.2.153 80 192.168.2.191 80
   no -x argument

3../tcpcopy -x 80-192.168.2.191 80

2012/09/18 16:34:57 +399 [error] it must have -x argument
2012/09/18 16:35:13 +235 [notice] tcpcopy version:0.6.1
2012/09/18 16:35:13 +235 [notice] target:80-192.168.2.191
2012/09/18 16:35:13 +235 [notice] set global port for tcpcopy
2012/09/18 16:35:13 +235 [notice] set global port for tcpcopy
2012/09/18 16:35:13 +235 [notice] create address-table, size:32
2012/09/18 16:35:13 +235 [error] Can not connect to remote server(0:36524) (Connection refused)

iptables 设置后端口不通，无法复制数据，求解

在tcp server端执行
sudo modprobe ip_queue
sudo iptables -I OUTPUT -p tcp --sport 80 -j QUEUE
sudo /usr/local/bin/intercept -d -l /home/ww/tcpcopy/run.log

iptables -L 后显示如下信息
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:80

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
QUEUE tcp -- anywhere anywhere tcp spt:80
ACCEPT tcp -- anywhere anywhere tcp dpt:80

此时无法telnet 此机器的80端口，80端口也无任何流量被复制过来

如果删除-I OUTPUT -p tcp --sport 80 -j QUEUE此条iptables策略，telnet此机器的80端口是ok的

不太清楚为什么

intercept 0.6.6版本出现死循环

intercept 0.6.6版本出现死循环，strace结果只有recvfrom和select，需要进一步分析

需要在configure.ac文件中增加--with-pcap方式

configure: WARNING: unrecognized options: --enable-mysqlsgt问题

目前原因未知，出现过两例，其中一例是CentOS release 6.3 的系统，待解决，先mark一下

目前土方法如下：
修改src/core/config.h文件，直接指定

#define TCPCOPY_MYSQL_SKIP 1

当然上述要想有效，要满足如下条件：

如果存在
#define TCPCOPY_MYSQL_NO_SKIP 1
那么必须改为
#define TCPCOPY_MYSQL_NO_SKIP 0

针对离线回放，需要实现加速回放功能，以节省测试时间

已经有两个用户提出这样的需求，预计清明节后实现

3.5 内核以后没有 linux/netfilter_ipv4/ip_queue.h

0.6.5 编译过程出现错误提示，找不到 linux/netfilter_ipv4/ip_queue.h 文件。

流量回访失败-求解

两台机器
192.168.0.219端口9999
192.168.0.220端口9999

在220上启动tcpcopy server
intercept -x 192.168.0.219 -l tran.log

在219机器上记录流量
tcpdump tcp dst port 9999 -vv -w /data/output.dat

流量记录完毕后，进行回放
tcpcopy -i /data/output.dat -x 9999:192.168.0.220:9999 -l tran.log

可220机器没有收到请求
两机的9999端口可以telnet通，220的36524端口可以telnet通
双机都是用
./configure --enable-offline进行编译
安装
libpcap-devel-0.9.4-15
libpcap-0.9.4-15
centos5 2.6.18-308.13.1.el5

server的错误
2012/12/31 16:44:57 +313 [notice] intercept version:0.6.6
2012/12/31 16:44:57 +313 [notice] -x parameter:192.168.0.219
2012/12/31 16:44:57 +313 [notice] create delay-table,size:65536
2012/12/31 16:44:57 +313 [notice] create router-table, size:131072
2012/12/31 16:44:57 +313 [notice] msg listen socket:4
2012/12/31 16:44:57 +313 [notice] firewall socket:5
2012/12/31 16:45:16 +315 [error] set signal handler:2
2012/12/31 16:45:16 +315 [notice] release_resources begin
2012/12/31 16:45:16 +315 [notice] destroy router table
2012/12/31 16:45:16 +315 [notice] fd null counter:0
2012/12/31 16:45:16 +315 [notice] total visit hash_find_node:0,compared:0
2012/12/31 16:45:16 +315 [notice] destroy items 0 in table name:router-table
2012/12/31 16:45:16 +315 [notice] destroy delay table,total:0
2012/12/31 16:45:16 +315 [notice] destroy items:0,free:0,total:0
2012/12/31 16:45:16 +315 [notice] create msg list:0,free:0,destr:0
2012/12/31 16:45:16 +315 [notice] total visit hash_find_node:0,compared:0
2012/12/31 16:45:16 +315 [notice] destroy items 0 in table name:delay-table
2012/12/31 16:45:16 +315 [notice] release_resources end except log file

clienkt的log
2012/12/31 16:47:47 +364 [notice] TCPCopy version:0.6.6
2012/12/31 16:47:47 +364 [notice] target:9999:192.168.0.220:9999
2012/12/31 16:47:47 +364 [notice] TCPCOPY_OFFLINE mode
2012/12/31 16:47:47 +364 [warn] target "9999:192.168.0.220:9999" is invalid
2012/12/31 16:47:47 +364 [notice] create address-table, size:32
2012/12/31 16:47:47 +364 [error] Can not connect to remote server(63:36524) (Connection timed out)

问个弱弱的问题

如果 被测试的机器的port 为80 那么。我就不能访问80了。通过正常的http 方式。有什么办法解决没有

当关闭tcpcopy client的时候，intercept会调用tc_event_del，但这时候却没有释放相应的ev，导致内存泄漏

single分支：由于机器太差，无法针对多线程并且single模式的tcpcopy进行性能充分测试

等有好机器的时候，针对
./configure --enable-single --enable-thread
进行公平的性能测试