hobama / tcpcopy Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/tcpcopy
License: Other
Automatically exported from code.google.com/p/tcpcopy
License: Other
Name: tcpcopy It is an online request replication tool and is mainly for testing tasks using netlink and raw sockets Description: It can help you find bugs in your online project without actually being online. And it can also be used to test the stress that a system could endure. For example, if your system already has "memcached" subsystem and you want to use "membase" to replace it, tcpcopy can assist you to test "membase". While your old memcached system is still running online, tcpcopy could copy the flow of packets from memcached to membase. From the point view of membase, the flow is accessing membase(just like membase online), and it will not affect memcached at all except network bandwidth and a little cpu load. Functionalities: 1) Distributed Stress Test You can use online data to test the stress that your target machine can endure. It is better than apache ab tool and you can find bugs that only occur during high-stress situations. 2) Hot Backup It is very suitable for backup tasks if connections are short-lived and the request loss rate is very low(1/100000). 3) Normal Online Test You can find whether the new system is stable and find bugs that only occur in actual online environments. 4) Comparison Test For example, you can use tcpcopy to compare the performances of apache and nginx. Characteristics: 1)real time 2)realistic 3)efficient 4)easy to use 5)distributed 6)significant How to use tcpcopy? preparing steps: 1) download souce code 2)./configure 3)make 4)make install running steps: the target machine(root privilege): modprobe ip_queue (if not running) iptables -I OUTPUT -p tcp --sport port -j QUEUE (if not set) ./interception online machine(root privilege): ./tcpcopy local_ip1[:local_ip2:...] local_port remote_ip remote_port An example: Suppose 13 and 14 are online machines, 148 is a target machine which is similar to the online machines, and 12321 is used both as local port and remote port. We use tcpcopy to test if 148 can endure two times of current online stress. Using tcpcopy to perform the above test task. the target machine(148) # modprobe ip_queue (if not run up) # iptables -I OUTPUT -p tcp --sport 12321 -j QUEUE (if not set) # ./interception online machine(13): # ./tcpcopy xx.xx.xx.13 12321 xx.xx.xx.148 12321 online machine(14): # ./tcpcopy xx.xx.xx.14 12321 xx.xx.xx.148 12321 Cpu load and memory usage are as follows: 13 cpu: 11124 adrun 15 0 193m 146m 744 S 18.6 7.3 495:31.56 asyn_server 11281 root 15 0 65144 40m 1076 S 12.3 2.0 0:47.89 tcpcopy 14 cpu: 16855 adrun 15 0 98.7m 55m 744 S 21.6 2.7 487:49.51 asyn_server 16429 root 15 0 41156 17m 1076 S 14.0 0.9 0:33.63 tcpcopy 148 cpu : 25609 root 15 0 76892 59m 764 S 49.6 2.9 63:03.14 asyn_server 20184 root 15 0 5624 4232 292 S 17.0 0.2 0:52.82 interception Access log analysis: 13 online machine: grep 'Tue 11:08' access_0913_11.log |wc -l :89316, 1489 reqs/sec 14 online machine: grep 'Tue 11:08' access_0913_11.log |wc -l :89309, 1488 reqs/sec 148 test machine: grep 'Tue 11:08' access_0913_11.log |wc -l :178175, 2969 reqs/sec request loss rate: (89316+89309-178175)/(89316+89309)=0.25% From the above, we can see that the target machine can endure two times of current online stress. What about the cpu load ? tcpcopy on online machine 13 occupies 12.3% of cpu load, tcpcopy on online 14 occupies 14% and interception on target machine 148 occupies 17%. We can see that the cpu load is very low here, and so is the memory usage. Attention: 1) It is tested on machines running linux (kernal 2.6 or above). 2) tcpcopy is similar to udp, so sometimes it may lose requests. 3) If you tcpcopy local request, please set lo MTU not more than 1500. 4) Interception(tcpcopy server) is single threaded. 5) tcpcopy needs root privilege. 6) long request(such as uploading a large file) is not working well here(no retransmission when sending packets to the target test server) 7) Check error.log if you have trouble and email to me([email protected] or [email protected]).
一、操作步骤:
1、启用udp多份复制;
2、出现内存泄露;
二、版本
tcpcopy-0.8.0版本
三、问题代码
tc_packets_module.c/L291
if (replica_num > 1) {
packet_valid = process_packet(true, packet, recv_len);
replicate_packs(packet, recv_len, replica_num);
} else {
packet_valid = process_packet(false, packet, recv_len);
}
四、正确修复后的代码
if (replica_num > 1) {
replicate_packs(packet, recv_len, replica_num);
}
packet_valid = process_packet(false, packet, recv_len);
Original issue reported on code.google.com by [email protected]
on 21 Aug 2013 at 3:23
Attachments:
What steps will reproduce the problem?
1. I installed the tcpcopy on Ubuntu 11.10 x64 and i run command interception
but was returned a error:
"can not bind:: Address family not supported by protocol"
on the error.log
[error] Fri May 4 10:46:42 2012 usec=193142 it can not bind address
[notice] Fri May 4 10:46:42 2012 usec=193171 releaseResources begin
[notice] Fri May 4 10:46:42 2012 usec=193192 firewall sock is closed
[notice] Fri May 4 10:46:42 2012 usec=193198 msg listen sock is closed
[notice] Fri May 4 10:46:42 2012 usec=193202 destroy router table
[notice] Fri May 4 10:46:42 2012 usec=201567 destroy items 0 in table
name:client--src table
[notice] Fri May 4 10:46:42 2012 usec=201650 destroy delayed table
[notice] Fri May 4 10:46:42 2012 usec=202050 destroy msg list
items:0,free:0,total:0
[notice] Fri May 4 10:46:42 2012 usec=202096 create msg
list:0,delayDel:0,destroyList:0
[notice] Fri May 4 10:46:42 2012 usec=205036 destroy items 0 in table
name:delay-table
[notice] Fri May 4 10:46:42 2012 usec=205127 releaseResources end except log
file
[notice] Fri May 4 11:31:31 2012 usec=649195 create table
delay-table,size:65536
[notice] Fri May 4 11:31:31 2012 usec=668874 create table client--src
table,size:262144
[notice] Fri May 4 11:31:31 2012 usec=668980 socket created successfully
[error] Fri May 4 11:31:31 2012 usec=669325 it can not bind address
[notice] Fri May 4 11:31:31 2012 usec=669337 releaseResources begin
[notice] Fri May 4 11:31:31 2012 usec=669344 firewall sock is closed
[notice] Fri May 4 11:31:31 2012 usec=669349 msg listen sock is closed
[notice] Fri May 4 11:31:31 2012 usec=669353 destroy router table
[notice] Fri May 4 11:31:31 2012 usec=677835 destroy items 0 in table
name:client--src table
[notice] Fri May 4 11:31:31 2012 usec=677896 destroy delayed table
[notice] Fri May 4 11:31:31 2012 usec=678256 destroy msg list
items:0,free:0,total:0
[notice] Fri May 4 11:31:31 2012 usec=678265 create msg
list:0,delayDel:0,destroyList:0
[notice] Fri May 4 11:31:31 2012 usec=681284 destroy items 0 in table
name:delay-table
[notice] Fri May 4 11:31:31 2012 usec=681351 releaseResources end except log
file
[notice] Fri May 4 13:31:20 2012 usec=910078 create table
delay-table,size:65536
[notice] Fri May 4 13:31:20 2012 usec=930001 create table client--src
table,size:262144
[notice] Fri May 4 13:31:20 2012 usec=930102 socket created successfully
[error] Fri May 4 13:31:20 2012 usec=930424 it can not bind address
[notice] Fri May 4 13:31:20 2012 usec=932630 releaseResources begin
[notice] Fri May 4 13:31:20 2012 usec=932700 firewall sock is closed
[notice] Fri May 4 13:31:20 2012 usec=932731 msg listen sock is closed
[notice] Fri May 4 13:31:20 2012 usec=932758 destroy router table
[notice] Fri May 4 13:31:20 2012 usec=941125 destroy items 0 in table
name:client--src table
[notice] Fri May 4 13:31:20 2012 usec=941201 destroy delayed table
[notice] Fri May 4 13:31:20 2012 usec=941588 destroy msg list
items:0,free:0,total:0
[notice] Fri May 4 13:31:20 2012 usec=941630 create msg
list:0,delayDel:0,destroyList:0
[notice] Fri May 4 13:31:20 2012 usec=944584 destroy items 0 in table
name:delay-table
[notice] Fri May 4 13:31:20 2012 usec=944667 releaseResources end except log
file
What version of the product are you using? On what operating system?
I'm using tcpcopy 0.3.5 and Ubuntu 11.10 x64 Linux 3.0.0-12-server.
Original issue reported on code.google.com by [email protected]
on 4 May 2012 at 5:51
What steps will reproduce the problem?
1.
2.
3.
What is the expected output? What do you see instead?
Please use labels and text to provide additional information.
{{{
1、multiple ip addresses
if you have multiple ip addresses,then you can use the following method:
./tcpcopy ip1:ip2:ip3:... port1 destIP destPort
2、if you see msg as the following
ip_queue: full at 1024 entries, dropping packets(s)
you can change the default ip queue size:
echo 4096 > /proc/sys/net/ipv4/ip_queue_maxlen
}}}
Original issue reported on code.google.com by [email protected]
on 7 May 2012 at 12:49
Archlinux,gcc 4.6.2
make all-recursive
make[1]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1”
Making all in src
make[2]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
”
Making all in log
make[3]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/log”
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT log.o
-MD -MP -MF ".deps/log.Tpo" -c -o log.o log.c; \
then mv -f ".deps/log.Tpo" ".deps/log.Po"; else rm -f ".deps/log.Tpo"; exit 1;
fi
rm -f liblog.a
ar cru liblog.a log.o
ranlib liblog.a
make[3]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/log”
Making all in event
make[3]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/event”
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
select_server.o -MD -MP -MF ".deps/select_server.Tpo" -c -o select_server.o
select_server.c; \
then mv -f ".deps/select_server.Tpo" ".deps/select_server.Po"; else rm -f
".deps/select_server.Tpo"; exit 1; fi
rm -f libevent.a
ar cru libevent.a select_server.o ../log/liblog.a
ranlib libevent.a
make[3]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/event”
Making all in communication
make[3]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/communication”
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT msg.o
-MD -MP -MF ".deps/msg.Tpo" -c -o msg.o msg.c; \
then mv -f ".deps/msg.Tpo" ".deps/msg.Po"; else rm -f ".deps/msg.Tpo"; exit 1;
fi
rm -f libmsg.a
ar cru libmsg.a msg.o ../log/liblog.a
ranlib libmsg.a
make[3]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/communication”
Making all in interception
make[3]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/interception”
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
router.o -MD -MP -MF ".deps/router.Tpo" -c -o router.o router.c; \
then mv -f ".deps/router.Tpo" ".deps/router.Po"; else rm -f ".deps/router.Tpo";
exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
delay.o -MD -MP -MF ".deps/delay.Tpo" -c -o delay.o delay.c; \
then mv -f ".deps/delay.Tpo" ".deps/delay.Po"; else rm -f ".deps/delay.Tpo";
exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
hash.o -MD -MP -MF ".deps/hash.Tpo" -c -o hash.o hash.c; \
then mv -f ".deps/hash.Tpo" ".deps/hash.Po"; else rm -f ".deps/hash.Tpo"; exit
1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
interception.o -MD -MP -MF ".deps/interception.Tpo" -c -o interception.o
interception.c; \
then mv -f ".deps/interception.Tpo" ".deps/interception.Po"; else rm -f
".deps/interception.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
linklist.o -MD -MP -MF ".deps/linklist.Tpo" -c -o linklist.o linklist.c; \
then mv -f ".deps/linklist.Tpo" ".deps/linklist.Po"; else rm -f
".deps/linklist.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
nl_firewall.o -MD -MP -MF ".deps/nl_firewall.Tpo" -c -o nl_firewall.o
nl_firewall.c; \
then mv -f ".deps/nl_firewall.Tpo" ".deps/nl_firewall.Po"; else rm -f
".deps/nl_firewall.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT nl.o
-MD -MP -MF ".deps/nl.Tpo" -c -o nl.o nl.c; \
then mv -f ".deps/nl.Tpo" ".deps/nl.Po"; else rm -f ".deps/nl.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
main.o -MD -MP -MF ".deps/main.Tpo" -c -o main.o main.c; \
then mv -f ".deps/main.Tpo" ".deps/main.Po"; else rm -f ".deps/main.Tpo"; exit
1; fi
/bin/sh ../../libtool --mode=link --tag=CC gcc -march=x86-64 -mtune=generic
-O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2
-Wl,-O1,--sort-common,--as-needed,-z,relro,--hash-style=gnu -o interception
router.o delay.o hash.o interception.o linklist.o nl_firewall.o nl.o main.o
../event/libevent.a ../log/liblog.a ../communication/libmsg.a
mkdir .libs
gcc -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -Wl,-O1 -Wl,--sort-common
-Wl,--as-needed -Wl,-z -Wl,relro -Wl,--hash-style=gnu -o interception router.o
delay.o hash.o interception.o linklist.o nl_firewall.o nl.o main.o
../event/libevent.a ../log/liblog.a ../communication/libmsg.a
make[3]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/interception”
Making all in tcpcopy
make[3]:
进入目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/tcpcopy”
if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
address.o -MD -MP -MF ".deps/address.Tpo" -c -o address.o address.c; \
then mv -f ".deps/address.Tpo" ".deps/address.Po"; else rm -f
".deps/address.Tpo"; exit 1; fi
if g++ -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
send.o -MD -MP -MF ".deps/send.Tpo" -c -o send.o send.cpp; \
then mv -f ".deps/send.Tpo" ".deps/send.Po"; else rm -f ".deps/send.Tpo"; exit
1; fi
if g++ -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
session.o -MD -MP -MF ".deps/session.Tpo" -c -o session.o session.cpp; \
then mv -f ".deps/session.Tpo" ".deps/session.Po"; else rm -f
".deps/session.Tpo"; exit 1; fi
In file included from /usr/include/string.h:642:0,
from ../log/log.h:5,
from session.h:4,
from session.cpp:5:
在函数‘void* memset(void*, int, size_t)’中,
内联自‘void session_st::sendFakedSynAckToBackend(iphdr*, tcphdr*)’于 session.cpp:941:27:
/usr/include/bits/string3.h:82:32:
警告:调用‘__warn_memset_zero_len’,声明有警告属性:memset
used with constant zero length parameter; this could be due to transposed
parameters [默认启用]
在函数‘void* memset(void*, int, size_t)’中,
内联自‘void session_st::sendFakedAckToBackend(iphdr*, tcphdr*, bool)’于 session.cpp:978:27:
/usr/include/bits/string3.h:82:32:
警告:调用‘__warn_memset_zero_len’,声明有警告属性:memset
used with constant zero length parameter; this could be due to transposed
parameters [默认启用]
在函数‘void* memset(void*, int, size_t)’中,
内联自‘void session_st::sendFakedFinToBackend(iphdr*, tcphdr*)’于 session.cpp:1018:27:
/usr/include/bits/string3.h:82:32:
警告:调用‘__warn_memset_zero_len’,声明有警告属性:memset
used with constant zero length parameter; this could be due to transposed
parameters [默认启用]
在函数‘void* memset(void*, int, size_t)’中,
内联自‘void session_st::sendFakedFinToBackByCliePack(iphdr*, tcphdr*)’于 session.cpp:1060:27:
/usr/include/bits/string3.h:82:32:
警告:调用‘__warn_memset_zero_len’,声明有警告属性:memset
used with constant zero length parameter; this could be due to transposed
parameters [默认启用]
if g++ -DHAVE_CONFIG_H -I. -I. -I../.. -march=x86-64 -mtune=generic -O2
-pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -MT
main.o -MD -MP -MF ".deps/main.Tpo" -c -o main.o main.cpp; \
then mv -f ".deps/main.Tpo" ".deps/main.Po"; else rm -f ".deps/main.Tpo"; exit
1; fi
main.cpp: 在函数‘void checkMemoryUsage(const char*)’中:
main.cpp:261:35: 错误:‘isdigit’在此作用域中尚未声明
make[3]: *** [main.o] 错误 1
make[3]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
/tcpcopy”
make[2]: *** [all-recursive] 错误 1
make[2]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1/src
”
make[1]: *** [all-recursive] 错误 1
make[1]:
离开目录“/var/abs/local/pkgman/LocalPackages/tcpcopy/src/tcpcopy-0.3.1”
make: *** [all] 错误 2
==> 错误: 在 build() 中发生一个错误。
正在放弃...
Original issue reported on code.google.com by [email protected]
on 28 Dec 2011 at 3:17
很酷的软件,谢谢。虽然我不懂你的代码,但是我已经打包��
� archlinux
用户了,那里的社区很活跃,估计你能得到更好的反馈,建��
�和补丁。
http://aur.archlinux.org/packages.php?ID=52764
Original issue reported on code.google.com by [email protected]
on 29 Sep 2011 at 8:08
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.