mellanox/sockperf

Network Benchmarking Utility

Introduction

sockperf is a network benchmarking utility over socket API that was designed for testing performance (latency and throughput) of high-performance systems (it is also good for testing performance of regular networking systems). It covers most of the socket API calls and options.

Specifically, in addition to the standard throughput tests, sockperf does the following:

• Measure latency of each discrete packet at sub-nanosecond resolution (using TSC register that counts CPU ticks with very low overhead).

• Does the above for both ping-pong mode and latency under load mode. This means that we measure latency of single packets even under load of millions of Packets Per Second (without waiting for reply of packet before sending subsequent packet on time)

• Enable spike analysis by providing histogram, with various percentiles of the packets’ latencies (for example: median, min, max, 99% percentile, and more), (this is in addition to average and standard deviation). Also, sockperf provides a full log with all packet’s tx/rx times that can be further analyzed with external tools, such as MS-Excel or matplotlib - All this without affecting the benchmark itself.

• Support MANY optional settings for good coverage of socket API and network configurations, while still keeping very low overhead in the fast path to allow cleanest results.

Prereqs: What you will need to compile sockperf on Unix systems

• Perl 5.8+ (used by the automake tools)

• GNU make tools: automake 1.7+, autoconf 2.57+, m4 1.4+ and libtool 1.4+

• A C++11 Compiler, among those tested are:

  • GCC
  • Clang
  • icc

Linux (Debian/Ubuntu): sudo apt install perl make automake autoconf m4 libtool-bin g++

FreeBSD: sudo pkg install gmake automake libtool

How to install

The sockperf package uses the GNU autotools compilation and installation framework.

./autogen.sh  (only when cloning from repository)
./configure --prefix=<path to install>
make
make install

Configuration

Type ./configure --help for a list of all the configure options. Some of the options are generic autoconf options, while the sockperf specific options are prefixed with "SOCKPERF:" in the help text.

• To enable TLS support

  • ./configure --prefix=<path to install> --with-tls=<path to OpenSSL install>
  • Use OpenSSL 3.0.0 or higher

• To enable unit tests

  • ./configure --prefix=<path to install> --enable-test

• To enable the documentation

  • ./configure --prefix=<path to install> --enable-doc

• To enable the special scripts

  • ./configure --prefix=<path to install> --enable-tool

• To compile with debug symbols and information:

  • ./configure --prefix=<path to install> --enable-debug
  • This will define the DEBUG variable at compile time.

To build for ARM

1. Define CROSS_COMPILE in the environment to point to the cross compilation tools, e.g. set CROSS_COMPILE=/opt/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/arm-linux-gnueabihf-
2. Use ./autogen.sh to create the configure script.
3. Invoke ./configure with the following options: ./configure CXX=${CROSS_COMPILE}g++ STRIP=${CROSS_COMPILE}strip LD=${CROSS_COMPILE}ld CC=${CROSS_COMPILE}gcc --host i386
4. Invoke make

To build for FreeBSD

• Make sure automake tools are installed.

Licensing

View Here

~Good luck!

sockperf's Issues

issue: 2890836 Support IPv6

imported from: http://argus-bg.dnsalias.org/issues/727

What need is this feature going to satisfy?
1. Tool should support IPv6

Original issue reported on code.google.com by [email protected] on 1 Apr 2011 at 2:00

"No VMA version info" is wrongly printed

What steps will reproduce the problem?
1. This string is wrongly printed even when running over VMA
2.
3.

What is the expected output? What do you see instead?
1. in case running without VMA then this message should not appear at all.
2. in case running over VMA, it will be nice to see VMA version; however, it is 
not important (fix it only if it is easy)

Please use labels and text to provide additional information.
please fix it both in sockperf_v1 and in sockperf_v2

Original issue reported on code.google.com by [email protected] on 30 Mar 2011 at 12:41

New method server_accept() is inlined with no need

This method "jumps" from fast path to slow path for accepting new connection.
There is no need to inline it since inlining it increases the code of the fast 
path for no need.
please remove the inline directive from such methods and move their 
implementation to *.cpp file (otherwise, it is implicitly inline directive)

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 6:33

it seems that TCP's accept() is called too much times, perhaps in non blocking mode

What steps will reproduce the problem?
1. run LD_PRELOAD=libvma.so valgrind -v  sockperf sr --tcp 
2. after the server completed its loading, press CTRL-C
3.

What is the expected output? What do you see instead?
valgrind will print its statistics.  I would expect low number of calls to the 
server_accept(int) method.
Instead I see that the suppression rule below was used huge number of times.

NOTE: it could be that this is normal.  Please verify!

Please use labels and text to provide additional information.

use the following in your valgrind's suppressions file:
# Use of uninitialised value of size 8
#    at 0x5548C4A: mlx4_poll_cq (in /usr/lib64/libmlx4-m-rdmav2.so)
#    by 0x4C69F18: cq_mgr::poll(ibv_wc*, int, unsigned long*) (in 
/usr/lib64/libvma.so.6.3.6)
#    by 0x4C6B83A: cq_mgr::poll_and_process_helper_rx(unsigned long*, void*) 
(in /usr/lib64/libvma.so.6.3.6)
#    by 0x4C6F0FE: ring::poll_and_process_element_rx(unsigned long*, void*) (in 
/usr/lib64/libvma.so.6.3.6)
#    by 0x4CB4E33: sockinfo_tcp::rx_wait_helper(int&, bool) (in 
/usr/lib64/libvma.so.6.3.6)
#    by 0x4CB72D2: sockinfo_tcp::accept(sockaddr*, unsigned int*) (in 
/usr/lib64/libvma.so.6.3.6)
#    by 0x65606B: Server<IoRecvfrom, SwitchOff, SwitchOff>::server_accept(int) 
(Server.cpp:272)
#    by 0x65905D: Server<IoRecvfrom, SwitchOff, SwitchOff>::doLoop() 
(Server.cpp:198)
#    by 0x65B789: void server_handler<IoRecvfrom, SwitchOff, SwitchOff>(int, 
int, int) (Server.h:58)
#    by 0x66AAB9: main (SockPerf.cpp:2818)
#  Uninitialised value was created by a stack allocation
#    at 0x554C410: mlx4_create_cq (in /usr/lib64/libmlx4-m-rdmav2.so)
# 
{
   rule-90
   Memcheck:Value8
   fun:mlx4_poll_cq
   fun:_ZN6cq_mgr4pollEP6ibv_wciPm
   fun:_ZN6cq_mgr26poll_and_process_helper_rxEPmPv
   fun:_ZN4ring27poll_and_process_element_rxEPmPv
   fun:_ZN12sockinfo_tcp14rx_wait_helperERib
   fun:_ZN12sockinfo_tcp6acceptEP8sockaddrPj
   fun:_ZN6ServerI10IoRecvfrom9SwitchOffS1_E13server_acceptEi
}

Original issue reported on code.google.com by [email protected] on 24 Sep 2012 at 8:24

sockperf pp --tcp error and abort with burst -b1000

imported from: http://argus-bg.dnsalias.org/issues/721
What steps will reproduce the problem?
1.sockperf sr --tcp
2.sockperf pp --tcp -i 1.1.1.28 -b 1000
3.(happend in some of the tests.  Need to check if this is still rellevant!

What is the expected output? What do you see instead?
client exits with the following error:
root@IBM29 vperf_2]# vperf pp --tcp -i 1.1.1.28 -b 1000
vperf: version #2.1.1801 
vperf: No VMA version info
vperf[CLIENT] send on:vperf: using recvfrom() to block on socket(s)

[ 0] IP = 1.1.1.28 PORT = 11111 # TCP
vperf: Warmup stage (sending a few dummy packets)...
vperf: Starting test...
vperf: PacketTimes.cpp:44:ERROR: _seqNo=2010186544 > m_maxSequenceNo=20001010
Please use labels and text to provide additional information.

Original issue reported on code.google.com by [email protected] on 29 Mar 2011 at 2:16

Suggest and develop additional TCP testing tools

imported from: http://argus-bg.dnsalias.org/issues/580

The comming VMA will include full blown TCP stack. This is much more 
complecated than handling UDP that we currently have in VMA. Hence, we need 
more than latency and throughput tests that vperf knows to do.

for example we would like to test behavior of TCP stack in case peer doesn't 
read from its side of the socket, and other tests that relates to recv window, 
acks congestion controll, and so on.
NOTE:

   1. These tests can be written in python, and not nessarily in C++
   2. We have heard about ixia (http://www.ixiacom.com/) that does similair things. Here we are talking about simpler level of tests. I am writing ixia, only for providing background for what we are talking about.
   3. This task should come after supporting TCP in vperf

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:29

memory leak upon TCP accept

What steps will reproduce the problem?
1. start server over valgrind: valgrind sockperf sr --tcp
2. start client using: sockperf pp --tcp -i <server-ip>
3. after client completed, press CTRL-C in the server

What is the expected output? What do you see instead?
clean termination of sockperf server and valgrind

Please use labels and text to provide additional information.
valgrind, reports 128KB leak in server_accept
==11270== 131,012 bytes in 1 blocks are definitely lost in loss record 3 of 3
==11270==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==11270==    by 0x656009: Server<IoRecvfrom, SwitchOff, 
SwitchOff>::server_accept(int) (Server.cpp:258)
==11270==    by 0x65905D: Server<IoRecvfrom, SwitchOff, SwitchOff>::doLoop() 
(Server.cpp:198)
==11270==    by 0x65B789: void server_handler<IoRecvfrom, SwitchOff, 
SwitchOff>(int, int, int) (Server.h:58)
==11270==    by 0x66AAB9: main (SockPerf.cpp:2818)

The leak is in the following line:
Server.cpp:258: tmp->recv.buf = (uint8_t*) malloc 
(sizeof(uint8_t)*2*MAX_PAYLOAD_SIZE);

This code was introduced in revision 133.

*Note: this leak is repeated for any time the server performs tcp accept*
Also note: there is similar leak, that happens one time in the initialization 
in the following line:
SockPerf.cpp 2693: tmp->recv.buf = (uint8_t*) malloc 
(sizeof(uint8_t)*2*MAX_PAYLOAD_SIZE);

*Please handle them both!*

Original issue reported on code.google.com by [email protected] on 23 Sep 2012 at 11:33

[deleted issue]

[deleted issue]

Sockperf can't be compiled in Debug/Release 32/64 bit in Windows

What steps will reproduce the problem?
1. When compiling sockperf in Windows in:

    * 64 bit Debug
    * 32 bit Debug
    * 32 bit Release

There is a linkage problem.

The following commit will fix the problem:

Author: Dotan Barak <[email protected]>
Date:   Wed Nov 6 14:49:06 2013 +0200

    Add missing libraries when compiling sockperf in Windows

    Add the missing libraries when compiling sockperf in Windows for the following:
    * 64 bit Debug
    * 32 bit Debug
    * 32 bit Release

    Signed-off-by: Dotan Barak <[email protected]>

diff --git a/win/project/sockperf.vcxproj b/win/project/sockperf.vcxproj
index 82a2bdf..7cd9d82 100644
--- a/win/project/sockperf.vcxproj
+++ b/win/project/sockperf.vcxproj
@@ -90,6 +90,7 @@
     <Link>
       <SubSystem>Console</SubSystem>
       <GenerateDebugInformation>true</GenerateDebugInformation>
+      
<AdditionalDependencies>Ws2_32.lib;Dbghelp.lib;%(AdditionalDependencies)</Additi
onalDependencies>^M
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
@@ -103,6 +104,7 @@
     <Link>
       <SubSystem>Console</SubSystem>
       <GenerateDebugInformation>true</GenerateDebugInformation>
+      
<AdditionalDependencies>Ws2_32.lib;Dbghelp.lib;%(AdditionalDependencies)</Additi
onalDependencies>^M
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
@@ -120,7 +122,7 @@
       <GenerateDebugInformation>true</GenerateDebugInformation>
       <EnableCOMDATFolding>true</EnableCOMDATFolding>
       <OptimizeReferences>true</OptimizeReferences>
-      <AdditionalDependencies>Ws2_32.lib, User32.lib, 
Dbghelp.lib</AdditionalDependencies>
+      <AdditionalDependencies>Ws2_32.lib;Dbghelp.lib</AdditionalDependencies>^M
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
@@ -178,4 +180,4 @@
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
-</Project>
\ No newline at end of file
+</Project>^M

Original issue reported on code.google.com by [email protected] on 6 Nov 2013 at 1:18

Handle the case that Tcp socket is not ready for write

imported from: http://argus-bg.dnsalias.org/issues/579

   1. Default: send blocking and force wait till send complete
   2. Option: Skip that send and jump to next send (consider, whether to inc sequence number or not)
   3. In both modes, add counter for number of times TCP was not ready for write (MSG_DONTWAIT that resulted in EWOULDBLOCK) and show that counter in summary statistics (together with dropped packets and so on). In case this error occurred, the statistics should tell the user about the other option to use the other mode

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:42

Sockperf in throughput mode with message size 256/128 bytes doesn't work over VMA

Steps to reproduce:
1.LD_PRELOAD=libvma.so sockperf sr -A 10000
2.LD_PRELOAD=libvma.so sockperf tp -i <srv IP> -m 256 -A 10000 -t 100

Expected:
The server successfully receives the traffic:
#The received packets counters increments.
#The number of received packet by the server is equal to the number
of transmitted packets by clinet

Actually:
One of the following:
1. The client receives a mesage with zero payload and terminates.
2. The client hangs while the server prints an abnormal results:
the number of received packets is not proportionally high to the number of sent 
packets, and even when the client is terminated the server continues to prints 
as if it recevie a new data till it is terminated with Ctl+C.

Versions:
Sockperf: 2.5.30
VMA: 5.0.6-0

- Seems that in some scenarios the server enters inside an infinite loop where 
it prints the statistic regarding the last received bytes.
- Maybe the server performs a wrong parsing of the receved message header?
- Maybe there is some corruption of the rx qeueue?

Please notice that the issue happens only with sockperf, it is not reproducable 
with other testing tools.

Original issue reported on code.google.com by [email protected] on 17 Apr 2011 at 8:27

When compiling Sockperf in Windows 64 bit there is a compilation warning

What steps will reproduce the problem?
1. When compiling Sockperf in Windows 64 bit the following compilation warning 
is printed:

    src\SockPerf.cpp(2430): warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data


The following patch will fix this issue:

Author: Dotan Barak <[email protected]>
Date:   Wed Nov 6 14:36:59 2013 +0200

    Fixed compilation warning in Windows 64 bit

    Fixed the following compilation warning in Windows:
    src\SockPerf.cpp(2430): warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data

    Signed-off-by: Dotan Barak <[email protected]>

diff --git a/src/SockPerf.cpp b/src/SockPerf.cpp
index f677546..a79282c 100644
--- a/src/SockPerf.cpp
+++ b/src/SockPerf.cpp
@@ -2425,7 +2425,7 @@ int sock_set_tos(int fd)
 {
        int rc = SOCKPERF_ERR_NONE;
        if (s_user_params.tos) {
-               size_t len = sizeof(s_user_params.tos);
+               socklen_t len = sizeof(s_user_params.tos);
                if (setsockopt(fd, IPPROTO_IP, IP_TOS, (char*)&s_user_params.tos, len) < 0) {
                        log_err("setsockopt(TOS), set  failed.  It could be that this option is not supported in your system");
                        rc = SOCKPERF_ERR_SOCKET;

Original issue reported on code.google.com by [email protected] on 6 Nov 2013 at 1:20

Support capability to set several threads on client

imported from: http://argus-bg.dnsalias.org/issues/715

What need is this feature going to satisfy?
1.Requirements means that in case the client has feed file with multiple 
sockets it will be able to assign them to N threads as it is done today in the 
server.

Original issue reported on code.google.com by [email protected] on 1 Apr 2011 at 1:54

Support thread-pool mode

imported from: http://argus-bg.dnsalias.org/issues/673

In the future we may want to support thread-pool mode, in which the accepted 
connections for 1 TCP listening socket are distributed between worker threads. 
Nothing to do at this phase.

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:44

Clean up idle TCP connections

imported from: http://argus-bg.dnsalias.org/issues/673

The server may need a way to recognize and clean "old" sockets that are not 
in use any more (i.e., client's TCP stack didn't send FIN packet).

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:47

support loopback connections - local network connection between server & client on same machine

To reproduce the problem:
1. ib0 = 4.4.4.10 (IP address of ib0 interface)
2. Run UDP server ob ib0: "sockperf sr -p 10000 -i 4.4.4.10"
3. Run UDP client on ib1 with same UDP port: "sockperf pp -i 4.4.4.10 -p 10000"

Expected output: ping-pong tests completes and prints latency results
Instead we see: UDP UC or MC: "no response from server"
Instead we see: TCP: "can't connect to server"

Original issue reported on code.google.com by [email protected] on 24 Jan 2013 at 11:43

import issues from the old system

What is the definition of this task?
1. Please import issues that are still relevant from the old Redmine system.
2. No need to import fixed issues
3. Please let me know if you need me to create additional labels.

Original issue reported on code.google.com by [email protected] on 29 Mar 2011 at 2:27

can run multiple MC server to receive the same ingress MC stream on single machine

issue do to last changes which removed the SO_REUSEADDR for all UDP

Original issue reported on code.google.com by [email protected] on 13 Feb 2013 at 9:36

remove gnu make from documentation

What is the definition of this task?
1. we mention gnu make in main.dox and maybe in other places like readme
2. this is confusing since user should only do ./configure && make
3. adjust documentation to the right usage

What is the expected output?
remove it from trunk and from sockperf_v1 and from sockperf_v2

Original issue reported on code.google.com by [email protected] on 30 Mar 2011 at 10:42

Usage typo - Manual routing

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?
mismatch in manual routing flags: mc-rx-if <--> rx_mc_if 

What version of the product are you using? On what operating system?


Please provide any additional information below.

Usage: sockperf server [options] [args]...
 sockperf server                          
 sockperf server [-i ip] [-p port] [--rx_mc_if ip] [--tx_mc_if ip]    same as udp_lat
 sockperf server -f file [-F s/p/e] [--rx_mc_if ip] [--tx_mc_if ip]

Options:
 -h,-?   --help,--usage         -Show the help message and exit.
         --tcp                  -Use TCP protocol (default UDP).
 -i      --ip                   -Listen on/send to ip <ip>.     
 -p      --port                 -Listen on/connect to port <port> (default 11111).
 -f      --file                 -Tread multiple ip+port combinations from file <file> (server uses select).
 -F      --iomux-type           -Type of multiple file descriptors handle [s|select|p|poll|e|epoll](default select).
         --timeout              -Set select/poll/epoll timeout to <msec>, -1 for infinite (default is 10 msec).     
 -a      --activity             -Measure activity by printing a '.' for the last <N> packets processed.             
 -A      --Activity             -Measure activity by printing the duration for last <N> packets processed.          
         --tcp-avoid-nodelay    -tcp-nodelay uses for delivering TCP Messages Immediately (default ON).             
         --mc-rx-if             -<ip> address of interface on which to receive mulitcast packets (can be other then route table).
         --mc-tx-if             -<ip> address of interface on which to transmit mulitcast packets (can be other then route table).
         --mc-loopback-enable   -Enables mc loopback (default disabled).                                                          
         --mc-ttl               -Limit the lifetime of the packet (default 2).     

Original issue reported on code.google.com by [email protected] on 26 May 2011 at 12:26

--load-vma does not load vma by default

What steps will reproduce the problem?
1. sockperf sr -i 226.6.6.6 --load-vma
2.
3.

What is the expected output? What do you see instead?
libvma.so should be load as default
I you use --load-vma=libvma.so it works

Please use labels and text to provide additional information.

Original issue reported on code.google.com by [email protected] on 3 Apr 2011 at 2:16

Support --tcp-skip-blocking-send switch in command line

imported from: http://argus-bg.dnsalias.org/issues/584

In case user used this new switch, we should:

   1. skip sending TCP packets that would block our thread.
   2. increment a new counter – equivalent to dropped-counter that exists on receive.
   3. display this counter at the end statistics when displaying dropped-counter.

Note: this counter should be in place, even if this flag is not used and we 
blocked on send operation.
In both cases, in case the counter > 0, the message at the end should educate 
the user about the other option.

Original issue reported on code.google.com by [email protected] on 1 Apr 2011 at 1:41

handling more fd's (sockets)

What is the definition of this task?
1. prepare sockperf to handle more fd's (sockets) then 1024.
2. modify sockperf so it reads the number of open files and knows to handle 
more inputs from the feedfile.

What is the expected output?
sockperf can handle large number of fd's: 1400, 2000, 5000, 10K..

Original issue reported on code.google.com by [email protected] on 16 Aug 2011 at 8:31

Changes to support cross compilation to ARM

What need is this feature going to satisfy?
1. Support x86 or ARM, depending on desired target

What is the expected output? What do you have currently? Any workaround?

I have changes in my sandbox that support compiling to x86 or an ARM target. I 
can clean up the modifications and submit as a patch if the maintainers agree 
this is useful. 

Thanks - Vince
[email protected]

Original issue reported on code.google.com by [email protected] on 30 Oct 2013 at 9:12

bad calc of max packet times array

What steps will reproduce the problem?
1.LD_PRELOAD=libvma.so sockperf server -i 224.18.7.81 -p 5001
2.LD_PRELOAD=libvma.so sockperf ping-pong -i 224.18.7.81 -p 5001 -t 3 -b 5
3.

What is the expected output? What do you see instead?
the client crashes with the following error:
ockperf: PacketTimes.cpp:60:ERROR: _seqNo=1600016 > m_maxSequenceNo=1600015 
(errno=25 Inappropriate ioctl for device)
sockperf: program exits because of an error.  current value of errno=25 
(Inappropriate ioctl for device)sockperf: [tid: 30599] ------
sockperf: [0] 0x578fff: sockperf [0x578fff]
sockperf: [1] 0x579090: sockperf [0x579090]
sockperf: [2] 0x57ad06: sockperf [0x57ad06]
sockperf: [3] 0x48a23b: sockperf [0x48a23b]
sockperf: [4] 0x48aa99: sockperf [0x48aa99]
sockperf: [5] 0x58a7fc: sockperf [0x58a7fc]
sockperf: [6] 0x58de96: sockperf [0x58de96]
sockperf: [7] 0x351421d994: /lib64/libc.so.6(__libc_start_main+0xf4) 
[0x351421d994]
sockperf: [8] 0x402e79: sockperf(__gxx_personality_v0+0x131) [0x402e79]


What version of the product are you using? On what operating system?


Please provide any additional information below.
looks like a problem with the calc of max pps

Original issue reported on code.google.com by [email protected] on 7 Apr 2011 at 9:25

Report maximum throughput that is safe without drops

Today the only way to know about drops - in throughput test - is gap-analysis 
in the server.
This only gives boolean answer. We need to find the max pps that is safe 
without drops.

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:09

Prepare initial point measurement results

imported from: http://argus-bg.dnsalias.org/issues/702

Environment:
Tools: udp_lat, vperf-0.7, vperf-1.x, vperf-2.x
UDP: os+eth2, os+ib0, vma+eth2, vma+ib0
Tests: ul, pp, tp
Options: recv, select, epoll, pps=max, msg(12,200)

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:51

duplicate packet with feedfile on many MC groups

What steps will reproduce the problem?
1. create feedfile with many (200~500) unique MC groups
2. run with selecet/epoll/poll
3. Notice the duplicated messages counter should not be greater then zero

Original issue reported on code.google.com by [email protected] on 3 Feb 2013 at 8:35

Sockperf with playback: reply for the very last packet(s) is lost

1.Host1: LD_PRELOAD=libvma.so sockperf server -i <MC-group>
2. Host2: LD_PRELOAD=libvma.so sockperf playback -i <MC-group>  --data-file 
/tmp/playback --reply-every=1 
3. The playback contain:

0.510550, 12
0.510651, 12
0.510790, 12
0.510907, 12
1.511156, 12
1.511290, 12
1.555555, 32
1.555566, 32
2.555555, 32
3.666666, 12
15.55555, 64
20.44444, 64
#-- end of file --

What is the expected output? What do you see instead?

playback file should load successfully. Send/received should be 12

Actual results: SentMessages=12; ReceiveMessages=11

Please use labels and text to provide additional information.

Original issue reported on code.google.com by [email protected] on 3 Apr 2011 at 3:48

add support for low latency sockets

Here is a sample patch
You can use it under any SW license you want to.


Add lls socket option support.
use --lls (value in usecs) to override global setting.
Right now we always get and print the value of the option before we set,
to show that the option works properly.
This should be removed in an official release.

---

 src/Defs.h       |    3 +++
 src/SockPerf.cpp |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 58 insertions(+), 3 deletions(-)

diff --git a/src/Defs.h b/src/Defs.h
index e38e3a4..041fa0d 100644
--- a/src/Defs.h
+++ b/src/Defs.h
@@ -161,6 +161,7 @@ enum {
     OPT_OUTPUT_PRECISION,           //35
     OPT_CLIENTPORT,                 //36
     OPT_CLIENTIP,                   //37
+    OPT_LLS,            //38
 };

 #define MODULE_NAME            "sockperf"
@@ -527,6 +528,8 @@ struct user_params_t {
 //    bool stream_mode; - use b_stream instead
     int mthread_server;
     struct timeval* select_timeout;
+    unsigned int lls_usecs;
+    bool lls_is_set;
     int sock_buff_size;
     int threads_num;
     char threads_affinity[MAX_ARGV_SIZE];
diff --git a/src/SockPerf.cpp b/src/SockPerf.cpp
index 41daf95..28569c9 100644
--- a/src/SockPerf.cpp
+++ b/src/SockPerf.cpp
@@ -207,6 +207,10 @@ static const AOPT_DESC  common_opt_desc[] =
         "Limit the lifetime of the message (default 2)."
     },
     {
+        OPT_LLS, AOPT_ARG, aopt_set_literal( 0 ), aopt_set_string( "lls" ),
+        "Turn on LLS via socket option (value = us to poll)."
+    },
+    {
         OPT_BUFFER_SIZE, AOPT_ARG, aopt_set_literal( 0 ), aopt_set_string( "buffer-size" ),
         "Set total socket receive/send buffer <size> in bytes (system defined by default)."
     },
@@ -292,7 +296,7 @@ static int proc_mode_help( int id, int argc, const char 
**argv )
     int   i = 0;

     printf(MODULE_NAME " is a tool for testing network latency and throughput.\n");
-    printf("version %s\n", STR(VERSION));
+    printf("version %s-lls\n", STR(VERSION));
     printf("\n");
     printf("Usage: " MODULE_NAME " <subcommand> [options] [args]\n");
     printf("Type: \'" MODULE_NAME " <subcommand> --help\' for help on a specific subcommand.\n");
@@ -1382,8 +1386,8 @@ static int proc_mode_server( int id, int argc, const char 
**argv )
         {
             'B', AOPT_NOARG,    aopt_set_literal( 'B' ), aopt_set_string( "Bridge" ),
             "Run in Bridge mode."
-        },
-*/
+        },
+*/
         {
              OPT_THREADS_NUM, AOPT_ARG, aopt_set_literal( 0 ), aopt_set_string( "threads-num" ),
              "Run <N> threads on server side (requires '-f' option)."
@@ -1789,6 +1793,26 @@ static int parse_common_opt( const AOPT_OBJECT 
*common_obj )
             s_user_params.is_nonblocked_send = true;
         }

+        if ( !rc && aopt_check(common_obj, OPT_LLS) ) {
+            const char* optarg = aopt_value(common_obj, OPT_LLS);
+            if (optarg) {
+                errno = 0;
+                int value = strtoul(optarg, NULL, 0);
+                if (errno != 0 || value < 0) {
+                    log_msg("'-%d' Invalid LLS value: %s", OPT_LLS, optarg);
+                    rc = SOCKPERF_ERR_BAD_ARGUMENT;
+                }
+                else {
+                    s_user_params.lls_usecs = value;
+                    s_user_params.lls_is_set = true;
+                }
+            }
+            else {
+                log_msg("'-%d' Invalid value", OPT_LLS);
+                rc = SOCKPERF_ERR_BAD_ARGUMENT;
+            }
+        }
+
         if ( !rc && aopt_check(common_obj, OPT_RECV_LOOPING) ) {

             const char* optarg = aopt_value(common_obj, OPT_RECV_LOOPING);
@@ -2296,6 +2320,29 @@ int sock_set_reuseaddr(int fd)
     return rc;
 }

+#ifndef SO_LL
+#define SO_LL 46
+#endif
+int sock_set_lls(int fd)
+{
+    int rc = SOCKPERF_ERR_NONE;
+    unsigned int lls;
+    int size = sizeof(lls);
+
+    if(getsockopt(fd, SOL_SOCKET, SO_LL, &lls, (socklen_t *)&size) < 0){
+        log_err("getsockopt(SO_LL) failed");
+        return SOCKPERF_ERR_SOCKET;
+    } else
+        log_msg("socket option SO_LL default was %u, changing to %u", lls, 
s_user_params.lls_usecs);
+
+    if (setsockopt(fd, SOL_SOCKET, SO_LL, &(s_user_params.lls_usecs), 
sizeof(s_user_params.lls_usecs)) < 0) {
+        log_err("setsockopt(SO_LL) failed");
+        rc = SOCKPERF_ERR_SOCKET;
+    }
+    return rc;
+}
+
+
 int sock_set_snd_rcv_bufs(int fd)
 {
     /*
@@ -2460,6 +2507,11 @@ int prepare_socket(int fd, struct fds_data *p_data)
     }

     if (!rc &&
+            (s_user_params.lls_is_set == true))
+    {
+        rc = sock_set_lls(fd);
+    }
+    if (!rc &&
             (s_user_params.sock_buff_size > 0))
     {
         rc = sock_set_snd_rcv_bufs(fd);

Original issue reported on code.google.com by [email protected] on 18 Jun 2013 at 6:16

rxTime.setNow() should be called immediately after msg_recvfrom

we had it in the past.  Meanwhile, many people added a lot of code between 
these 2 lines.  

As a result we take the rxTime a bit later than we could.  This had some minor 
effect of our accuracy.  There is no need for this.

Original issue reported on code.google.com by [email protected] on 26 Nov 2012 at 12:44

Add capabilities to bind client to specific if (local ip address)

similar to the '--client_port' flag support

Original issue reported on code.google.com by [email protected] on 29 Jan 2013 at 8:38

segmentation fault when running multithread process

What steps will reproduce the problem?
1. Host1: sockeprf sr -f <feed> -F s --threds-num <>
2. Host2: sockeprf pp -f <feed> -F s 

multithread process creates segmentation fault on server side
TCP and UDP

What is the expected output? What do you see instead?
Segmentation fault on server side

What version of the product are you using? On what operating system?
2.5.27, RH6

Please provide any additional information below.

Output:

sockperf:  == version #2.5.27 ==                                     
sockperf: No VMA version info                                        
sockperf: [SERVER] listen on:                                        
[ 0] IP = 1.1.1.1         PORT = 12347 # UDP                         
sockperf: Warmup stage (sending a few dummy packets)...              
sockperf: [tid 19668] using select() to block on socket(s)           
sockperf: [SERVER] listen on:                                        
[ 0] IP = 1.1.1.1         PORT = 12349 # UDP                         
sockperf: Warmup stage (sending a few dummy packets)...              
sockperf: [tid 19670] using select() to block on socket(s)           
sockperf: [SERVER] listen on:                                        
[ 0] IP = 1.1.1.1         PORT = 12348 # UDP                         
sockperf: Warmup stage (sending a few dummy packets)...              
sockperf: [tid 19669] using select() to block on socket(s)           
sockperf: [SERVER] listen on:                                        
[ 0] IP = 1.1.1.1         PORT = 12346 # UDP                         
sockperf: Warmup stage (sending a few dummy packets)...              
sockperf: [tid 19667] using select() to block on socket(s)           
sockperf: [SERVER] listen on:                                        
[ 0] IP = 1.1.1.1         PORT = 12345 # UDP                         
sockperf: Warmup stage (sending a few dummy packets)...              
sockperf: [tid 19666] using select() to block on socket(s)           
^Csockperf: Got signal 2 - exiting                                   
sockperf: Total 53040 messages received and handled                  
Segmentation fault (core dumped) 

(gdb) bt
#0  0x00000033af20c670 in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000587622 in server_select_per_thread() ()              
#2  0x000000000059a371 in main (

Original issue reported on code.google.com by [email protected] on 29 Mar 2011 at 9:06

Cant run vperf under load with small pps

What steps will reproduce the problem?
1. Host1: LD_PRELOAD=libvma.so vperf server --tcp
2. Host2: LD_PRELOAD=libvma.so vperf under-load --tcp -i <server_ip> --pps 1

What is the expected output? What do you see instead?
vperf under-load --tcp -i 1.1.1.1 --pps=1
vperf: version #2.4.1868M
vperf: No VMA version info
vperf: Client.cpp:412:ERROR: Can`t connect socket (errno=111 Connection refused)
vperf: program exits because of an error. current value of errno=111 
(Connection refused)vperf: [tid: 11731] ------
vperf: [0] 0x56e647: vperf() [0x56e647]
vperf: [1] 0x42d7e6: vperf() [0x42d7e6]
vperf: [2] 0x4c9ae9: vperf() [0x4c9ae9]
vperf: [3] 0x4e6836: vperf() [0x4e6836]
vperf: [4] 0x583695: vperf() [0x583695]
vperf: [5] 0x583f12: vperf() [0x583f12]
vperf: [6] 0x311961ec5d: /lib64/libc.so.6(__libc_start_main+0xfd) [0x311961ec5d]
vperf: [7] 0x403229: vperf() [0x403229]


What version of the product are you using? On what operating system?
vperf 2.4.1868

Please provide any additional information below.
1.Happnes with pps up to approx 100.
2.Happens on both TCP and UDP

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 6:25

copy the code from old repository

branches/v0 = udp_lat

branches/v1 = original Avner's vperf

branches/v2 = vperf that also supports TCP

Original issue reported on code.google.com by [email protected] on 28 Mar 2011 at 8:45

Thread Affinity on server

Tool should support thread affinity on server side

Original issue reported on code.google.com by [email protected] on 1 Apr 2011 at 1:45

Fail to run sockperf ping-pong with (-F) iomux-type r

What steps will reproduce the problem?
1.Create MCG file and copy it to server and client 
7.1.1.1:1000
227.1.1.2:1001
227.1.1.3:1002
227.1.1.4:1003
227.1.1.5:1004

2.Add route to specified device 
#route add -net 224.0.0.0 netmask 240.0.0.0 <device>
3.Run on server :
#sockperf server --nonblocked -f /tmp/sockperf_5 -F r --threads-num 5
Run on client :
#sockperf ping-pong -m 12 -t 10 --mps=MAX -f /tmp/sockperf_5 -F 4

What is the expected output? What do you see instead?
Test end with summary result of send and recievce message during specified time 


What version of the product are you using? On what operating system?
Red Hat Enterprise Linux Server release 6.4 (Santiago)
Sockperf version 2.5.225 

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 4 Aug 2013 at 12:47

Add basic check of values for numerical arguments

imported from: http://argus-bg.dnsalias.org/issues/614

What is the expected output? What do you have currently? Any workaround?
Report error during setting incorrect value for option expected numeric data.

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:27

Throughput summary consider should also consider TCP header, not only UDP header

What steps will reproduce the problem?
(note, there are 2 problems: 1 - about msg-size recomendation, and 2 - about 
truput calculation)

The following message is incorrect for TCP.
vperf: NOTE: test was performed, using msg-size=12. For getting maximum 
throughput consider using --msg_size=1472
vperf: Summary: Message Rate is 456309 [msg/sec]
vperf: Summary: BandWidth is 17.407 MBps (139.254 Mbps)

please replace it with something like:
vperf: NOTE: test was performed, using msg-size=12. For getting maximum 
throughput consider using --msg_size=1472 (UDP), or --msg_size=??? (TCP)

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 6:32

no hton/ntoh around the message size

there's no hton/ntoh around the message size

Original issue reported on code.google.com by [email protected] on 17 Jul 2011 at 10:24

[deleted issue]

[deleted issue]

distinguish between TCP statistics vs. UDP statistics (in case mixed feed file)

What need is this feature going to satisfy?
1. statistics should be reported per socket or per socket type
2. it make no sense to provide one average for TCP & UDP & MC
3. (this is only relevant in case feed file contains mixed socket types)

What is the expected output? What do you have currently? Any workaround?
For each server (currently we support 1 server, but the infrastructure supports 
N servers), the client should print statistics based on socket type

Workaround:
user can use homogeneous feed files and repeat the test per file

Original issue reported on code.google.com by [email protected] on 5 Apr 2011 at 6:50

add -rdynamic to makefile for stacktrace printout

please add -rdynamic to link command in Makefile.am
can be done using AM_LDFLAGS = -rdynamic

bellow is the description from man gcc
-rdynamic
Pass the flag -export-dynamic to the ELF linker, on targets that support it. 
This instructs the linker to add all symbols, not only used ones, to the 
dynamic symbol table. This option is needed for some uses of "dlopen" or to 
allow obtaining backtraces from within a program.

(note: we already use backtrace printouts upon error - this change will show 
them better)

Original issue reported on code.google.com by [email protected] on 31 Aug 2011 at 7:57

Patch to allow setting TOS

What is the definition of this task?
1.Allow specifying Type Of Service (TOS) for data.

What is the expected output?
Verify in IP header that the DSCP bits are set correctly.


Patch is for version: 2.5.193
Tested on Linux x86_64 build, verified output with tcpdump

Original issue reported on code.google.com by [email protected] on 18 Apr 2013 at 2:24

Attachments:

• patch.sockperf_2.5.193.allowTOS

Process irrelevant switches in friendly manner

imported from: http://argus-bg.dnsalias.org/issues/535

What steps will reproduce the problem?

1. playback usage tells about irrelevant switches, such as --time, --range, 
--burst, --msg-size

What is the expected output? What do you see instead?

The purpose of this feature is to have simplified usage for each sub command of 
vperf without irrelevant switches.
Please check carefully all switches of all sub commands and make sure they are 
correct.
In this occasion, please verify that whenever user provide incorrect switch, 
vperf stops with appropriate message and not continue silently.

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:39

Fix vma-perf-envelope.sh to set not Addaptive-rx

It appears that Coalesce parameters were not optimal for our needs. It probably 
entered noise to your results. 

Fix vma-perf-envelope.sh to do it automatically (today, it only handles 
rx-frames and rx-usecs, not Addaptive-rx)

Using ethtool:

the original configuration on mir9 was as follows:

[avnerb@bgate ~]$

[avnerb@bgate ~]$ ssh mir9 sudo ethtool -c eth2

Coalesce parameters for eth2:

Adaptive RX: on TX: off

stats-block-usecs: 0

sample-interval: 0

pkt-rate-low: 400000

pkt-rate-high: 450000

rx-usecs: 16

rx-frames: 88

rx-usecs-irq: 0

rx-frames-irq: 0

tx-usecs: 0

tx-frames: 0

tx-usecs-irq: 0

tx-frames-irq: 0

rx-usecs-low: 0

rx-frame-low: 0

tx-usecs-low: 0

tx-frame-low: 0

rx-usecs-high: 128

rx-frame-high: 0

tx-usecs-high: 0

tx-frame-high: 0

[avnerb@bgate ~]$

[avnerb@bgate ~]$

Above settings are bad for performance under heavy traffic.

For fixing it, please use:

[avnerb@bgate ~]$ ssh mir9 sudo ethtool -C eth2 adaptive-rx off rx-usecs 0 
rx-frames 1

To see the results, use the 1st command again, and see the difference in the 3 
highlighted fields.

Original issue reported on code.google.com by [email protected] on 4 Apr 2011 at 7:19

Compiliation of sockperf in Windows is failing

What steps will reproduce the problem?
1. Compiling sockperf in Windows fails

The reason for this is that the compiler doesn't know is the type of 'uint16_t'.

The following patch fixes this issue:


commit 30be64ad3c0fcf3e761bc5abbe1b3ebf1b73fb3c
Author: Dotan Barak <[email protected]>
Date:   Wed Nov 6 14:35:32 2013 +0200

    Add missing header for compilation in Windows

    Signed-off-by: Dotan Barak <[email protected]>

diff --git a/src/Defs.h b/src/Defs.h
index 4aeb1f2..e1eca64 100644
--- a/src/Defs.h
+++ b/src/Defs.h
@@ -36,6 +36,7 @@
 #include <sys/types.h>
 #include <unordered_map>
 #include <Winbase.h>
+#include <stdint.h>

 typedef uint16_t in_port_t;

Original issue reported on code.google.com by [email protected] on 6 Nov 2013 at 1:13

Port tool to Windows OSs

imported from: http://argus-bg.dnsalias.org/issues/714

Original issue reported on code.google.com by [email protected] on 1 Apr 2011 at 1:49

ARM - Linux compile support

Would like to request for support on ARM platforms . This is not available in 
the current version 

Original issue reported on code.google.com by [email protected] on 13 Jul 2011 at 4:03

• Merged into: #47

Pongs from multi servers in multicast with underload & pingpong

imported from: http://argus-bg.dnsalias.org/issues/716

What need is this feature going to satisfy?
1.
The feature is needed in case there is one client that send MC packets to N 
servers, all of them are expected to respond. We need to manage the replies and 
know which reply came from which server. Suggestion: add server-id to the 
header, this can be instead of the C/S bit (client will use zero in this 
field). Need to decide how to assign ids to servers, perhaps use long enough 
random number.

Anyhow,
PacketTimes array is already designed to accept MC pongs from multiple servers.
Also the statistics at the end already designed to print results for each 
server separately.
The only needed thing in the client is to recognize from each server came each 
packet and put the rxTime of the packet in the appropriate column of 
PacketTimes array.

2.
This is one of the last feature that older VTG (I think thats was its name) 
used to have which vperf still does not include.

PS: Another design possibility, instead of server id, might be to you the 
recvfrom(from_addr) to map the replies into the results table.

Original issue reported on code.google.com by [email protected] on 1 Apr 2011 at 1:58