sockperf crashed due to segment fault when using libvma

[g199209]

sockperf crashed due to segment fault when using libvma

[g199209]

The v5.8-2.0.3.0 LTS MLNX_OFED driver contains v3.10 sockperf, it also crashed.

[igor-ivanov]

Hello @g199209.
Could you clarify

• the issue is reproduced with 4716b150645180735fdbf816adede0a39dc64e86
• the issue does not seen before 4716b150645180735fdbf816adede0a39dc64e86
• the issue exists in sockperf v3.10 too
• libvma is taken from v5.8-2.0.3.0 LTS MLNX_OFED

Is it correct?

[g199209]

Hello @g199209. Could you clarify

• the issue is reproduced with 4716b150645180735fdbf816adede0a39dc64e86
• the issue does not seen before 4716b150645180735fdbf816adede0a39dc64e86
• the issue exists in sockperf v3.10 too
• libvma is taken from v5.8-2.0.3.0 LTS MLNX_OFED

Is it correct?

Yes.

[g199209]

Is there any plan to fix this bug?

[g199209]

@igor-ivanov Any progress?

[igor-ivanov]

Hello @g199209,

The failure you described does not happen on my setup.
libvma(v9.7.2) - from MLNX OFED 5.8-2.0.3.0
sockperf(version #3.8-21.git4716b1506451)

server:

$ sudo sockperf sr -i 192.168.105.3 --load-vma=libvma.so
 VMA INFO: ---------------------------------------------------------------------------
 VMA INFO: VMA_VERSION: 9.7.2-1 Release built on Nov 14 2022 17:03:52
 VMA INFO: Cmd Line: sockperf sr -i 192.168.105.3 --load-vma=libvma.so
 VMA INFO: OFED Version: MLNX_OFED_LINUX-5.9-0.5.5.2:
 VMA INFO: ---------------------------------------------------------------------------
 VMA INFO: Log Level                      INFO                       [VMA_TRACELEVEL]
 VMA INFO: ---------------------------------------------------------------------------
sockperf: == version #3.8-21.git4716b1506451 ==
sockperf: [SERVER] listen on:
[ 0] IP = 192.168.105.3   PORT = 11111 # UDP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: [tid 1570954] using recvfrom() to block on socket(s)
^Csockperf: Test end (interrupted by user)
sockperf: Total 204243 messages received and handled
sockperf: cleanupAfterLoop() exit

client:

$ sudo sockperf pp -i 192.168.105.3 --load-vma=libvma.so
 VMA INFO: ---------------------------------------------------------------------------
 VMA INFO: VMA_VERSION: 9.7.2-1 Release built on Nov 14 2022 17:03:52
 VMA INFO: Cmd Line: sockperf pp -i 192.168.105.3 --load-vma=libvma.so
 VMA INFO: OFED Version: MLNX_OFED_LINUX-5.9-0.5.5.2:
 VMA INFO: ---------------------------------------------------------------------------
 VMA INFO: Log Level                      INFO                       [VMA_TRACELEVEL]
 VMA INFO: ---------------------------------------------------------------------------
sockperf: == version #3.8-21.git4716b1506451 ==
sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s)

[ 0] IP = 192.168.105.3   PORT = 11111 # UDP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=1.000 sec; Warm up time=400 msec; SentMessages=204243; ReceivedMessages=204242
sockperf: ========= Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=0.550 sec; SentMessages=126961; ReceivedMessages=126961
sockperf: ====> avg-latency=2.155 (std-dev=0.271, mean-ad=0.087, median-ad=0.103, siqr=0.075, cv=0.126, std-error=0.001, 99.0% ci=[2.153, 2.157])
sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
sockperf: Summary: Latency is 2.155 usec
sockperf: Total 126961 observations; each percentile contains 1269.61 observations
sockperf: ---> <MAX> observation =   48.565
sockperf: ---> percentile 99.999 =   45.589
sockperf: ---> percentile 99.990 =    4.478
sockperf: ---> percentile 99.900 =    2.634
sockperf: ---> percentile 99.000 =    2.374
sockperf: ---> percentile 90.000 =    2.264
sockperf: ---> percentile 75.000 =    2.224
sockperf: ---> percentile 50.000 =    2.184
sockperf: ---> percentile 25.000 =    2.073
sockperf: ---> <MIN> observation =    1.823

It does not happen with sockperf v3.10 too.
Please consider compiling current libvma (https://github.com/Mellanox/libvma) and sockperf from sources and check the failure case on your setup.

[igor-ivanov]

@g199209 please share more details to reproduce the issue or close it in case it is not observed.

[g199209]

@g199209 please share more details to reproduce the issue or close it in case it is not observed.

I'll compile libvma from source and test again these days asap.

[g199209]

@igor-ivanov My colleague have tried to build libvma, but failed: Mellanox/libvma#1053

[igor-ivanov]

@igor-ivanov My colleague have tried to build libvma, but failed: Mellanox/libvma#1053

From my understanding it succeeded to build but could not find a way to create package.

[g199209]

@igor-ivanov My colleague have tried to build libvma, but failed: Mellanox/libvma#1053

From my understanding it succeeded to build but could not find a way to create package.

We suspect that this may not be the correct way to build. Do you also use it to build and create packages manually later?

[igor-ivanov]

@igor-ivanov My colleague have tried to build libvma, but failed: Mellanox/libvma#1053

From my understanding it succeeded to build but could not find a way to create package.

We suspect that this may not be the correct way to build. Do you also use it to build and create packages manually later?

What issue do you see with build_pkg.sh? It is used in practice now.

[igor-ivanov]

@g199209 have you had a chance for the issue reproduction?