Comments (14)
Commit is the kernel version you use, for example "3.10.0-862.el7.x86_64" in your results. But you need to compare different kernel versions, so you need to run the test in another kernel version too.
It's possible to compare with other axis key. Please try
$ lkp ncompare -h
And use "-c" to specify axis key.
from lkp-tests.
Will try to upgrade the kernel and run a benchmark (unixbench) to compare.
Sorry I didn't get the idea to run lkp ncompare
against the following results:
[root@ip-10-22-2-226 lkp]# lkp result fio-basic
/lkp/result/fio-basic/10s-8t-1SSD-xfs-read-4k-sync-5g/ip-10-22-2-226.us-west-2.compute.internal/rhel/x86_64-rhel-7.2/gcc-7/3.10.0-862.el7.x86_64/0/
/lkp/result/fio-basic/10s-8t-1SSD-xfs-read-4k-sync-5g/ip-10-22-2-226.us-west-2.compute.internal/rhel/x86_64-rhel-7.2/gcc-7/3.10.0-862.el7.x86_64/1/
Could you help clarify the usage and/or give me an example?
Sorry for inconvenience.
from lkp-tests.
After trying another kernel, I still can not run lkp ncompare
successfully.
[root@ip-10-22-2-226 gcc]# pwd
/lkp/result/unixbench/30s-_8_-__whetstone-double____shell1___/ip-10-22-2-226.us-west-2.compute.internal/rhel/defconfig/gcc
[root@ip-10-22-2-226 gcc]# ls
3.10.0-862.19.1.el7.x86_64 3.10.0-862.el7.x86_64
[root@ip-10-22-2-226 gcc]#
[root@ip-10-22-2-226 gcc]# cat ./3.10.0-862.19.1.el7.x86_64/0/stats.json
{
"unixbench.score": 1913.7,
"unixbench.workload": 33146292187.8,
"unixbench.time.user_time": 332.02,
"unixbench.time.system_time": 455.92,
"unixbench.time.percent_of_cpu_this_job_got": 155,
"unixbench.time.elapsed_time": 506.3,
"unixbench.time.elapsed_time.max": 506.3,
"unixbench.time.maximum_resident_set_size": 6176,
"unixbench.time.major_page_faults": 14,
"unixbench.time.minor_page_faults": 88560475,
"unixbench.time.voluntary_context_switches": 8687259,
"unixbench.time.involuntary_context_switches": 77727,
"unixbench.time.file_system_inputs": 7728,
"unixbench.time.file_system_outputs": 2453808,
"unixbench.time.page_size": 4096,
"time.user_time": 332.02,
"time.system_time": 455.92,
"time.percent_of_cpu_this_job_got": 155,
"time.elapsed_time": 506.3,
"time.elapsed_time.max": 506.3,
"time.maximum_resident_set_size": 6176,
"time.major_page_faults": 14,
"time.minor_page_faults": 88560475,
"time.voluntary_context_switches": 8687259,
"time.involuntary_context_switches": 77727,
"time.file_system_inputs": 7728,
"time.file_system_outputs": 2453808,
"time.page_size": 4096
}[root@ip-10-22-2-226 gcc]# cat ./3.10.0-862.el7.x86_64/0/stats.json
{
"unixbench.score": 1854.5,
"unixbench.workload": 33124226438.600002,
"unixbench.time.user_time": 328.77,
"unixbench.time.system_time": 459.53,
"unixbench.time.percent_of_cpu_this_job_got": 155,
"unixbench.time.elapsed_time": 506.46,
"unixbench.time.elapsed_time.max": 506.46,
"unixbench.time.maximum_resident_set_size": 6184,
"unixbench.time.minor_page_faults": 86516980,
"unixbench.time.voluntary_context_switches": 7950482,
"unixbench.time.involuntary_context_switches": 77330,
"unixbench.time.file_system_outputs": 2435096,
"unixbench.time.page_size": 4096,
"time.user_time": 328.77,
"time.system_time": 459.53,
"time.percent_of_cpu_this_job_got": 155,
"time.elapsed_time": 506.46,
"time.elapsed_time.max": 506.46,
"time.maximum_resident_set_size": 6184,
"time.minor_page_faults": 86516980,
"time.voluntary_context_switches": 7950482,
"time.involuntary_context_switches": 77330,
"time.file_system_outputs": 2435096,
"time.page_size": 4096
}
The lkp ncompare
command has no outputs.
[root@ip-10-22-2-226 gcc]# lkp result unixbench
/lkp/result/unixbench/30s-_8_-__whetstone-double____shell1___/ip-10-22-2-226.us-west-2.compute.internal/rhel/defconfig/gcc/3.10.0-862.19.1.el7.x86_64/0/
/lkp/result/unixbench/30s-_8_-__whetstone-double____shell1___/ip-10-22-2-226.us-west-2.compute.internal/rhel/defconfig/gcc/3.10.0-862.el7.x86_64/0/
[root@ip-10-22-2-226 gcc]# lkp ncompare -s commit='3.10.0-862.el7.x86_64' -o commit='3.10.0-900.el7.x86_64'
[root@ip-10-22-2-226 gcc]#
from lkp-tests.
At least 3 runs for 1 kernel are needed to compare.
from lkp-tests.
At least 3 runs for 1 kernel are needed to compare.
OK. Will try it. Suggest to add some outputs there if no enough samples. Thanks!
from lkp-tests.
At least 3 runs for 1 kernel are needed to compare.
OK. Will try it. Suggest to add some outputs there if no enough samples. Thanks!
Still not work after 3 runs, is there any other mistakes?
[root@ip-10-22-2-226 gcc]# pwd
/lkp/result/unixbench/30s-_8_-__whetstone-double____shell1___/ip-10-22-2-226.us-west-2.compute.internal/rhel/defconfig/gcc
[root@ip-10-22-2-226 gcc]# ls -lad */*
drwxrwsr-x. 3 root root 4096 Sep 29 03:53 3.10.0-862.19.1.el7.x86_64/0
drwxrwsr-x. 3 root root 4096 Sep 29 05:38 3.10.0-862.19.1.el7.x86_64/1
drwxrwsr-x. 3 root root 4096 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/2
-rw-r--r--. 1 root root 1188 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/avg.csv
-rw-r--r--. 1 root root 1358 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/avg.json
-rw-r--r--. 1 root root 399 Sep 29 03:53 3.10.0-862.19.1.el7.x86_64/desc.yaml
-rw-r--r--. 1 root root 1816 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/matrix.csv
-rw-r--r--. 1 root root 2522 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/matrix.json
-rw-r--r--. 1 root root 1258 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/stddev.csv
-rw-r--r--. 1 root root 1428 Sep 29 05:46 3.10.0-862.19.1.el7.x86_64/stddev.json
drwxrwsr-x. 3 root root 4096 Sep 29 02:52 3.10.0-862.el7.x86_64/0
drwxrwsr-x. 3 root root 4096 Sep 29 06:04 3.10.0-862.el7.x86_64/1
drwxrwsr-x. 3 root root 4096 Sep 29 06:14 3.10.0-862.el7.x86_64/2
-rw-r--r--. 1 root root 1235 Sep 29 06:14 3.10.0-862.el7.x86_64/avg.csv
-rw-r--r--. 1 root root 1405 Sep 29 06:14 3.10.0-862.el7.x86_64/avg.json
-rw-r--r--. 1 root root 394 Sep 29 02:52 3.10.0-862.el7.x86_64/desc.yaml
-rw-r--r--. 1 root root 1824 Sep 29 06:14 3.10.0-862.el7.x86_64/matrix.csv
-rw-r--r--. 1 root root 2558 Sep 29 06:14 3.10.0-862.el7.x86_64/matrix.json
-rw-r--r--. 1 root root 1257 Sep 29 06:14 3.10.0-862.el7.x86_64/stddev.csv
-rw-r--r--. 1 root root 1427 Sep 29 06:14 3.10.0-862.el7.x86_64/stddev.json
[root@ip-10-22-2-226 gcc]#
[root@ip-10-22-2-226 gcc]# lkp ncompare -s commit='3.10.0-862.el7.x86_64' -o commit='3.10.0-900.el7.x86_64'
[root@ip-10-22-2-226 gcc]# echo $?
0
[root@ip-10-22-2-226 gcc]#
from lkp-tests.
Try
$ lkp ncompare -s commit='3.10.0-862.19.1.el7.x86_64' -o commit='3.10.0-862.el7.x86_64'
and
$ lkp ncompare -s commit='3.10.0-862.19.1.el7.x86_64' -o commit='3.10.0-862.el7.x86_64' -a
?
from lkp-tests.
The command with -a
worked.
[root@ip-10-22-2-226 ~]# lkp ncompare -s commit='3.10.0-862.19.1.el7.x86_64' -o commit='3.10.0-862.el7.x86_64'
[root@ip-10-22-2-226 ~]# echo $?
0
[root@ip-10-22-2-226 ~]# lkp ncompare -s commit='3.10.0-862.19.1.el7.x86_64' -o commit='3.10.0-862.el7.x86_64' -a
=========================================================================================
compiler/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase:
gcc/defconfig/[8]/rhel/30s/ip-10-22-2-226.us-west-2.compute.internal/["whetstone-double", "shell1", "shell8", "pipe", "context1", "spawn", "syscall", "dhry2reg", "fstime", "fsbuffer", "fsdisk"]/unixbench
commit:
3.10.0-862.19.1.el7.x86_64
3.10.0-862.el7.x86_64
3.10.0-862.19.1. 3.10.0-862.el7.x86_64
---------------- --------------------------
%stddev %change %stddev
\ | \
1901 -0.9% 1885 unixbench.score
506.37 +0.0% 506.48 unixbench.time.elapsed_time
506.37 +0.0% 506.48 unixbench.time.elapsed_time.max
2576 ±141% +8.9% 2806 ±141% unixbench.time.file_system_inputs
2456501 +0.9% 2477546 unixbench.time.file_system_outputs
77728 -0.3% 77488 unixbench.time.involuntary_context_switches
4.67 ±141% +14.3% 5.33 ±141% unixbench.time.major_page_faults
6178 +0.0% 6181 unixbench.time.maximum_resident_set_size
88371177 -0.1% 88292389 unixbench.time.minor_page_faults
4096 +0.0% 4096 unixbench.time.page_size
155.00 +0.0% 155.00 unixbench.time.percent_of_cpu_this_job_got
456.57 +0.0% 456.79 unixbench.time.system_time
331.53 -0.0% 331.45 unixbench.time.user_time
8632176 -2.6% 8407320 ± 3% unixbench.time.voluntary_context_switches
3.311e+10 -0.0% 3.31e+10 unixbench.workload
506.37 +0.0% 506.48 time.elapsed_time
506.37 +0.0% 506.48 time.elapsed_time.max
2576 ±141% +8.9% 2806 ±141% time.file_system_inputs
2456501 +0.9% 2477546 time.file_system_outputs
77728 -0.3% 77488 time.involuntary_context_switches
4.67 ±141% +14.3% 5.33 ±141% time.major_page_faults
6178 +0.0% 6181 time.maximum_resident_set_size
88371177 -0.1% 88292389 time.minor_page_faults
4096 +0.0% 4096 time.page_size
155.00 +0.0% 155.00 time.percent_of_cpu_this_job_got
456.57 +0.0% 456.79 time.system_time
331.53 -0.0% 331.45 time.user_time
8632176 -2.6% 8407320 ± 3% time.voluntary_context_switches
[root@ip-10-22-2-226 ~]#
from lkp-tests.
I guess the two kernel versions are too similar. There's no performance difference, so you need "-a".
from lkp-tests.
May I run lkp-tests
on 2 different hosts (the same capacity) and use lkp ncompare
to summary the benchmark results? If so, I can compare RHEL7.5 against RHEL7.6, there should be some difference.
from lkp-tests.
It's not easy to merge results from different machine now yet. That will be a TODO for us. Maybe you can close this one and submit a new issue for that? So we will keep track of it and make it resolved.
from lkp-tests.
It's not easy to merge results from different machine now yet. That will be a TODO for us. Maybe you can close this one and submit a new issue for that? So we will keep track of it and make it resolved.
I'm all for it, this feature would be very useful.
BTW, still one more question:
Can lkp ncompare
be used for comparing the KPI of 2 different SSD devices based on the same kernel? I guess no, but if it can, please let me know. Thanks!
from lkp-tests.
It's possible, but tricky. Firstly you need a way to differentiate 2 SSD devices. Now in host file, you cannot do that. That is a feature request. It's not only for lkp ncompare
, but also for lkp run
. So maybe create another issue for this too. Thanks for pointing this out.
from lkp-tests.
It's possible, but tricky. Firstly you need a way to differentiate 2 SSD devices. Now in host file, you cannot do that. That is a feature request. It's not only for
lkp ncompare
, but also forlkp run
. So maybe create another issue for this too. Thanks for pointing this out.
Thanks, will do!
from lkp-tests.
Related Issues (20)
- stats/kernel-selftests: resolve rubocop issues of VmStater class HOT 3
- stats/kernel-selftests: resolve rubocop issues of X86Stater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of ResctrlStater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of CapabilitiesStater class HOT 4
- stats/kernel-selftests: resolve rubocop issues of KmodStater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of NetfilterStater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of BpfStater class
- stats/kernel-selftests: resolve rubocop issues of VmallocStater class HOT 1
- perf-sanity-tests: analyze subtests of each perf test
- docker: support fedora:38 HOT 1
- docker: support ubuntu:22.04 HOT 1
- docker: support archlinux:base HOT 1
- locktorture: enable extra torture_type
- kunit: enable extra tests
- hwsim: resolve git clone timeout HOT 1
- Data_store error after lkp run in fedora:38 & archlinux:base docker
- Suggest using the `Closes:` tag instead of the `Link:` one HOT 2
- lkp install throws multiple errors HOT 1
- fail to install will-it-scale on fedora 38 HOT 1
- Incorrect paths and deprecated methods
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lkp-tests.