Comments (14)
xilabao
commented on June 3, 2024
1
Question: we have few runs of benchmarks which we had done with older version of LKP, is there a way to port them to new layout/changes done recently in LKP?
- remove /lkp/paths (optional) to avoid duplicate paths in the output of command
lkp rt - run
lkp merge /lkp/resultto generate new data for old results.
from lkp-tests.
bssrikanth
commented on June 3, 2024
Also seeing issues with lkp _rm:
ssh root@dellj853host echo yes | lkp _rm dellj853host rhel ebizzy v5.16'
23:18:34 /home/LKP/lkp-tests/lib/data_store.rb:84:in `from_data': undefined method `each' for nil:NilClass (NoMethodError)
23:18:34 from /home/LKP/lkp-tests/lib/data_store.rb:79:in `load'
23:18:34 from /home/LKP/lkp-tests/lib/data_store.rb:67:in `initialize'
23:18:34 from /home/LKP/lkp-tests/lib/common.rb:24:in `new'
23:18:34 from /home/LKP/lkp-tests/lib/common.rb:24:in `block in add_cached_method'
23:18:34 from /home/LKP/lkp-tests/lib/data_store.rb:241:in `open'
23:18:34 from /home/LKP/lkp-tests/lib/data_store.rb:834:in `open'
23:18:34 from /home/LKP/lkp-tests/lib/nresult_root.rb:248:in `open'
23:18:34 from /home/LKP/lkp-tests/lib/nresult_root.rb:278:in `initialize'
23:18:34 from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `new'
23:18:34 from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `mrt_table_set'
23:18:34 from /home/LKP/lkp-tests/lib/result_root.rb:384:in `delete_mrt'
23:18:34 from /home/LKP/lkp-tests/lkp-exec/delete_mrts:12:in `block in main'
23:18:34 from /home/LKP/lkp-tests/lkp-exec/delete_mrts:8:in `each_line'
23:18:34 from /home/LKP/lkp-tests/lkp-exec/delete_mrts:8:in `main'
23:18:34 from /home/LKP/lkp-tests/lkp-exec/delete_mrts:16:in `<main>'
23:18:34 Done
from lkp-tests.
bssrikanth
commented on June 3, 2024
I see a comment in #129 that some of regressions are fixed, so will retry with latest branch.
from lkp-tests.
bssrikanth
commented on June 3, 2024
I see a comment in #129 that some of regressions are fixed, so will retry with latest branch.
I saw that fix posted #129 was for split-job, I still see issues with ncompare and _rm on latest build:
[root@dellj853host LKP]# lkp ncompare -s commit=5.15.2-v5.15 -s commit=5.16.2-v5.16 -a -k
Traceback (most recent call last):
17: from /home/LKP/lkp-tests/sbin/ncompare:6:in `<main>'
16: from /home/LKP/lkp-tests/lib/compare.rb:1150:in `compare'
15: from /home/LKP/lkp-tests/lib/compare.rb:1140:in `parse_argv'
14: from /home/LKP/lkp-tests/lib/compare.rb:1140:in `map'
13: from /home/LKP/lkp-tests/lib/compare.rb:1142:in `block in parse_argv'
12: from /home/LKP/lkp-tests/lib/compare.rb:1142:in `to_a'
11: from /home/LKP/lkp-tests/lib/nresult_root.rb:391:in `each'
10: from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `mrt_table_set'
9: from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `new'
8: from /home/LKP/lkp-tests/lib/nresult_root.rb:278:in `initialize'
7: from /home/LKP/lkp-tests/lib/nresult_root.rb:248:in `open'
6: from /home/LKP/lkp-tests/lib/data_store.rb:834:in `open'
5: from /home/LKP/lkp-tests/lib/data_store.rb:241:in `open'
4: from /home/LKP/lkp-tests/lib/common.rb:24:in `block in add_cached_method'
3: from /home/LKP/lkp-tests/lib/common.rb:24:in `new'
2: from /home/LKP/lkp-tests/lib/data_store.rb:67:in `initialize'
1: from /home/LKP/lkp-tests/lib/data_store.rb:79:in `load'
/home/LKP/lkp-tests/lib/data_store.rb:84:in `from_data': undefined method `each' for nil:NilClass (NoMethodError)
[root@dellj853host LKP]# lkp _rm dellj853host rhel ebizzy v5.16
The command will remove the following 1 _result_root:
/lkp/result/ebizzy/10s-10x-200%/dellj853host/rhel/defconfig/gcc/5.16.2-v5.16/
!!! Do you really want to do that? [No/yes]: yes
Traceback (most recent call last):
15: from /home/LKP/lkp-tests/lkp-exec/delete_mrts:16:in `<main>'
14: from /home/LKP/lkp-tests/lkp-exec/delete_mrts:8:in `main'
13: from /home/LKP/lkp-tests/lkp-exec/delete_mrts:8:in `each_line'
12: from /home/LKP/lkp-tests/lkp-exec/delete_mrts:12:in `block in main'
11: from /home/LKP/lkp-tests/lib/result_root.rb:384:in `delete_mrt'
10: from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `mrt_table_set'
9: from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `new'
8: from /home/LKP/lkp-tests/lib/nresult_root.rb:278:in `initialize'
7: from /home/LKP/lkp-tests/lib/nresult_root.rb:248:in `open'
6: from /home/LKP/lkp-tests/lib/data_store.rb:834:in `open'
5: from /home/LKP/lkp-tests/lib/data_store.rb:241:in `open'
4: from /home/LKP/lkp-tests/lib/common.rb:24:in `block in add_cached_method'
3: from /home/LKP/lkp-tests/lib/common.rb:24:in `new'
2: from /home/LKP/lkp-tests/lib/data_store.rb:67:in `initialize'
1: from /home/LKP/lkp-tests/lib/data_store.rb:79:in `load'
/home/LKP/lkp-tests/lib/data_store.rb:84:in `from_data': undefined method `each' for nil:NilClass (NoMethodError)
Done
from lkp-tests.
xilabao
commented on June 3, 2024
Hi @bssrikanth, please update to the latest code and remove the /lkp/data/mresult_root/linux_perf directory, could you give it a shot?
from lkp-tests.
bssrikanth
commented on June 3, 2024
@xilabao I updated lkp to latest code. Still I see same issue after deleting the mentioned folder.
commit 5953709d9512933e335ddb8c297ec5dec2f82804 (origin/master, origin/HEAD)
Author: Chen Rong <[email protected]>
Date: Thu Mar 31 15:14:07 2022 +0800
lib/data_store.rb: fix 297:in 'initialize': wrong number of arguments
Traceback (most recent call last):
6: from /lkp-tests/bin/run-local:165:in `<main>'
5: from /lkp-tests/lib/nresult_root.rb:339:in `create_tables_layout'
4: from /lkp-tests/lib/nresult_root.rb:257:in `create_layout'
3: from /lkp-tests/lib/nresult_root.rb:243:in `create_layout'
2: from /lkp-tests/lib/data_store.rb:225:in `add_index'
1: from /lkp-tests/lib/data_store.rb:225:in `new'
/lkp-tests/lib/data_store.rb:297:in `initialize': wrong number of arguments (given 2, expected 1) (ArgumentError)
Fixes: 9e624cd599f5 (rubocop: cleanup Style/OptionalBooleanParameter offenses)
Signed-off-by: Chen Rong <[email protected]>
Signed-off-by: Philip Li <[email protected]>
[root@dellj853host lkp-tests]# ls -alrt /lkp/data/mresult_root/linux_perf
ls: cannot access '/lkp/data/mresult_root/linux_perf': No such file or directory
[root@dellj853host lkp-tests]# lkp result ebizzy
/lkp/result/ebizzy/10s-10x-200%/dellj853host/rhel/defconfig/gcc/5.16.2-build43/0/
/lkp/result/ebizzy/10s-10x-200%/dellj853host/rhel/defconfig/gcc/5.16.2-v5.16/0/
[root@dellj853host lkp-tests]# lkp result hackbench
/lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-build43/0/
/lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-v5.16/0/
[root@dellj853host lkp-tests]# lkp ncompare -s testcase=hackbench/commit=5.16.2-build43 -s commit=5.16.2-v5.16 -a -k
Traceback (most recent call last):
17: from /home/LKP/lkp-tests/sbin/ncompare:6:in `<main>'
16: from /home/LKP/lkp-tests/lib/compare.rb:1150:in `compare'
15: from /home/LKP/lkp-tests/lib/compare.rb:1140:in `parse_argv'
14: from /home/LKP/lkp-tests/lib/compare.rb:1140:in `map'
13: from /home/LKP/lkp-tests/lib/compare.rb:1142:in `block in parse_argv'
12: from /home/LKP/lkp-tests/lib/compare.rb:1142:in `to_a'
11: from /home/LKP/lkp-tests/lib/nresult_root.rb:388:in `each'
10: from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `mrt_table_set'
9: from /home/LKP/lkp-tests/lib/nresult_root.rb:345:in `new'
8: from /home/LKP/lkp-tests/lib/nresult_root.rb:278:in `initialize'
7: from /home/LKP/lkp-tests/lib/nresult_root.rb:248:in `open'
6: from /home/LKP/lkp-tests/lib/data_store.rb:834:in `open'
5: from /home/LKP/lkp-tests/lib/data_store.rb:241:in `open'
4: from /home/LKP/lkp-tests/lib/common.rb:24:in `block in add_cached_method'
3: from /home/LKP/lkp-tests/lib/common.rb:24:in `new'
2: from /home/LKP/lkp-tests/lib/data_store.rb:67:in `initialize'
1: from /home/LKP/lkp-tests/lib/data_store.rb:79:in `load'
/home/LKP/lkp-tests/lib/data_store.rb:84:in `from_data': undefined method `each' for nil:NilClass (NoMethodError)
from lkp-tests.
xilabao
commented on June 3, 2024
@bssrikanth sorry, forgot to mention that old results are broken, you need to run lkp merge RESUTL_ROOT , .e.g lkp merge /lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-build43/
from lkp-tests.
bssrikanth
commented on June 3, 2024
@bssrikanth sorry, forgot to mention that old results are broken, you need to run
lkp merge RESUTL_ROOT, .e.glkp merge /lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-build43/
I had not used old results. The results folder I mentioned were freshly run with lkp installed from latest lkp-tests master. Hope that would not require lkp merge, please confirm.
from lkp-tests.
xilabao
commented on June 3, 2024
were these results generated (uselkp run) by new code after you removed /lkp/data/mresult_root/linux_perf ? @bssrikanth
[root@dellj853host lkp-tests]# lkp result hackbench
/lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-build43/0/
/lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-v5.16/0/
from lkp-tests.
bssrikanth
commented on June 3, 2024
were these results generated (use
lkp run) by new code after you removed/lkp/data/mresult_root/linux_perf? @bssrikanth[root@dellj853host lkp-tests]# lkp result hackbench /lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-build43/0/ /lkp/result/hackbench/socket-8-threads-1600%/dellj853host/rhel/defconfig/gcc/5.16.2-v5.16/0/
Yes that's correct, new results after removing that folder
from lkp-tests.
xilabao
commented on June 3, 2024
Below are my results, can you share the results in your environment?
# rm -r /lkp/data/mresult_root/linux_perf/
# lkp run hackbench-socket-8-threads-1600%.yaml
...
# tree /lkp/data/mresult_root/linux_perf/
/lkp/data/mresult_root/linux_perf/
|-- index
| |-- DataStore::AxisIndex-commit
| | |-- 5.
| | | `-- 5.13.0-37-generic
| | `-- index.yaml
| `-- DataStore::DateIndex
| |-- 2022-03-31
| `-- index.yaml
|-- layout.yaml
`-- storage
`-- 47
`-- 47e5abb469d4de2df7a6f90d37279f42b57cedae -> /lkp/result/hackbench/socket-8-threads-1600%/rongch2-desk1/debian/defconfig/gcc-8/5.13.0-37-generic
# cat /lkp/data/mresult_root/linux_perf/layout.yaml
---
:maps: []
:comrpess_matrix:
from lkp-tests.
bssrikanth
commented on June 3, 2024
@xilabao below are more details,
Recreate steps with latest lkp code [ head -> 5953709 ] , noticed that stack trace is a bit different compared to what I hit before:
1. Remove linux_perf, before running hackbench with latest lkp:
[root@dellj853host LKP]# rm -rf /lkp/data/mresult_root/linux_perf/
2. Run hackbench
[root@dellj853host LKP]# lkp run ./hackbench.yaml
..
3. Note linux_perf and layout contents:
[root@dellj853host LKP]# tree /lkp/data/mresult_root/linux_perf/
/lkp/data/mresult_root/linux_perf/
├── index
│ ├── DataStore::AxisIndex-commit
│ │ ├── 5.
│ │ │ └── 5.16.2-v5.16-amdsos
│ │ └── index.yaml
│ └── DataStore::DateIndex
│ ├── 2022-04-01
│ └── index.yaml
├── layout.yaml
└── storage
└── 1a
└── 1a8cbf91d684e86a2f6f46002783f0f677bd8c22 -> /lkp/result/hackbench/socket-1-threads-1600%/dellj853host.amd.com/rhel/defconfig/gcc/5.16.2-v5.16-amdsos
7 directories, 5 files
[root@dellj853host LKP]# cat /lkp/data/mresult_root/linux_perf/layout.yaml
---
:maps: []
:comrpess_matrix:
[root@dellj853host LKP]#
4. Reboot to a different kernel, to run hackbench on it
[root@dellj853host LKP]# lkp run ./hackbench.yaml
..
[root@dellj853host LKP]# tree /lkp/data/mresult_root/linux_perf/
/lkp/data/mresult_root/linux_perf/
├── index
│ ├── DataStore::AxisIndex-commit
│ │ ├── 5.
│ │ │ ├── 5.15.2-v5.15-amdsos
│ │ │ └── 5.16.2-v5.16-amdsos
│ │ └── index.yaml
│ └── DataStore::DateIndex
│ ├── 2022-04-01
│ └── index.yaml
├── layout.yaml
└── storage
├── 1a
│ └── 1a8cbf91d684e86a2f6f46002783f0f677bd8c22 -> /lkp/result/hackbench/socket-1-threads-1600%/dellj853host.amd.com/rhel/defconfig/gcc/5.16.2-v5.16-amdsos
└── d3
└── d308f409c80b4bc52690854d1d6517ff0dfc952c -> /lkp/result/hackbench/socket-1-threads-1600%/dellj853host.amd.com/rhel/defconfig/gcc/5.15.2-v5.15-amdsos
9 directories, 6 files
[root@dellj853host LKP]# cat /lkp/data/mresult_root/linux_perf/layout.yaml
---
:maps: []
:comrpess_matrix:
5. Verify the results:
[root@dellj853host LKP]# lkp result hackbench
/lkp/result/hackbench/socket-1-threads-1600%/dellj853host.amd.com/rhel/defconfig/gcc/5.15.2-v5.15-amdsos/0/
/lkp/result/hackbench/socket-1-threads-1600%/dellj853host.amd.com/rhel/defconfig/gcc/5.16.2-v5.16-amdsos/0/
6. Run ncompare:
[root@dellj853host LKP]# lkp ncompare -s commit=5.15.2-v5.15-amdsos -s commit=5.16.2-v5.16-amdsos -a -k
Traceback (most recent call last):
10: from /home/LKP/lkp-tests/sbin/ncompare:6:in `<main>'
9: from /home/LKP/lkp-tests/lib/compare.rb:1150:in `compare'
8: from /home/LKP/lkp-tests/lib/compare.rb:1140:in `parse_argv'
7: from /home/LKP/lkp-tests/lib/compare.rb:1140:in `map'
6: from /home/LKP/lkp-tests/lib/compare.rb:1142:in `block in parse_argv'
5: from /home/LKP/lkp-tests/lib/compare.rb:1142:in `to_a'
4: from /home/LKP/lkp-tests/lib/nresult_root.rb:391:in `each'
3: from /home/LKP/lkp-tests/lib/nresult_root.rb:391:in `each'
2: from /home/LKP/lkp-tests/lib/nresult_root.rb:392:in `block in each'
1: from /home/LKP/lkp-tests/lib/nresult_root.rb:383:in `block in each'
/home/LKP/lkp-tests/lib/data_store.rb:753:in `each': undefined method `each' for nil:NilClass (NoMethodError)
from lkp-tests.
xilabao
commented on June 3, 2024
@bssrikanth I can't reproduce it, can you remove the whole directory "/lkp/data/mresult_root/" and try again?
from lkp-tests.
bssrikanth
commented on June 3, 2024
@bssrikanth I can't reproduce it, can you remove the whole directory "/lkp/data/mresult_root/" and try again?
@xilabao that worked..
Removed whole mresult_root folder, and then ran hackbench on 2 kernel levels:
[root@dellj853host LKP]# lkp ncompare -s commit=5.15.2-v5.15-amdsos -s commit=5.16.2-v5.16-amdsos -a -k
=========================================================================================
compiler/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase:
gcc/socket/1/defconfig/threads/1600%/rhel/dellj853host.amd.com/hackbench
commit:
5.15.2-v5.15-amdsos
5.16.2-v5.16-amdsos
5.15.2-v5.15-amd 5.16.2-v5.16-amdsos
---------------- ---------------------------
%stddev %change %stddev
\ | \
275907 +10.9% 306109 hackbench.throughput
Question: we have few runs of benchmarks which we had done with older version of LKP, is there a way to port them to new layout/changes done recently in LKP?
from lkp-tests.
Related Issues (20)
- stats/kernel-selftests: resolve rubocop issues of VmStater class HOT 3
- stats/kernel-selftests: resolve rubocop issues of X86Stater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of ResctrlStater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of CapabilitiesStater class HOT 4
- stats/kernel-selftests: resolve rubocop issues of KmodStater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of NetfilterStater class HOT 1
- stats/kernel-selftests: resolve rubocop issues of BpfStater class
- stats/kernel-selftests: resolve rubocop issues of VmallocStater class HOT 1
- perf-sanity-tests: analyze subtests of each perf test
- docker: support fedora:38 HOT 1
- docker: support ubuntu:22.04 HOT 1
- docker: support archlinux:base HOT 1
- locktorture: enable extra torture_type
- kunit: enable extra tests
- hwsim: resolve git clone timeout HOT 1
- Data_store error after lkp run in fedora:38 & archlinux:base docker
- Suggest using the `Closes:` tag instead of the `Link:` one HOT 2
- lkp install throws multiple errors HOT 1
- fail to install will-it-scale on fedora 38 HOT 1
- Incorrect paths and deprecated methods
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lkp-tests.