rm due to not enough HDD disks

[intel]

rm due to not enough HDD disks

[liuyd96]

Sorry for delay.
please check whether lkp-tests/host/${HOSTNAME} exists.

1. If lkp-tests/host/${HOSTNAME} exist.
   You could edit hdd/ssd partitions directly to specify testing partitions. Could refer to other files under lkp-tests/hosts

liuyd@liuyd-ubuntu1804:~/lkp-tests/hosts$ cat liuyd-ubuntu1804
nr_cpu: 2
memory: 3G
hdd_partitions:
ssd_partitions:
local_run: 1

2. You could specify hdd/ssd when you install job by option "--hdd/--ssd"
   lkp install --hdd /dev/testing_hdd ./iozone-2HDD-ext4-kyber.yaml .

liuyd@liuyd-ubuntu1804:~/lkp-tests/hosts$ sudo lkp install --help
[sudo] password for liuyd:
Usage: lkp install [options] [<script>/]
options:
-f|--force: force to install testsuites
--hdd partition: HDD partition for IO tests
--ssd partition: SSD partition for IO tests

[Ankita-020696]

Hi,
I tried these steps but did not work for me. It is giving same error. Can you provide me any other solution to fix it.
Thanks,
Ankita

[liuyd96]

Hi, @Ankita-020696.
Sorry for the delay. After add hdd partition info into lkp-tests/host/${HOSTNAME}.
Please run "lkp split-job iozone.yaml" again.

During lkp split, we added some info like memory, disk etc. to job.yaml.
And lkp parsed job.yaml to get needed value.
So when you edit host/${HOSTNAME}, you need split this job again to add new disk info.

Thanks.

1. Edit lkp-tests/host/${HOSTNAME}
2. lkp split-job iozone.yaml again (After split job, please check whether ./iozone-2HDD-ext4-kyber.yaml has hdd_partitions info)
3. lkp run ./iozone-2HDD-ext4-kyber.yaml

root@lkp-boy:/home/zhangzh/lkp-tests/jobs# lkp run ./iozone-2HDD-ext4-kyber.yaml
[snip]
Traceback (most recent call last):
6: from /home/zhangzh/lkp-tests/bin/run-local:131:in <main>' 5: from /home/zhangzh/lkp-tests/lib/job.rb:691:in expand_params'
4: from /home/zhangzh/lkp-tests/lib/job.rb:77:in for_each_in' 3: from /home/zhangzh/lkp-tests/lib/job.rb:77:in each'
2: from /home/zhangzh/lkp-tests/lib/job.rb:78:in block in for_each_in' 1: from /home/zhangzh/lkp-tests/lib/job.rb:694:in block in expand_params'

/home/zhangzh/lkp-tests/lib/job.rb:669:in `run_filter': /home/zhangzh/lkp-tests/filters/disk: exitstatus 1 (Job::ParamError)
Edit host file and split again.
root@lkp-boy:/home/zhangzh/lkp-tests/jobs# lkp split-job iozone.yaml
iozone.yaml => ./iozone-2HDD-xfs-kyber.yaml
iozone.yaml => ./iozone-2HDD-ext4-kyber.yaml
iozone.yaml => ./iozone-2HDD-btrfs-kyber.yaml
root@lkp-boy:/home/zhangzh/lkp-tests/jobs# lkp run ./iozone-2HDD-ext4-kyber.yaml
ls: cannot access '/dev/vda3': No such file or directory
ls: cannot access '/dev/vda4': No such file or directory

[liuyd96]

If you choose the way of "lkp install --hdd /dev/XXX". You need remove lkp-tests/host/${HOSTNAME} firstly.

Remove lkp-tests/host/${HOSTNAME} ==> lkp install --hdd /dev/vda1 --hdd/dev/vda2 job.yaml ==> lkp run job.yaml

[Ankita-020696]

Hi, @Ankita-020696.
Sorry for the delay. After add hdd partition info into lkp-tests/host/${HOSTNAME}.
Please run "lkp split-job iozone.yaml" again.

During lkp split, we added some info like memory, disk etc. to job.yaml.
And lkp parsed job.yaml to get needed value.
So when you edit host/${HOSTNAME}, you need split this job again to add new disk info.

Thanks.

1. Edit lkp-tests/host/${HOSTNAME}

2. lkp split-job iozone.yaml again (After split job, please check whether  ./iozone-2HDD-ext4-kyber.yaml has hdd_partitions info)

3. lkp run   ./iozone-2HDD-ext4-kyber.yaml

root@lkp-boy:/home/zhangzh/lkp-tests/jobs# lkp run ./iozone-2HDD-ext4-kyber.yaml
[snip]
Traceback (most recent call last):
6: from /home/zhangzh/lkp-tests/bin/run-local:131:in <main>' 5: from /home/zhangzh/lkp-tests/lib/job.rb:691:in expand_params'
4: from /home/zhangzh/lkp-tests/lib/job.rb:77:in for_each_in' 3: from /home/zhangzh/lkp-tests/lib/job.rb:77:in each'
2: from /home/zhangzh/lkp-tests/lib/job.rb:78:in block in for_each_in' 1: from /home/zhangzh/lkp-tests/lib/job.rb:694:in block in expand_params'

/home/zhangzh/lkp-tests/lib/job.rb:669:in `run_filter': /home/zhangzh/lkp-tests/filters/disk: exitstatus 1 (Job::ParamError)
Edit host file and split again.
root@lkp-boy:/home/zhangzh/lkp-tests/jobs# lkp split-job iozone.yaml
iozone.yaml => ./iozone-2HDD-xfs-kyber.yaml
iozone.yaml => ./iozone-2HDD-ext4-kyber.yaml
iozone.yaml => ./iozone-2HDD-btrfs-kyber.yaml
root@lkp-boy:/home/zhangzh/lkp-tests/jobs# lkp run ./iozone-2HDD-ext4-kyber.yaml
ls: cannot access '/dev/vda3': No such file or directory
ls: cannot access '/dev/vda4': No such file or directory

Hi,
1.I tried by editing {HOSTNAME}, this is giving
Command: sudo lkp run ./iozone-2HDD-ext4-kyber.yaml
rm due to not enough HDD disks
need: 2, has: 1
Traceback (most recent call last):
6: from /home/ankita/Ankita/lkp-tests/bin/run-local:131:in <main>' 5: from /home/ankita/Ankita/lkp-tests/lib/job.rb:691:in expand_params'
4: from /home/ankita/Ankita/lkp-tests/lib/job.rb:77:in for_each_in' 3: from /home/ankita/Ankita/lkp-tests/lib/job.rb:77:in each'
2: from /home/ankita/Ankita/lkp-tests/lib/job.rb:78:in block in for_each_in' 1: from /home/ankita/Ankita/lkp-tests/lib/job.rb:694:in block in expand_params'
/home/ankita/Ankita/lkp-tests/lib/job.rb:669:in `run_filter': /home/ankita/Ankita/lkp-tests/filters/disk: exitstatus 1 (Job::ParamError)

2. When I am using "lkp install --hdd /dev/XXX" (removed hostname), I get

rm due to not enough HDD disks
need: 2, has: 0
Traceback (most recent call last):
6: from /home/ankita/Ankita/lkp-tests/bin/run-local:131:in <main>' 5: from /home/ankita/Ankita/lkp-tests/lib/job.rb:691:in expand_params'
4: from /home/ankita/Ankita/lkp-tests/lib/job.rb:77:in for_each_in' 3: from /home/ankita/Ankita/lkp-tests/lib/job.rb:77:in each'
2: from /home/ankita/Ankita/lkp-tests/lib/job.rb:78:in block in for_each_in' 1: from /home/ankita/Ankita/lkp-tests/lib/job.rb:694:in block in expand_params'
/home/ankita/Ankita/lkp-tests/lib/job.rb:669:in `run_filter': /home/ankita/Ankita/lkp-tests/filters/disk: exitstatus 1 (Job::ParamError)

Thanks

[liuyd96]

Please share your host/${HOSTNAME}, ./iozone-2HDD-ext4-kyber.yaml and cmdline.

[Ankita-020696]

host/${HOSTNAME}:
cat ankita-Inspiron-15
nr_cpu: 4
memory: 11G
hdd_partitions: 2
ssd_partitions:
local_run: 1

cat ./iozone-2HDD-ext4-kyber.yaml

#! jobs/iozone.yaml
suite: iozone
testcase: iozone
category: benchmark
disk: 2HDD
fs: ext4
iosched: kyber
iozone:
job_origin: jobs/iozone.yaml
testbox: ankita-Inspiron-15-3567
arch: x86_64
tbox_group: ankita-Inspiron-15

#! hosts/ankita-Inspiron-15
nr_cpu: 4
memory: 11G
hdd_partitions: 2
ssd_partitions:
local_run: 1

#! include/category/benchmark
kmsg:
boot-time:
uptime:
iostat:
heartbeat:
vmstat:
numa-numastat:
numa-vmstat:
numa-meminfo:
proc-vmstat:
proc-stat:
meminfo:
slabinfo:
interrupts:
lock_stat:
latency_stats:
softirqs:
bdi_dev_mapping:
diskstats:
nfsstat:
cpuidle:
cpufreq-stats:
turbostat:
sched_debug:
perf-stat:
mpstat:
perf-profile:

#! include/disk/nr_hdd
need_kconfig:

• CONFIG_BLK_DEV_SD
• CONFIG_SCSI
• CONFIG_BLOCK=y
• CONFIG_SATA_AHCI
• CONFIG_SATA_AHCI_PLATFORM
• CONFIG_ATA
• CONFIG_PCI=y
• CONFIG_BLK_DEV_DM
• CONFIG_EXT4_FS

#! include/iozone

#! include/fs/OTHERS

#! default params
kconfig: x86_64-rhel-7.6
compiler: gcc-7

Thanks

[liuyd96]

hdd_partitions: 2

You should add real disk, not number.
e.g. hdd_partitions: /dev/vda1 /dev/vda2
Please refer other host file
e.g.

liuyd@lkp-boy:~/lkp-tests/hosts$ cat lkp-ivb-2ep1
model: Ivy Bridge-EP
nr_node: 2
nr_cpu: 48
memory: 64G
nr_hdd_partitions: 3
hdd_partitions:
/dev/disk/by-id/ata-WDC_WD1003FBYZ-010FB0_WD-WCAW36795753-part1
/dev/disk/by-id/ata-WDC_WD1003FBYZ-010FB0_WD-WCAW36795753-part2
/dev/disk/by-id/ata-WDC_WD1003FBYZ-010FB0_WD-WCAW36795753-part3
rootfs_partition: /dev/disk/by-id/ata-INTEL_SSDSC2BB800G4_PHWL42040057800RGN-part1
brand: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz

After edit host file correctly, run lkp split iozone.yaml again.

[Ankita-020696]

Hi,
This worked for me.
Thanks

[Ankita-020696]

Can u tell me,

1. How much size will each partition need for iozone?
2. How iozone is calculating parent throughput & child throughput?

Thanks

[liuyd96]

Hi,

How much size will each partition need for iozone?

There is no limit about disk space i think. we use iozone option '-a' to run test.
'-a' will set testing variables automatically, e.g block size.

How iozone is calculating parent throughput & child throughput?

I'm sorry, i'm not very clear about iozone itself. Maybe you shold ask iozone community.
We just enable it in LKP testing framework and analyse the regression.