Good Afternoon,
I'm currently in the middle of testing some drives for a server upgrade. I'm trying to test an NVME using CPDT and it gives me consistent results on Windows. I've also confirmed the read and write speeds are consistent with some other colleagues as well on their Windows PC along with a Linux Server that we've tested. However, when we tested the NVME on our intended server, we get results that are comparable to that of a HDD/SSD mixed... Even in one case, where we tested the speed of our RAID SSD's, the speed of that RAID SSD was farrr higher than that of the NVME which shouldn't be the case. I've used other tools such as dd which gives us the results that we need for the NVME's (albeit we did get some inconsistent results with dd when it came to doing 8000 counts of 1M files using dd. For some odd reason, the speeds were the same with both the RAID SSD and NVME. When I did a 5GB long file, NVME's were two times faster than the SSD). We also tested another tool called "fio" and I've tried mimicking the workload similar to that of Crystal Disk Mark which shows us the results that we want for NVME speeds of the drive that we have. RAID SSD for fio also showed us the appropriated speeds as well
On that note, I was wondering if there has been any issues regarding the testing of the software when it comes to NVME's or just different OS's, etc. I'll attach some information to see if that could help you troubleshoot and if you could help us to find the root cause of this issue (if it is at all), then please do let us know. Thanks!
OS: Oracle Linux 8 (RHEL 8)
CPDT Test NVME
1/5 Sequential write [4MB] block Avg: 467.78MB/s Min÷Max: 247.05 ÷ 971.34, Time: 1m34s
2/5 Sequential read [4MB] block Avg: 771.82MB/s Min÷Max: 321.30 ÷ 2770.56, Time: 1m45s
3/5 Random write [4KB] block Avg: 0.35MB/s Min÷Max: 0.30 ÷ 2.09, Time: 0m08s
4/5 Random read [4KB] block Avg: 0.49MB/s Min÷Max: 0.37 ÷ 43.46, Time: 0m08s
5/5 Memory copy [4MB] block Avg: 3076.63MB/s Min÷Max: 2774.84 ÷ 3226.67, Time: 0m04s
Write Score*: 1.57 MB/s
Read Score*: 1.84 MB/s
CPDT Test RAID SSD
1/5 Sequential write [4MB] block Avg: 985.32MB/s Min÷Max: 758.35 ÷ 1261.51, Time: 1m58s
2/5 Sequential read [4MB] block Avg: 1223.27MB/s Min÷Max: 245.54 ÷ 3728.39, Time: 1m57s
3/5 Random write [4KB] block Avg: 13.38MB/s Min÷Max: 9.63 ÷ 15.42, Time: 0m08s
4/5 Random read [4KB] block Avg: 2.42MB/s Min÷Max: 0.39 ÷ 36.93, Time: 0m08s
5/5 Memory copy [4MB] block Avg: 2912.35MB/s Min÷Max: 2863.23 ÷ 2995.09, Time: 0m05s
Write Score*: 63.42 MB/s
Read Score*: 7.15 MB/s
fio test NVME
Run status group 0 (all jobs):
READ: bw=1777MiB/s (1863MB/s), 1777MiB/s-1777MiB/s (1863MB/s-1863MB/s), io=5000MiB (5243MB), run=2814-2814msec
Run status group 1 (all jobs):
WRITE: bw=2128MiB/s (2231MB/s), 2128MiB/s-2128MiB/s (2231MB/s-2231MB/s), io=5000MiB (5243MB), run=2350-2350msec
Run status group 2 (all jobs):
READ: bw=1238MiB/s (1298MB/s), 1238MiB/s-1238MiB/s (1298MB/s-1298MB/s), io=5000MiB (5243MB), run=4039-4039msec
Run status group 3 (all jobs):
WRITE: bw=2301MiB/s (2413MB/s), 2301MiB/s-2301MiB/s (2413MB/s-2413MB/s), io=5000MiB (5243MB), run=2173-2173msec
Run status group 4 (all jobs):
READ: bw=771MiB/s (809MB/s), 771MiB/s-771MiB/s (809MB/s-809MB/s), io=5000MiB (5243MB), run=6481-6481msec
Run status group 5 (all jobs):
WRITE: bw=742MiB/s (778MB/s), 742MiB/s-742MiB/s (778MB/s-778MB/s), io=5000MiB (5243MB), run=6735-6735msec
Disk stats (read/write):
nvme0n1: ios=1300000/1286714, merge=0/0, ticks=125861/16736, in_queue=526, util=94.94%
fio test RAID SSD
Run status group 0 (all jobs):
READ: bw=612MiB/s (641MB/s), 612MiB/s-612MiB/s (641MB/s-641MB/s), io=5000MiB (5243MB), run=8175-8175msec
Run status group 1 (all jobs):
WRITE: bw=524MiB/s (550MB/s), 524MiB/s-524MiB/s (550MB/s-550MB/s), io=5000MiB (5243MB), run=9533-9533msec
Run status group 2 (all jobs):
READ: bw=492MiB/s (516MB/s), 492MiB/s-492MiB/s (516MB/s-516MB/s), io=5000MiB (5243MB), run=10163-10163msec
Run status group 3 (all jobs):
WRITE: bw=524MiB/s (549MB/s), 524MiB/s-524MiB/s (549MB/s-549MB/s), io=5000MiB (5243MB), run=9545-9545msec
Run status group 4 (all jobs):
READ: bw=309MiB/s (324MB/s), 309MiB/s-309MiB/s (324MB/s-324MB/s), io=5000MiB (5243MB), run=16205-16205msec
Run status group 5 (all jobs):
WRITE: bw=93.9MiB/s (98.5MB/s), 93.9MiB/s-93.9MiB/s (98.5MB/s-98.5MB/s), io=5000MiB (5243MB), run=53227-53227msec
Disk stats (read/write):
dm-0: ios=1295000/1289995, merge=0/0, ticks=251376/1561387, in_queue=1812763, util=98.81%, aggrios=1320000/1320047, aggrmerge=0/15, aggrticks=272111/1584269, aggrin_queue=915625, aggrutil=98.79%
sda: ios=1320000/1320047, merge=0/15, ticks=272111/1584269, in_queue=915625, util=98.79%
dd test NVME (8000 x 1M files)
8000+0 records in
8000+0 records out
8388608000 bytes (8.4 GB, 7.8 GiB) copied, 3.53487 s, 2.4 GB/s
real 0m3.537s
user 0m0.011s
sys 0m3.514s
dd test RAID SSD (8000 x 1M files)
8000+0 records in
8000+0 records out
8388608000 bytes (8.4 GB, 7.8 GiB) copied, 3.66226 s, 2.3 GB/s
real 0m3.664s
user 0m0.005s
sys 0m3.646s
dd test NVME (5GB file)
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 2.29852 s, 934 MB/s
real 0m2.857s
user 0m0.002s
sys 0m2.180s
dd test RAID SSD (5GB file)
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 5.11639 s, 420 MB/s
real 0m5.655s
user 0m0.001s
sys 0m2.168s