wainwrightmark/criterion-cycles-per-byte

`CyclesPerByte` measures clock cycles using the x86 or x86_64 `rdtsc` instruction.

criterion-cycles-per-byte's Issues

RDTSC does not measure cycles.

As the subject says, RDTSC does not measure cycles. "Invariant RDTSC" became the standard after the Pentium 4, which turned RDTSC into a high resolution timer which is invariant with respect to core frequency and power states. Despite now being a high resolution timer, an RDTSC tick is not a fixed unit of time (e.g. a nanosecond) across different computers.

If you want to perform benchmark timing in cycles, there are a number of issues. First, you have to determine the conversion rate from RDTSC ticks to clock cycles. This can sometimes be queried from the operating system (which usually measures it on boot using its own timing loop) or you can measure it with your own timing loop by relying on known 1-cycle latencies for basic ALU instructions. But regardless of whether you query it from the OS or measure it from scratch, you will run into the problem that core frequency is highly variable and therefore all the measurements need to be done at a fixed core frequency, usually by forcing the processor to stay at its base frequency for both the timing loop and the benchmark itself, which requires admin privileges and is highly OS dependent.

It's tricky to do everything correctly, but the operational requirements (requiring admin to force the CPU to base frequency, the inability to do this portably on Windows, etc) are probably the biggest obstacle for a turn-key solution like you're trying to provide here. But at a minimum I don't think it's good to advertise to your users that you're measuring cycles when you're just measuring RDTSC ticks.

If you're interested in what's involved in doing this properly on Linux, you can look at how uarch-bench does it.

Here's the launcher script: https://github.com/travisdowns/uarch-bench/blob/master/uarch-bench.sh
Here's the timing calibration loop: https://github.com/travisdowns/uarch-bench/blob/master/timers.cpp

My recommendation is that you don't try to measure clock cycles outside of carefully controlled settings like uarch-bench.