cpp-testing/ut-benchmark

Benchmarks - Unit Testing Frameworks

ut-benchmark's Issues

Difference in measurements between googletest and the rest of these libraries

Catch2 and doctest are "single header" but they are effectively a .cpp file and a .h file clumped together. I see that ut has also introduced the concept of BOOST_UT_IMPLEMENTATION but it currently provides much less functionality compared to Catch2 and doctest. In order to make the measurements completely fair one would need to either include the compilation of the googletest static library itself (which is obviously weird), or make static libraries for the implementations (test runners) of Catch2/doctest (and ut) and link against them (and when measuring time the compilation of the static library should be excluded - only the other .cpp file + linking should be considered). I'd argue that all 4 libraries should go through the process of linking a static library to make things equal.

doctest even provides the 2 parts (.cpp and a .h from which the final header is assembled) as separate files: https://github.com/onqtam/doctest/tree/master/doctest/parts

It would be cool if you could incorporate the use of FAST_CHECK_EQ(a, b) in a separate additional benchmark for doctest instead of the tradicional CHECK(a == b), but this is irrelevant to the current issue.