daanx / mimalloc-bench Goto Github PK

View Code? Open in Web Editor NEW

338.0 338.0 51.0 692 KB

Suite for benchmarking malloc implementations.

License: MIT License

CMake 0.59% C++ 7.51% C 85.20% SMT 0.05% Shell 6.02% Ruby 0.12% Python 0.52%

mimalloc-bench's People

Contributors

Stargazers

Watchers

Forkers

jserv dersteffi thro-infm18 mjansson bpowers mjp41 footka fujiz michalbiesek number1tsar struct ays7 beyonddream-productions pengking ipandayu mashoujiangplusai emarci15 suissewalter moloach rmn30 ursudanandrei tvaleev charles1614 devinus lhog hencounterhr maximus- homerl xiaoxiang781216 shitdarling ccase28 randoruf icodein 1c3t3a ttwjss bey2001 kaisun314 wanghanchi mashoujiang gfa99 gmh5225 cmovcc ckosan srinivasyadav18 siela1915 clayne tianyouli katshup nwtnni luzhixing12345

mimalloc-bench's Issues

alloc-test uses incorrect types

alloc-test uses size_t to store time intervals, and performs arithmetic with large multiplications on them without promoting to a larger type. This causes integer overflows and invalid results on 32-bit targets that aren't extremely fast.

Some other tests may have similar issues; I haven't checked yet.

Add smmalloc

https://github.com/SergeyMakeev/smmalloc

Don't use sudo when ran as root

When running in a throwaway docker environment, most people will not care about creating a dedicated user, as highlighted in #117, so we might want to drop the usage of sudo in this case. And print a big fat red warning about running stuff as root of course :)

http://www.microquill.com seems not accessible

is there any known alternative URL eventually ?

Result of print

What does each number that prints when you run the benchmark mean?

Such as
cfrac mi 08.31 3672 8.31 0.00 0 362

larsonN mi 1.118 3769812 279.90 3.95 0 950760

hd fails to build unless llvm-dev is installed first

--------------------------------------------
build hd: version 9d137ef37
--------------------------------------------

/tmp/mimalloc-bench/extern /tmp/mimalloc-bench
/tmp/mimalloc-bench/extern/Hoard already exists; no need to git clone
HEAD is now at 9d137ef Updated TLS model.
clang++ -std=c++14 -flto -O3 -DNDEBUG -ffast-math -fno-builtin-malloc -Wall -Wextra -Wshadow -Wconversion -Wuninitialized  -g -W -Wconversion -Wall -I/usr/include/nptl -fno-builtin-malloc -pipe -fPIC -DNDEBUG  -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -shared   source/libhoard.cpp source/unixtls.cpp -Bsymbolic -o libhoard.so -ldl -lpthread
/usr/bin/ld: /usr/lib/llvm-9/bin/../lib/LLVMgold.so: error loading plugin: /usr/lib/llvm-9/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [GNUmakefile:200: Linux-gcc-unknown] Error 1

Linking with gold requires having the llvm-dev package installed, at least on Debian. Please consider adding that to the script's list of packages to install.

Provide Dockerfile or ready-made container on dockerhub

Hi!

Trying to build / bench mimalloc-bench inside a pristine ubuntu:20.04 container and I'm seeing the ./build-bench-env.sh packages lean is not providing enough standard packages to kickstart from a bare distribution like containerized ubuntu.

During the course of installation I had to install ~10 extra packages and yet I didn't manage to run bench properly. Perhaps a Dockerfile or a pre-built container on dockerhub could ease the process for those not wanting to "pollute" the main OS with lots of packages and external software.

Please provide script for generating graphs

#2 (comment)

Provide a way to install ebizzy

The ebizzy benchmark is supported in bench.sh, but there is no way to install it.

Rename the repository

It would be great to rename this repository since:

it's not only about mimalloc anymore
it's confusing
it makes people suspicious of a bias toward mimalloc

Something like mbench, malloc-bench, …

Mention which allocators need system-wide install

For those not on Debian/Ubuntu it would be very helpful to know which allocators need to be installed system-wide and which are fetched by the build script.

Even better would be to split that script into a local build script that work on any Linux and isolate the parts that assumes Debian/Ubuntu. It would be easier for others to then add the scripts for Arch, Fedora, Gentoo, ...

Build fails on Ubuntu 20.04

Running ./build-bench-env.sh all fails in the section build sm. I am running Ubuntu 20.04 (in WSL2, though I don't think that's relevant), with g++ 10.3.0. I will provide additional information, if necessary.

--------------------------------------------
build sm: version 709663f
--------------------------------------------

~/allocator-test/mimalloc-bench/extern ~/allocator-test/mimalloc-bench
Cloning into 'SuperMalloc'...
remote: Enumerating objects: 2756, done.
remote: Total 2756 (delta 0), reused 0 (delta 0), pack-reused 2756
Receiving objects: 100% (2756/2756), 5.07 MiB | 1.61 MiB/s, done.
Resolving deltas: 100% (1857/1857), done.
Note: switching to '709663f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 709663f Merge pull request #48 from Willtor/master
set -e; rm -f ../release/env.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/env.cc -MG -MF ../release/env.d.$$; \
              sed 's,\(env\)\.o[ :]*,../release/\1.o ../release/env.d : ,g' < ../release/env.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/env.d; \
              rm -f ../release/env.d.$$
set -e; rm -f ../release/has_tsx.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/has_tsx.cc -MG -MF ../release/has_tsx.d.$$; \
              sed 's,\(has_tsx\)\.o[ :]*,../release/\1.o ../release/has_tsx.d : ,g' < ../release/has_tsx.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/has_tsx.d; \
              rm -f ../release/has_tsx.d.$$
set -e; rm -f ../release/futex_mutex.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/futex_mutex.cc -MG -MF ../release/futex_mutex.d.$$; \
              sed 's,\(futex_mutex\)\.o[ :]*,../release/\1.o ../release/futex_mutex.d : ,g' < ../release/futex_mutex.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/futex_mutex.d; \
              rm -f ../release/futex_mutex.d.$$
set -e; rm -f ../release/stats.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/stats.cc -MG -MF ../release/stats.d.$$; \
              sed 's,\(stats\)\.o[ :]*,../release/\1.o ../release/stats.d : ,g' < ../release/stats.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/stats.d; \
              rm -f ../release/stats.d.$$
set -e; rm -f ../release/footprint.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/footprint.cc -MG -MF ../release/footprint.d.$$; \
              sed 's,\(footprint\)\.o[ :]*,../release/\1.o ../release/footprint.d : ,g' < ../release/footprint.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/footprint.d; \
              rm -f ../release/footprint.d.$$
set -e; rm -f ../release/bassert.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/bassert.cc -MG -MF ../release/bassert.d.$$; \
              sed 's,\(bassert\)\.o[ :]*,../release/\1.o ../release/bassert.d : ,g' < ../release/bassert.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/bassert.d; \
              rm -f ../release/bassert.d.$$
set -e; rm -f ../release/cache.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/cache.cc -MG -MF ../release/cache.d.$$; \
              sed 's,\(cache\)\.o[ :]*,../release/\1.o ../release/cache.d : ,g' < ../release/cache.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/cache.d; \
              rm -f ../release/cache.d.$$
set -e; rm -f ../release/small_malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/small_malloc.cc -MG -MF ../release/small_malloc.d.$$; \
              sed 's,\(small_malloc\)\.o[ :]*,../release/\1.o ../release/small_malloc.d : ,g' < ../release/small_malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/small_malloc.d; \
              rm -f ../release/small_malloc.d.$$
set -e; rm -f ../release/large_malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/large_malloc.cc -MG -MF ../release/large_malloc.d.$$; \
              sed 's,\(large_malloc\)\.o[ :]*,../release/\1.o ../release/large_malloc.d : ,g' < ../release/large_malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/large_malloc.d; \
              rm -f ../release/large_malloc.d.$$
set -e; rm -f ../release/huge_malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/huge_malloc.cc -MG -MF ../release/huge_malloc.d.$$; \
              sed 's,\(huge_malloc\)\.o[ :]*,../release/\1.o ../release/huge_malloc.d : ,g' < ../release/huge_malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/huge_malloc.d; \
              rm -f ../release/huge_malloc.d.$$
set -e; rm -f ../release/rng.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/rng.cc -MG -MF ../release/rng.d.$$; \
              sed 's,\(rng\)\.o[ :]*,../release/\1.o ../release/rng.d : ,g' < ../release/rng.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/rng.d; \
              rm -f ../release/rng.d.$$
set -e; rm -f ../release/makechunk.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/makechunk.cc -MG -MF ../release/makechunk.d.$$; \
              sed 's,\(makechunk\)\.o[ :]*,../release/\1.o ../release/makechunk.d : ,g' < ../release/makechunk.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/makechunk.d; \
              rm -f ../release/makechunk.d.$$
set -e; rm -f ../release/malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/malloc.cc -MG -MF ../release/malloc.d.$$; \
              sed 's,\(malloc\)\.o[ :]*,../release/\1.o ../release/malloc.d : ,g' < ../release/malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/malloc.d; \
              rm -f ../release/malloc.d.$$
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -I../src  -c ../src/bassert.cc -o ../release/bassert.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11  ../src/objsizes.cc ../release/bassert.o -o ../release/objsizes
./../release/objsizes  ../release/generated_constants.cc >  ../release/generated_constants.h
set -e; rm -f ../release/cache.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/cache.cc -MG -MF ../release/cache.d.$$; \
              sed 's,\(cache\)\.o[ :]*,../release/\1.o ../release/cache.d : ,g' < ../release/cache.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/cache.d; \
              rm -f ../release/cache.d.$$
set -e; rm -f ../release/small_malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/small_malloc.cc -MG -MF ../release/small_malloc.d.$$; \
              sed 's,\(small_malloc\)\.o[ :]*,../release/\1.o ../release/small_malloc.d : ,g' < ../release/small_malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/small_malloc.d; \
              rm -f ../release/small_malloc.d.$$
set -e; rm -f ../release/large_malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/large_malloc.cc -MG -MF ../release/large_malloc.d.$$; \
              sed 's,\(large_malloc\)\.o[ :]*,../release/\1.o ../release/large_malloc.d : ,g' < ../release/large_malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/large_malloc.d; \
              rm -f ../release/large_malloc.d.$$
set -e; rm -f ../release/huge_malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/huge_malloc.cc -MG -MF ../release/huge_malloc.d.$$; \
              sed 's,\(huge_malloc\)\.o[ :]*,../release/\1.o ../release/huge_malloc.d : ,g' < ../release/huge_malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/huge_malloc.d; \
              rm -f ../release/huge_malloc.d.$$
set -e; rm -f ../release/makechunk.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/makechunk.cc -MG -MF ../release/makechunk.d.$$; \
              sed 's,\(makechunk\)\.o[ :]*,../release/\1.o ../release/makechunk.d : ,g' < ../release/makechunk.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/makechunk.d; \
              rm -f ../release/makechunk.d.$$
set -e; rm -f ../release/malloc.d; \
              g++ -MM -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  ../src/malloc.cc -MG -MF ../release/malloc.d.$$; \
              sed 's,\(malloc\)\.o[ :]*,../release/\1.o ../release/malloc.d : ,g' < ../release/malloc.d.$$ \
               | sed 's,generated_constants.h,../release/generated_constants.h,' > ../release/malloc.d; \
              rm -f ../release/malloc.d.$$
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/malloc.cc -o ../release/malloc.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/makechunk.cc -o ../release/makechunk.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/rng.cc -o ../release/rng.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/huge_malloc.cc -o ../release/huge_malloc.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/large_malloc.cc -o ../release/large_malloc.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/small_malloc.cc -o ../release/small_malloc.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/cache.cc -o ../release/cache.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/footprint.cc -o ../release/footprint.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/stats.cc -o ../release/stats.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/futex_mutex.cc -o ../release/futex_mutex.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src   -c -o ../release/generated_constants.o ../release/generated_constants.cc
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/has_tsx.cc -o ../release/has_tsx.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11     -I../release   -I../src  -c ../src/env.cc -o ../release/env.o
mkdir -p ../release/lib
gcc-ar cr ../release/lib/supermalloc.a  ../release/malloc.o  ../release/makechunk.o  ../release/rng.o  ../release/huge_malloc.o  ../release/large_malloc.o  ../release/small_malloc.o  ../release/cache.o  ../release/bassert.o  ../release/footprint.o  ../release/stats.o  ../release/futex_mutex.o  ../release/generated_constants.o  ../release/has_tsx.o  ../release/env.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11   ../release/malloc.o  ../release/makechunk.o  ../release/rng.o  ../release/huge_malloc.o  ../release/large_malloc.o  ../release/small_malloc.o  ../release/cache.o  ../release/bassert.o  ../release/footprint.o  ../release/stats.o  ../release/futex_mutex.o  ../release/generated_constants.o  ../release/has_tsx.o  ../release/env.o -shared -ldl -o ../release/lib/libsupermalloc.so
cc -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c11    -I../release   -I../src  -c ../tests/aligned_alloc.c -o ../release/aligned_alloc.o
g++ -W -Wall  -O3 -flto -ggdb -pthread -fPIC -mrtm  -std=c++11  ../release/aligned_alloc.o -ldl -L../release/lib -Wl,-rpath,../release/lib -ldl ../release/lib/libsupermalloc.so -o ../release/aligned_alloc
lto1: fatal error: bytecode stream in file ‘../release/aligned_alloc.o’ generated with GCC compiler older than 10.0
compilation terminated.
lto-wrapper: fatal error: g++ returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make: *** [../Makefile.include:122: ../release/aligned_alloc] Error 1
rm ../release/aligned_alloc.o

Add libpas comparison

A new malloc() has appeared: https://github.com/WebKit/WebKit/blob/main/Source/bmalloc/libpas/Documentation.md
Some discussion: https://news.ycombinator.com/item?id=31581727
There's some interesting things going on with the always_inline parts.

Add glibc's malloc tests

Please add both of glibc's malloc tests for all of the allocators:

glibc/benchtests/bench-malloc-simple.c
glibc/benchtests/bench-malloc-thread.c

Get them here: https://www.gnu.org/software/libc/sources.html:

git clone https://sourceware.org/git/glibc.git
cd glibc
git checkout master

Mirror: https://github.com/bminor/glibc/tree/master/benchtests

Document the benchmarks

It would be nice to have, for each benchmark:

This is also a nice opportunity to assess their relevance, and get rid of the unneeded ones.

Add a memory-hungry benchmark

I think that currently, we don't have any memory-hungy benchmark. It might be interesting to add one.

sc fails to build on current compilers

Trying to run build-bench-env.sh on a system with GCC 10.2.0 resulted in this error from sc:

--------------------------------------------
build sc: version master
--------------------------------------------

/tmp/mimalloc-bench/extern /tmp/mimalloc-bench
Cloning into 'scalloc'...
remote: Enumerating objects: 3308, done.
remote: Total 3308 (delta 0), reused 0 (delta 0), pack-reused 3308
Receiving objects: 100% (3308/3308), 1.03 MiB | 4.95 MiB/s, done.
Resolving deltas: 100% (2261/2261), done.
Already on 'master'
Your branch is up to date with 'origin/master'.
updating dependencies for scalloc

gyp... 

Cloning into 'build/gyp'...
remote: Total 18491 (delta 12809), reused 18491 (delta 12809)
Receiving objects: 100% (18491/18491), 8.60 MiB | 5.14 MiB/s, done.
Resolving deltas: 100% (12809/12809), done.

gyp... done
  CXX(target) out/Release/obj.target/scalloc/src/glue.o
In file included from src/platform/assert.h:8,
                 from src/arena.h:12,
                 from src/glue.h:12,
                 from src/glue.cc:5:
src/log.h: In function ‘void LogPrintf(int, const char*, const char*, ...)’:
src/log.h:51:10: error: ‘char* strncat(char*, const char*, size_t)’ specified bound 1 equals source length [-Werror=stringop-overflow=]
   51 |   strncat(buffer, ":", 1);
      |   ~~~~~~~^~~~~~~~~~~~~~~~
src/log.h:55:10: error: ‘char* strncat(char*, const char*, size_t)’ specified bound 1 equals source length [-Werror=stringop-overflow=]
   55 |   strncat(buffer, " ", 1);
      |   ~~~~~~~^~~~~~~~~~~~~~~~
src/log.h:75:10: error: ‘char* strncat(char*, const char*, size_t)’ specified bound 1 equals source length [-Werror=stringop-overflow=]
   75 |   strncat(buffer, "\n", 1);
      |   ~~~~~~~^~~~~~~~~~~~~~~~~
src/log.h:70:12: error: ‘char* strncpy(char*, const char*, size_t)’ output truncated before terminating nul copying 3 bytes from a string of the same length [-Werror=stringop-truncation]
   70 |     strncpy(rest_start + (rest - strlen(truncate_suffix)),
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   71 |             truncate_suffix,
      |             ~~~~~~~~~~~~~~~~
   72 |             strlen(truncate_suffix));
      |             ~~~~~~~~~~~~~~~~~~~~~~~~
src/log.h:49:10: error: ‘char* strncat(char*, const char*, size_t)’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
   49 |   strncat(buffer, file, strlen(file));
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/log.h:53:10: error: ‘char* strncat(char*, const char*, size_t)’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
   53 |   strncat(buffer, line_buffer, strlen(line_buffer));
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
make: *** [scalloc.target.mk:118: out/Release/obj.target/scalloc/src/glue.o] Error 1

Disable the tests when building the allocators

tcmalloc is building all of its tests and benchmarks
snmalloc is building a ton of tests

missing 'test' in bench.sh

there seems to be a 'test' missing in this line:

mimalloc-bench/bench.sh

Line 85 in 1d3414f

if "$use_packages" = "1"; then

Add bmalloc

https://opensource.apple.com/tarballs/bmalloc/, looks like it's OSX-only though.

Add musl's malloc-ng

The source can be found here, and shouldn't be hard to integrate. I'll likely do it soon™ if nobody does it before.

Musl's malloc-ng is musl default memory allocator, and is thus used in Alpine Linux by default, and on some versions of Void, Gentoo, OpenWRT, …

Cc: @richfelker

Missing debian packages

Script must install curl,automake and bsdextrautils on debian. Running from debian official docker images fails

Add snmalloc-secure

Apparently, snmalloc ≥ 6.0 has [some sweet hardening]( It does looks pretty nice: https://github.com/microsoft/snmalloc/tree/main/docs/security ).

Add a complex benchmark

Some allocators are doing tradeoffs to perform very well on long-running complex workloads, so we should have some benchmarks to measure this. Maybe something like firefox running on a couple of webpages, or Apache?

Or maybe some lightweight javascript engine, running some javascript things.
@TheShiftedBit suggested to compile nodejs or python to use the memory allocator, and run some workload on it.

build-bench-env.sh fails to install alt allocators (and no bench.sh?)

Ubuntu Disco with more sys info below. The errors below all come from running the build-bench-env.sh file. Note also there is no bench.sh file in this repo and build-bench-env.sh has nothing to create a bench.sh.

I'd recommend checking out the instructions for each of the installers. Also it's generally bad form to install things for users or make directories outside of your repository. If you need to do both of these I would recommend zipping all of this into a docker container.

Hoard Compilation Errors

HEAD is now at c856b43 Fixed release target.
git clone https://github.com/emeryberger/Heap-Layers
Cloning into 'Heap-Layers'...
remote: Enumerating objects: 1710, done.
remote: Total 1710 (delta 0), reused 0 (delta 0), pack-reused 1710
Receiving objects: 100% (1710/1710), 402.50 KiB | 4.38 MiB/s, done.
Resolving deltas: 100% (1121/1121), done.
clang++ -std=c++14 -O3 -DNDEBUG -ffast-math -fno-builtin-malloc -Wall -Wextra -Wshadow -Wconversion -Wuninitialized -g -W -Wconversion -Wall -I/usr/include/nptl -fno-builtin-malloc -pipe -fPIC -DNDEBUG  -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -shared   source/libhoard.cpp source/unixtls.cpp Heap-Layers/wrappers/gnuwrapper.cpp -Bsymbolic -o libhoard.so -ldl -lpthread
clang++ -std=c++14 -O3 -DNDEBUG -ffast-math -fno-builtin-malloc -Wall -Wextra -Wshadow -Wconversion -Wuninitialized -g -W -Wconversion -Wall -I/usr/include/nptl -fno-builtin-malloc -pipe -fPIC -DNDEBUG  -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -shared   source/libhoard.cpp source/unixtls.cpp Heap-Layers/wrappers/gnuwrapper.cpp -Bsymbolic -o libhoard.so -ldl -lpthread
cp libhoard.so /usr/lib
cp: cannot stat '/usr/lib/libhoard.so': Too many levels of symbolic links
make: *** [GNUmakefile:187: Linux-gcc-x86_64-install] Error 1
~/open_source/open_projects/misc/mimalloc-bench

SuperMalloc

/usr/include/stdlib.h:583:14: note: in a call to allocation function ‘aligned_alloc’ declared here
 extern void *aligned_alloc (size_t __alignment, size_t __size)
              ^
lto1: all warnings being treated as errors
lto-wrapper: fatal error: g++ returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make: *** [../Makefile.include:122: ../release/aligned_alloc] Error 1
rm ../release/aligned_alloc.o
~/open_source/open_projects/misc/mimalloc-bench

rpmalloc

[10/48] CC test/thread.c
FAILED: build/ninja/linux/release/b'x86_64'/test-57ec084/thread-35aa063.o 
clang -MMD -MT 'build/ninja/linux/release/b'\''x86_64'\''/test-57ec084/thread-35aa063.o' -MF 'build/ninja/linux/release/b'\''x86_64'\''/test-57ec084/thread-35aa063.o'.d -I. -Irpmalloc -Itest -DRPMALLOC_COMPILE=1 -funit-at-a-time -fstrict-aliasing -fno-math-errno -ffinite-math-only -funsafe-math-optimizations -fno-trapping-math -ffast-math -D_GNU_SOURCE=1 -W -Werror -pedantic -Wall -Weverything -Wno-padded -Wno-documentation-unknown-command -std=c11  -DBUILD_RELEASE=1 -O3 -g -funroll-loops -DENABLE_ASSERTS=1 -DENABLE_STATISTICS=1 -c test/thread.c -o 'build/ninja/linux/release/b'\''x86_64'\''/test-57ec084/thread-35aa063.o'
test/thread.c:95:2: error: implicit use of sequentially-consistent atomic may incur stronger memory barriers than necessary [-Werror,-Watomic-implicit-seq-cst]
        __sync_synchronize();
        ^~~~~~~~~~~~~~~~~~
test/thread.c:104:2: error: implicit use of sequentially-consistent atomic may incur stronger memory barriers than necessary [-Werror,-Watomic-implicit-seq-cst]
        __sync_synchronize();
        ^~~~~~~~~~~~~~~~~~
2 errors generated.

Sys Info

uname -a
# Linux [comp] 5.0.0-15-generic #16-Ubuntu SMP 
# Mon May 6 17:41:33 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

g++ -v
# Using built-in specs.
# COLLECT_GCC=g++-9
# COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
# OFFLOAD_TARGET_NAMES=nvptx-none
# OFFLOAD_TARGET_DEFAULT=1
# Target: x86_64-linux-gnu
# Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.1.0-2ubuntu2~19.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
# Thread model: posix
# gcc version 9.1.0 (Ubuntu 9.1.0-2ubuntu2~19.04) 

cmake --version
# cmake version 3.13.4

bench.sh help mentions `math-lib` but doesn't recognize it

The help for bench.sh mentions an argument math-lib, but doesn't recognize that argument if passed; it seems to be expecting mathlib, with no dash.

Use full names for allocators

Currently, it's a bit confusing what the two-letter codes are referring to: what are the differences between mi, xdmi, xmi, dmi, smi, xsmi, … also, what is sc compared to scudo? What is tlsf ? And so on.

I would have been happy to do it myself, but for some allocators, I actually have no clue what they're referring to :D

Run the CI in musl

It would be nice to run the CI in an alpine linux container, to see how allocators are behaving on musl

Decide what we want to do with scalloc

scalloc hasn't been updated since 2016, and uses the gyp build system which is broken with python3.10.

What we keep it, even if it's not building in the CI?
If so, shall we spend time trying to fix its build system working with upstream?

need new allocators

Update allocator list with

https://www.qt.io/blog/a-fast-and-thread-safe-pool-allocator-for-qt-part-1

According of coder fastest allocator.

And tcmalloc latest available from github source

And if scalloc fails with GCc 10.2 install automaticaly GCC 8 and build specific with gcc-8 g++-8 with CC=GCC-8 CXX=g++-8

It.should.build

Add buddy_alloc

https://github.com/spaskalev/buddy_alloc

Add ruby allocator benchmark

Can you add a test for this?

https://github.com/SamSaffron/allocator_bench

Curious how it stacks up

redis build fails with multiple definition of SDS_NOINIT

/usr/bin/ld: server.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: sds.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: ziplist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: networking.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: util.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: object.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: db.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: replication.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: rdb.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: t_string.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: t_list.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: t_set.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: t_zset.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: t_hash.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: config.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: aof.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: pubsub.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: multi.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: debug.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: sort.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: syncio.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: cluster.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: crc16.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: slowlog.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: scripting.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: bio.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: rio.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: bitops.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: sentinel.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: notify.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: blocked.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: hyperloglog.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: latency.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: sparkline.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: redis-check-rdb.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: redis-check-aof.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: geo.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: lazyfree.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: module.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: evict.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: expire.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: childinfo.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: defrag.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: t_stream.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: lolwut.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here
/usr/bin/ld: lolwut5.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: multiple definition of `SDS_NOINIT'; quicklist.o:/tmp/mimalloc-bench/extern/redis-5.0.3/src/sds.h:37: first defined here

This is a bug in that version of redis when building with gcc 10:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=957751

Hardcoded path for LD_PRELOAD

I can build part of mimalloc-bench on Linux/Aarch64 target. However, I found hardcoded path for LD_PRELOAD:

out/bench$ ../../bench.sh rp cfrac
...
---- cfrac

run cfrac rp: LD_PRELOAD=/tmp/mimalloc-bench/extern/rpmalloc/bin/linux/release/x86-64/librpmalloc.so ./cfrac 175451865205073170563711388363274837927895
ERROR: ld.so: object '/tmp/mimalloc-bench/extern/rpmalloc/bin/linux/release/x86-64/librpmalloc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.

It should not be x86-64 since I am testing aarch64.

Add rocksdb to the benchmarks

rocksdb has a neat benchmarking tool, and it written with hardware-efficiency in mind, so it should highlight some allocators shortcoming.

Moreover, it's buildable with a makefile, so integrating it shouldn't be hard.

Race condition in the rptest benchmark

Running isoalloc with the rptest benchmark resulted in a reproducible segfault, hinting at a race condition in rptest, as found by @struct:

root@67dbecee929b:/# clang -o rptest *.c -lpthread -I. -ggdb -fsanitize=thread
root@67dbecee929b:/# ./rptest 2 0 1 2 500 1000 100 8 16000
crt          2 threads random linear size [8,16000] 500 loops 1000 allocs 100 ops: ==================
WARNING: ThreadSanitizer: data race (pid=17173)
  Write of size 4 at 0x000000fec030 by main thread:
    #0 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:777:19 (rptest+0x4b51b4)
    #1 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Previous read of size 4 at 0x000000fec030 by thread T1:
    #0 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:355:10 (rptest+0x4b5a34)
    #1 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  As if synchronized via sleep:
    #0 nanosleep <null> (rptest+0x4215d8)
    #1 thread_sleep /code/jmmb/mimalloc-bench/bench/rptest/thread.c:78:2 (rptest+0x4b80f4)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:775:3 (rptest+0x4b51a0)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Location is global 'benchmark_start' of size 4 at 0x000000fec030 (rptest+0x000000fec030)

  Thread T1 (tid=17175, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

SUMMARY: ThreadSanitizer: data race /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:777:19 in benchmark_run
==================
==================
WARNING: ThreadSanitizer: data race (pid=17173)
  Read of size 8 at 0xfffff5503f68 by thread T1:
    #0 atomic_load_ptr /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:23:14 (rptest+0x4b7e10)
    #1 put_cross_thread_memory /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:312:10 (rptest+0x4b7d14)
    #2 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:469:5 (rptest+0x4b6d1c)
    #3 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  Previous write of size 8 at 0xfffff5503f68 by main thread:
    #0 atomic_store_ptr /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:28:17 (rptest+0x4b79d4)
    #1 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:767:4 (rptest+0x4b4ff4)
    #2 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  As if synchronized via sleep:
    #0 nanosleep <null> (rptest+0x4215d8)
    #1 thread_sleep /code/jmmb/mimalloc-bench/bench/rptest/thread.c:78:2 (rptest+0x4b80f4)
    #2 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:356:3 (rptest+0x4b5a4c)
    #3 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  Location is heap block of size 272 at 0xfffff5503e80 allocated by main thread:
    #0 calloc <null> (rptest+0x4228b4)
    #1 benchmark_malloc /code/jmmb/mimalloc-bench/bench/rptest/./benchmark.h:41:13 (rptest+0x4b3c5c)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:723:8 (rptest+0x4b49a8)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Thread T1 (tid=17175, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

SUMMARY: ThreadSanitizer: data race /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:23:14 in atomic_load_ptr
==================
==================
WARNING: ThreadSanitizer: data race (pid=17173)
  Atomic write of size 8 at 0xfffff5503ee0 by thread T2:
    #0 __tsan_atomic64_compare_exchange_val <null> (rptest+0x4790d4)
    #1 atomic_cas_ptr /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:77:9 (rptest+0x4b7e8c)
    #2 put_cross_thread_memory /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:315:12 (rptest+0x4b7d7c)
    #3 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:469:5 (rptest+0x4b6d1c)
    #4 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  Previous read of size 8 at 0xfffff5503ee0 by thread T1:
    #0 atomic_load_ptr /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:23:14 (rptest+0x4b7e10)
    #1 get_cross_thread_memory /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:323:10 (rptest+0x4b7c50)
    #2 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:368:14 (rptest+0x4b5b88)
    #3 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  Location is heap block of size 272 at 0xfffff5503e80 allocated by main thread:
    #0 calloc <null> (rptest+0x4228b4)
    #1 benchmark_malloc /code/jmmb/mimalloc-bench/bench/rptest/./benchmark.h:41:13 (rptest+0x4b3c5c)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:723:8 (rptest+0x4b49a8)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Thread T2 (tid=17176, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Thread T1 (tid=17175, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

SUMMARY: ThreadSanitizer: data race (/code/jmmb/mimalloc-bench/bench/rptest/rptest+0x4790d4) in __tsan_atomic64_compare_exchange_val
==================
==================
WARNING: ThreadSanitizer: data race (pid=17173)
  Read of size 4 at 0x000000fec034 by thread T2:
    #0 atomic_load32 /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:38:14 (rptest+0x4b7a20)
    #1 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:473:8 (rptest+0x4b6d3c)
    #2 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  Previous atomic write of size 4 at 0x000000fec034 by thread T1:
    #0 __tsan_atomic32_fetch_add <null> (rptest+0x4735a0)
    #1 atomic_incr32 /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:52:9 (rptest+0x4b7dc4)
    #2 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:511:3 (rptest+0x4b6d9c)
    #3 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  Location is global 'benchmark_threads_sync' of size 4 at 0x000000fec034 (rptest+0x000000fec034)

  Thread T2 (tid=17176, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Thread T1 (tid=17175, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

SUMMARY: ThreadSanitizer: data race /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:38:14 in atomic_load32
==================
==================
WARNING: ThreadSanitizer: data race (pid=17173)
  Read of size 4 at 0xfffff5503ee8 by main thread:
    #0 atomic_load32 /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:38:14 (rptest+0x4b7a20)
    #1 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:789:38 (rptest+0x4b5258)
    #2 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Previous atomic write of size 4 at 0xfffff5503ee8 by thread T2:
    #0 __tsan_atomic32_fetch_add <null> (rptest+0x4735a0)
    #1 atomic_add32 /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:62:9 (rptest+0x4b7c04)
    #2 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:398:5 (rptest+0x4b601c)
    #3 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  As if synchronized via sleep:
    #0 nanosleep <null> (rptest+0x4215d8)
    #1 thread_sleep /code/jmmb/mimalloc-bench/bench/rptest/thread.c:78:2 (rptest+0x4b80f4)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:784:3 (rptest+0x4b51fc)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Location is heap block of size 272 at 0xfffff5503e80 allocated by main thread:
    #0 calloc <null> (rptest+0x4228b4)
    #1 benchmark_malloc /code/jmmb/mimalloc-bench/bench/rptest/./benchmark.h:41:13 (rptest+0x4b3c5c)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:723:8 (rptest+0x4b49a8)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Thread T2 (tid=17176, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

SUMMARY: ThreadSanitizer: data race /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:38:14 in atomic_load32
==================
==================
WARNING: ThreadSanitizer: data race (pid=17173)
  Read of size 4 at 0xfffff5503f70 by main thread:
    #0 atomic_load32 /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:38:14 (rptest+0x4b7a20)
    #1 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:789:38 (rptest+0x4b5258)
    #2 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Previous atomic write of size 4 at 0xfffff5503f70 by thread T2:
    #0 __tsan_atomic32_fetch_add <null> (rptest+0x4735a0)
    #1 atomic_add32 /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:62:9 (rptest+0x4b7c04)
    #2 benchmark_worker /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:460:32 (rptest+0x4b6b54)
    #3 thread_entry /code/jmmb/mimalloc-bench/bench/rptest/thread.c:29:2 (rptest+0x4b7fc4)

  As if synchronized via sleep:
    #0 nanosleep <null> (rptest+0x4215d8)
    #1 thread_sleep /code/jmmb/mimalloc-bench/bench/rptest/thread.c:78:2 (rptest+0x4b80f4)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:784:3 (rptest+0x4b51fc)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Location is heap block of size 272 at 0xfffff5503e80 allocated by main thread:
    #0 calloc <null> (rptest+0x4228b4)
    #1 benchmark_malloc /code/jmmb/mimalloc-bench/bench/rptest/./benchmark.h:41:13 (rptest+0x4b3c5c)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:723:8 (rptest+0x4b49a8)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

  Thread T2 (tid=17176, running) created by main thread at:
    #0 pthread_create <null> (rptest+0x4241b4)
    #1 thread_run /code/jmmb/mimalloc-bench/bench/rptest/thread.c:41:12 (rptest+0x4b7f14)
    #2 benchmark_run /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:772:29 (rptest+0x4b5128)
    #3 main /code/jmmb/mimalloc-bench/bench/rptest/rptest.c:901:9 (rptest+0x4b7ba8)

SUMMARY: ThreadSanitizer: data race /code/jmmb/mimalloc-bench/bench/rptest/./atomic.h:38:14 in atomic_load32
==================
........756290 memory ops/CPU second (59MiB peak, 3MiB -> 58MiB bytes sample, 1375% overhead)
ThreadSanitizer: reported 6 warnings

Running binaries with tbb fails to find libtbbmalloc.so.2

run cfrac tbb: LD_PRELOAD=/tmp/mimalloc-bench/extern/tbb/build/linux_intel64_gcc_cc10.2.0_libc2.31_kernel5.7.0_release/libtbbmalloc_proxy.so.2 ./cfrac 17545186520507317056371138836327483792789528
./cfrac: error while loading shared libraries: libtbbmalloc.so.2: cannot open shared object file: No such file or directory

Migrate to the maintained version of TBB

https://github.com/wjakob/tbb → https://github.com/oneapi-src/oneTBB

Add the new TCMalloc

Unfortunately, there are two allocators both called tcmalloc. You are using the one from the gperftools repo, but a newer fork from upstream which uses restartable sequences is available at https://github.com/google/tcmalloc. It would be nice to be able to see results for both of them along with mimalloc.

Make of of emeryberger's allocators collection

emeryberger has some interesting allocators that we should include in mimalloc-bench.

Improve the speed of the benchmarks

Currently the CI takes around 90min to run: this is a bit excessive. Here is what we could do to reduce a bit this number:

Change the parameters for the redis benchmark: SYSMALLOC=1 /__w/mimalloc-bench/mimalloc-bench/extern/redis-6.2.6/src/redis-benchmark -r 1000000 -n 1000000 -q -P 16 lpush a 1 2 3 4 5 lrange a 1 5 takes around a full minute, given that we have ~16 allocators, it adds up rapidly.
- Done in 410f665
Lean takes around 1-3 minutes per allocator, and is unmaintained. Shall we try to move to lean4 ?

Please consider adding scudo

Please consider adding the scudo allocator. It's available in the compiler-rt/lib/scudo/standalone directory of https://github.com/llvm/llvm-project/ , and from that directory, you can build a shared library using clang++ -flto -fuse-ld=lld -fPIC -std=c++14 -fno-exceptions -fno-rtti -fvisibility=internal -msse4.2 -O3 -I include -shared -o libscudo.so *.cpp -pthread

Add freeguard

https://github.com/UTSASRG/FreeGuard

Provide a way to run spec and document it

Spec 2017 is supported in bench.sh, but there is no easy way to install it, nor documentation about what it does.

Do we want to make the benchmark portable, shell-wise?

Currently, the ./build-bench-env.sh and ./bench.sh scripts do contain a lot of bash-specific constructs.
Do we want to make it work on other shells like sh? Eg. Alpine Linux doesn't use bash by default, but busybox' ash.
Do we even care?

Provide a way to bench a commit range of a specific allocator

It would be great to be able to tell mimalloc-bench "I want to bench this allocator, between this commit and this commit.", to find performances regressions. One might even go even further, and specify a difference threshold, to be able to use it in git bisect.

new[] and delete in Larson.cpp

@daanx the Larson benchmarking does not correctly match new[] to delete[] calls. It allocates as an array, but deletes as an object: For instance allocation here

mimalloc-bench/bench/larson/larson.cpp

Line 379 in bcddf9f

blkp[cblks] = new char[blk_size] ;

and deallocation here

mimalloc-bench/bench/larson/larson.cpp

Line 391 in bcddf9f

delete blkp[victim] ;

This means that it is supplying size parameters to delete assuming it is deallocating a char rather than a char array. For snmalloc this was causing a lot of memory wastage as it was assuming the size was correct.

I am happy to PR a fix, but wanted to check if you would take it.

Use native compilation for allocators

As discussed today, there are pros and cons about using native compilation for allocators (like in #54), but mostly pros, boiling down to allocators taking advantage of modern CPU features shouldn't be penalized by using the lowest common denominator.

Since not every build system/scripts used have a --enable-native-and-all-perf-stuff-go-go-go option, one way to make use of --march-native would be to simply pass it via the CFLAGS and CXXFLAGS environment variables.

Results here are very hard to understand

Shades of colors make it extremely hard to tell what is going on.

Wondering, can you

Raw results in text with percentage?
Avoid using shades of colors?

documentation: Chrome does not use tcmalloc

Chrome doesn’t use tcmalloc and hasn’t for a long time. They’re using PartitionAlloc on all platforms they’re targeting and have already removed the relevant code from //base/allocator.