spikebike / latencytest Goto Github PK

This benchmark p.c is a simple latency benchmark that uses INT64 arrays and
while (p) { p=a[p] } for pointer chasing.  For a user defined pagesize 
each cache line is visited randomly, exactly once before moving to the next page

To verify:

1) compile with -DPRINT:
	gcc -DPRINT -O3 -Wall -pedantic p.c -o p && ./p | egrep "^0x" | wc -l

2) verify that you see "8388608" which is 1GB / 128 byte cachelines

3) run:
	gcc -O3 -DPRINT -Wall -pedantic p.c && ./p | egrep "^0x" | sort | uniq > log
	wc -l log

4) verify that it says 8388608, this indicates that there were exactly that number of unique addresses visited (no cache friendly repeats).

5) run:
	go run hexdiff.go log  | uniq # or perl, but go takes 5 seconds and perl takes 6+ minutes

6) verify that the output is a single line of "Difference: 128", this shows
   that all 8388608 pointers were exactly 128 bytes apart.

7) in summary:
	* this code visits 8M unique cache lines
	* when printed each 8M address is unique
	* each address is in a single cacheline (no cacheline load can load 2 addresses)
	* the code is single threaded and each load is dependent on the previous load
	* this happens on a Ryzen 7900 at 10ns per load.

Example run:

	$ gcc -O3 -Wall -pedantic p.c -o p  && ./p 4096 128
	Allocated 1073741824 bytes or 134217728 INT64s succeeded.
	cacheLinesPerPage=32
	Initialized 8388608 cachelines
	Shuffle succeded
	visited 8388608 cachelines


This program allocates a 1GB array than for each page (512 INT64s) it builds an array with:

	for (i = 0; i < pageSize/sizeof(int64_t); i = i + perCacheLine)
   {
		b[i] = i;
		cnt++;
	}

So the beginning of each cacheline has the index of the array member of the next cache line, if printed, one cache line per 4KB page.  Compile with -DVERIFY

base=0000 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496 0 
base=0032 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496 0 

This would lead to sequential cache line access, which is easy to prefetch to fix that
the cachelines are shuffled:

base=0000 96 480 176 224 432 112 416 400 16 288 384 240 320 496 64 448 160 464 32 256 336 272 0 304 80 208 192 128 352 368 144 48 
base=0032 336 304 144 0 320 32 256 480 416 288 272 208 176 432 448 64 368 96 496 224 80 400 464 16 48 240 160 352 112 384 192 128 

Then I walk those cacheliens randomly with: while (p) { p = b[p]; } 

So each cacheline of a page is loaded randomly, exactly once, then it proceeds to the next page.

To see how the array is setup pre and post shuffle compile with gcc -DVERIFY