Low results for 3090?

[Randl]

Low results for 3090?

[tlkh]

I had noted in the readme that I have yet to update the batch size for 3090. Previously I had run both at the batch size, but after comments from other people I managed to improve the M1 Max performance (very marginally) by increasing the batch size, but I have yet to re-run the 3090 benchmarks at the larger batch size. I do expect it to improve.

[Randl]

Cool! I think having best-performing results for both platforms would give people understanding what can you achieve in terms of peak performance.

[tlkh]

Batch size is kept smaller, in line with the batch size running on M1 Max. This is for apple-to-apples comparison.

I did note on the results that the 3090 would perform better at higher batch sizes.

[Randl]

But batch size for 3090 is actually lower than for M1?
You could put numbers for best performance in both cases (i.e. run optimal bs for M1 on both and for 3090 on both) if you don't want to compare different bs.

[tlkh]

I have updated the readme. I can't get his level of performance even at BS=256. I suspect Ross Wightman is using PyTorch and it ends up being more performant for some reason. This is why DL benchmarking is hard... it is about the software as much as it is about the hardware.