Bump to revm 3.0

[ziyadedher]

Bump to revm 3.0

[ziyadedher]

Thanks for the bump! Will do this today. I'm on the road and don't have my main computer, so I might just update all the results on my laptop. Either way, relative results should still be stable.

[ziyadedher]

Benchmarks are running on my laptop now, couldn't get pypy to compile properly on my Mac (at least not for Python 3.9), so just gonna do without for now. Will post a PR once the benchmark results come through.

[ziyadedher]

I'm seeing some surprising results on my MacBook (maybe ignore ethereumjs since my node-js runner is probably very optimized for work):

	evmone	revm	pyrevm	geth	py-evm.cpython	ethereumjs
sum	69.2ms	100.4ms	218ms	231.4ms	21.7272s	31.3376s
relative	1.000x	1.451x	3.150x	3.344x	313.977x	452.855x
erc20.approval-transfer	7.4ms	10.2ms	21.8ms	17.4ms	1.374s	1.8832s
erc20.mint	5.2ms	6.2ms	16.8ms	18.4ms	1.2822s	2.8656s
erc20.transfer	8.6ms	12ms	24.4ms	26ms	1.8158s	3.3676s
snailtracer	45ms	67ms	143ms	157ms	15.455s	21.592s
ten-thousand-hashes	3ms	5ms	12ms	12.6ms	1.8002s	1.6292s

I want to re-run these benchmarks when I'm back at my computer as a sanity check, but digging a bit deeper I see this:

	revm.old	revm.new
sum	94.2ms	109.3ms
relative	1.000x	1.160x
erc20.approval-transfer	10.6ms	11.4ms
erc20.mint	6.2ms	7.4ms
erc20.transfer	11.4ms	13ms
snailtracer	61.6ms	72.5ms
ten-thousand-hashes	4.4ms	5ms

Where revm.old is using pre-#212 revm-interpreter and revm.new is using latest at 1.1.1.

In the meantime, @rakita do you have any thoughts on a potential regression here? I think I'm actually seeing a slight regression from the old "release" of revm-interpreter to the latest one (1.1.1). Is the way the runner is setup still optimal?

@gakonst does this surprise you given how you feel Foundry or Reth perf has been?

[rakita]

This is expected, we removed one optimization (gas block) for more flexibility in lib usage, I got a little bit less performance degradation as a lot of jump predictions did they work, but i think this is run in docker so it is more effected.

I would say run it a few more times (2/3) just to confirm it is similar results and that would be it.

[gakonst]

hi @ziyadedher wonder if you have some capacity for this? revm has gotten some improvements lately and would love to get an update in the table

[gakonst]

Thx we'll investigate!

[ziyadedher]

Sounds good! Please keep me in the loop, I definitely wanna make sure that this benchmark suite is not misleading.