90 percentile calculation make comparisons across browser versions impossible.

[birkskyum]

90 percentile calculation make comparisons across browser versions impossible.

[krausest]

I added a note to the results page (https://krausest.github.io/js-framework-benchmark/index.html) and one regarding the weighted average on the results page (https://krausest.github.io/js-framework-benchmark/current.html)

[leeoniya]

i think there's a long-standing goal of ensuring vanillajs is 1.00 across the board (at least in perf, if not size and mem). imo a static basline is probably unlikely until that's closer to reality.

[krausest]

Hmm. Maybe I should have put a large disclaimer on the new results...

Actually I don't plan to adjust the weights with each browser version. I plan to do so only when they change significantly.
But comparison across browser versions can be dangerous since the benchmark drivers changes from time to time (as in chrome 118 when we measured to the commit event and changed the slow down factors).

The new calculation has two advantages:

• It reduces the influence of the two noisy benchmarks
• It allows less crazy, lower slow down factors.

[birkskyum]

That are good points. Sounds like a great idea then with some overall disclaimers about what can, and more importantly can't, be readily compared in this benchmark. I i.e. wasn't aware that cross browser-version comparisons generally were dangerous until now.