Introduce benchmark + performance regression checks

[azjezz]

Introduce benchmark + performance regression checks

[Ocramius]

as in 2 actions? at least for me (the one time i tried it) this resulted in less stable results

No no, same action and container: nothing guarantees that you will get two boxes running on a similar hardware otherwise.

[azjezz]

some functions will always be slower than core ( e.g: Regex\matches will always perform worst than preg_match, due to additional error handling ), the question is how much difference is considered acceptable.

[Ocramius]

Obviously, Psl will always be slower than core: it is required to keep the baseline under control though, as performance regressions often go unnoticed.

Also, knowing how much slower the library is allows for downstream consumers to take an informed decision as to when/where to use it.

[dantleech]

r.e. running in CI - I'm doing this with a bash script with assertions at a tolerance of 10% (but this is too optimistic, 15% might be better). It's still an experimental area (i also tried two separate actions to combine their results, but this doesn't work as the actions can run in completely different envs I guess).

[Ocramius]

Sounds feasible - even with a very high tolerance, it's more about looking at it before releasing, I guess?

Intra-worker performance should be relatively stable

[dantleech]

it's more about looking at it before releasing, I guess?

with a 15% tolerance it should catch obvious regressions, I guess if you want to check more subtly then before release/at PR time.

Intra-worker performance should be relatively stable

as in 2 actions? at least for me (the one time i tried it) this resulted in less stable results