Giter Site home page Giter Site logo

zhenxiongqian / test-runner-benchmarks Goto Github PK

View Code? Open in Web Editor NEW

This project forked from wangsongc/test-runner-benchmarks

0.0 0.0 0.0 12.45 MB

A repository to measure performance of Jest vs. Jasmine vs. Vitest

JavaScript 0.76% TypeScript 75.64% CSS 23.60%

test-runner-benchmarks's Introduction

test-runner-benchmarks

This repo is setup to test the performance of various test runners. Specially to:

Setup

  1. Install hyperfine via these instructions:
  2. Install dependencies:
yarn

Then you can run benchmarks via:

hyperfine --warmup 1 \
    'yarn workspace jasmine test' \
    'yarn workspace jest test' \
    'yarn workspace vitest test' \
    'yarn workspace vitest test --pool=vmThreads' \
    'yarn workspace vitest test --poolOptions.threads.isolate=false' \
    'yarn workspace bun test'

Note

These benchmarks are supported on MacOS and Linux. Windows is not supported at this time.

Suites

  • jasmine: This is our baseline, using Jasmine and happy-dom.
  • jest: Same test suite, but running using Jest.
  • vitest: Same test suite, but running using Vitest. NOTE: That benchmarks include vitest with the --poolOptions.threads.isolate setting both enabled and disabled due to this issue
  • bun: Same test suite, but running using Bun.

Results

Benchmarks are run via GitHub Actions. You can check the latest run results here.

Philosophy

  • Use hyperfine for consistent and reproducible benchmark collection
  • Discard the first run (via --warmup 1) to let various caches build up
  • Use minimal configurations (ie. stock configurations)
  • Tests should represent real-world scenarios (in this case, they are copies of real files used in real projects)
  • Tests should be updated for each test runner's best practices and APIs to give them the best chance possible to be optimized (eg. Jasmine uses APIs like createSpy() whereas Jest has jest.fn() and Vitest has vi.fn())

Other Suites

  • jest-dot: It was suggested that using Jest's dot reporter might result in faster performance. In the past this benchmark repo had a jest-dot suite to validate this but after many runs, it had nearly no impact on performance. The suite has since been removed.
  • jest-goloveychuk: GitHub user @goloveychuk suggested a solution which reduces Jest's memory usage. This solution was added and tested, but the performance impact was not any different.
  • fastest-jest-runner: Same as jest but using fastest-jest-runner. This solution was tested for several months but its performance in this benchmark was far worse than any of the others (including the baseline jest). It was removed in 2023-02-25.
  • jest-swc: Same as jest but using @swc/jest instead of ts-jest. It showed virtually no impact on performance. It was removed in 2023-05-22.

test-runner-benchmarks's People

Contributors

dependabot[bot] avatar evhaus avatar renovate-bot avatar renovate[bot] avatar wangsongc avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.