playwright-community/jest-playwright

Running tests using Jest & Playwright

Playwright Community

A central home for tutorials, tooling, and showcases of the Playwright ecosystem.

This is the source of playwright.tech. A Next.js based application hosted on Vercel.

jest-playwright's Issues

Feature: Add jest-playwright as an ESLint environment

https://eslint.org/docs/user-guide/configuring#specifying-environments has a way to configure environments. It would be cool if jest-playwright was part of that, but I am not sure how feasible that is. This would avoid having to specify the globals that come with jest-playwright.

Maybe not worth it and then you can close this, but I just thought I'd bring up the idea even how minor it might be.

[Question] Is the dev-server command expected to run for each test suite / spec file?

I have the following jest-playwright.config.js:

  server: {
    command: 'npm run test-server',
    port: 4000,
    launchTimeout: 30000,
  },

Where test-server is just:

    "server": "bin/rails server",
    "test-server": "RAILS_ENV=test npm run server -- -p 4000",

When I execute my tests individually, say jest -- test/e2e/mytest.spec.js, It works fine (runs the server, runs the test, closes the server via SIGTERM). However, when I run all tests via jest, there is a race condition happening, where before the previous server is terminated, another server is attempting to start, however something somewhere complains:

Another process is listening on 4000. Should I kill it for you? On linux, this may require you to enter your password. (Y/n)

Which effectively hangs my tests. How do I make it that only one server is run for all tests?

[Question] Should beforeAll still default to 5000ms?

I have a spec file with a beforeAll function that runs more than 5000ms:

describe("Expired Password", () => {
  beforeAll(async () => {
    await fs.promises.mkdir(screenshotPath, { recursive: true })

    await page.goto(baseUrl)

    await Promise.all([
      page.waitForNavigation(),
      new Promise(async (resolve, reject) => {
        try {
          // TODO: Determine if mobile
          if (true) {
            await page.click('.burger')
            await page.click('"Login"')
          } else {
            await page.click('"Login"')
          }

          resolve()
        } catch (e) {
          reject(e)
        }
      })
    ])

    await page.fill("#user_username", "myusername")
    await page.fill("#user_password", "mypassword")
    await Promise.all([
      page.waitForNavigation(),
      page.click('input[value="Log in"]'),
    ])

    if (page.coverage) {
      await Promise.all([
        page.coverage.startJSCoverage(),
        page.coverage.startCSSCoverage(),
        page.goto(baseUrl + "/change-password")
      ])
      const [jsCoverage, cssCoverage] = await Promise.all([
        page.coverage.stopJSCoverage(),
        page.coverage.stopCSSCoverage(),
      ])
    } else {
      await page.goto(baseUrl + "/change-password")
    }
  })
})

which gives me an error:

Timeout - Async callback was not invoked within the 5000ms timeout specified by
jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

If I add the timeout argument, say 30000ms beforeAll(async () => {}, 30000) or move it to the test function:

describe("Expired Password", () => {
  beforeAll(async () => {
    await fs.promises.mkdir(screenshotPath, { recursive: true })

    await page.goto(baseUrl)
  })

  it("1. should load the expired password page", async () => {
    try {
      await Promise.all([
        page.waitForNavigation(),
        new Promise(async (resolve, reject) => {
          try {
            // TODO: Determine if mobile
            if (true) {
              await page.click('.burger')
              await page.click('"Login"')
            } else {
              await page.click('"Login"')
            }

            resolve()
          } catch (e) {
            reject(e)
          }
        })
      ])

      await page.fill("#user_username", "myusername")
      await page.fill("#user_password", "mypassword")
      await Promise.all([
        page.waitForNavigation(),
        page.click('input[value="Log in"]'),
      ])

      if (page.coverage) {
        await Promise.all([
          page.coverage.startJSCoverage(),
          page.coverage.startCSSCoverage(),
          page.goto(baseUrl + "/change-password")
        ])
        const [jsCoverage, cssCoverage] = await Promise.all([
          page.coverage.stopJSCoverage(),
          page.coverage.stopCSSCoverage(),
        ])
      } else {
        await page.goto(baseUrl + "/change-password")
      }

      const newPassword = await page.$("#new_password")
      const confirmPassword = await page.$("#confirm_password")

      await page.screenshot({
        path: path.join(screenshotPath, `1.png`),
        fullPage: true,
      })

      expect(await newPassword.visibleRatio()).toBe(1)
      expect(await confirmPassword.visibleRatio()).toBe(1)
    } catch (e) {
      return Promise.reject(e)
    }
  })
})

Then it works fine. Is this the expected behavior? Or should it wait in a similar manner to the it() and test() functions?

jestPlaywright.debug() support in Jest-circus

I'm going to be honest to say I've not investigated this yet, but seeing as jest-playwright's grandfather is jest-puppeteer, I expect this is not supported.

For reference for this issue, see argos-ci/jest-puppeteer#255

I wonder if there any plans to support jest-circus. While jest-circus works kinda fine out of the box, I noticed that jestPlaywright.debug() does not really work as expected.

Add a coverage visualisation tool

Like:

• https://codecov.io
• https://coveralls.io

what do you think? I prefer codecov. Very trivial to add, I can take over that issue if you want.

Documentation improvements

Need some extra information about different ways of configuration
#4 (comment)
#4 (comment)

Migrate to typescript

Thinking about rewriting current codebase with TS, cause there is no so much code right now, it mustn't take much time with it

Add 'server' option to https://github.com/mmarkelov/jest-playwright#configuration

https://github.com/mmarkelov/jest-playwright#start-a-server was recently added and it explains how to add a server, but maybe also the actual opion should be shown in https://github.com/mmarkelov/jest-playwright#configuration?

Unable to register new selector engines

const {
  selectorEngine,
} = require('query-selector-shadow-dom/plugins/playwright');
const playwright = require('playwright');

describe('Test', () => {
  it('tests', async () => {
    await page.goto('http://google.com/');
    console.log(playwright.selectors);
    await playwright.selectors.register(selectorEngine, { name: 'shadow' });
    console.log(playwright.selectors);
    await page.waitForSelector('shadow=body');
  });
});

I am using https://github.com/Georgegriff/query-selector-shadow-dom, as per above, to register a new selector engine. However, when invoking the waitForSelector it returns Evaluation failed: Error: Unknown engine shadow while parsing selector shadow=body.

I think this might be due to that I import playwright in the suite, when jest-playwright itself already imported it somewhere else. So how would I register a selector engine with jest-playwright?

cc @Georgegriff in case he has some input

Example not working

The example on the main page does not work:

describe('Google', () => {
  beforeAll(async () => {
    await page.goto('https://whatismybrowser.com/')
  })

  it('should display "google" text on page', async () => {
    const browser = await page.$eval('.string-major a', el => el.text)
    expect(browser).toContain('Chrome')
  })
})

It gives the result:

  Google
    \u2715 should display "google" text on page (6ms)

  \u25cf Google \u203a should display "google" text on page

    Error: failed to find element matching selector ".string-major a"

       9 | 
      10 |   it('should display "google" text on page', async () => {
    > 11 |     const browser = await page.$eval('.string-major a', el => el.text);
         |                     ^
      12 |     expect(browser).toContain('Chrome');
      13 |   });
      14 | });

feat: Whitelist of files for actual Playwright tests

Use case:

We have a bunch of tests files in our project, but only some of them require a browser instance. Currently we open for every test with Jest a dedicated browser.

Ideas:

• a) introduce a whitelist, so that only for a specific glob pattern (can also be full file path) will the browser in the end being created

- b) use a single browser instance for all the tests and only create a new context for each file (docs in #72)

Regression: Can't use single browser anymore

Please see this comment on commit 0a36fa6.

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet.
We recommend using:

• CircleCI
• Travis CI
• Buildkite
• CodeShip
• Azure Pipelines
• TeamCity
• Buddy
• AppVeyor
  But Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet.
We recommend using:

• CircleCI
• Travis CI
• Buildkite
• CodeShip
• Azure Pipelines
• TeamCity
• Buddy
• AppVeyor
  But Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.

Feature request: Opt out of failure on pageerror

Hey team. Got a feature request here. It looks like a pageerror will always fail the current test (based on this LOC).

jest-playwright/src/PlaywrightEnvironment.ts

Line 16 in 1dcafbd

process.emit('uncaughtException', error)

Could we allow this to be turned off? In my use case, I have a third-party script sometimes throws errors that I want to ignore, but I want to know if my first-party scripts throw an error.

Thanks!

Feature: Prepare for jest-circus

Next version of Jest will default to using jest-circus as their test runner. Already now many projects change the default runner to instead use jest-circus due to it's superioirity, see https://github.com/facebook/jest/tree/master/packages/jest-circus

jest-circus supports new events fired that can be listened to in test environments via the handleTestEvent function, see https://jestjs.io/docs/en/configuration#testenvironment-string

As part of this, another PR is ongoing to make them async, see jestjs/jest#9397

With all this in mind, I have two things that would be cool if jest-playwright is a bit proactive to support:

1. Support for jest-circus. Maybe this already works?
2. Support for async handleTestEvent. This could be done easily by a user of jest-playwright by extending the environment, as such https://github.com/smooth-code/jest-puppeteer#extend-puppeteerenvironment. I am thus wondering if this should be the recommended way also in jest-playright (in the readme) or via some other means to catch these events.

note: We want to use this ourselves to take screenshots upon failures.

jest-dev-server support

Contemplating to move to Playwright, but due to dependency to jest-puppeteer I need to make sure this plugin will work just as well. I could not find in the documentation if https://github.com/smooth-code/jest-puppeteer#start-a-server is supported. Could this be added and if it already is, the README updated with information about it.

[Question] Multiple page.goto in a spec/test file

I have a test where I page.goto and page.close for each test. Something like this:

describe('Google', () => {
    it('should display "google" text on page', async () => {
        await page.goto('https://whatismybrowser.com/')
        const browser = await page.$eval('.string-major a', el => el.text)
        expect(browser).toContain('Chrome')
        await page.close()
    })

    it('this is the second test', async () => {
        await page.goto('http://whatsmyuseragent.org/')
        const browser = await page.$eval('.intro-text', el => el.text)
        expect(browser).toContain('Chrome')
        await page.close()
    })
})

How can I achieve this, as the test crashes because the browser gets closed? Or should I just separate the tests instead, one file per page.goto, then just wrap each page.goto in a beforeAll function?

playwright-firefox and friends always installed

playwright-firefox, playwright-chromium and playwright-webkit always get installed

$ node --version
v12.14.1

$ npm --version
6.13.4

$ npm ls playwright-firefox
[email protected]
└─┬ [email protected]
  └── [email protected]

$ npm ls playwright-webkit
[email protected]
└─┬ [email protected]
  └── [email protected] 

This is because they are declared as optionalDependencies: https://github.com/mmarkelov/jest-playwright/blob/v0.0.10/package.json#L48-L54

This generates inside package-lock.json:

"jest-playwright-preset": {
  "version": "0.0.10",
  "resolved": "https://registry.npmjs.org/jest-playwright-preset/-/jest-playwright-preset-0.0.10.tgz",
  "integrity": "sha512-iE7hO1v4UEvEpXXxF/7xLEnifsbceqd6UmDm82dACKakx7JAD9T3JdJkBX2j3HGOBlDYMdBeK5wdhfnWFQXDJA==",
  "dev": true,
  "requires": {
    "jest-dev-server": "^4.4.0",
    "playwright-chromium": ">=0.11.0",
    "playwright-core": ">=0.11.0",
    "playwright-firefox": ">=0.11.0",
    "playwright-webkit": ">=0.11.0"
  }
}

optionalDependencies does not work like you think: https://docs.npmjs.com/files/package.json#optionaldependencies

If a dependency can be used, but you would like npm to proceed if it cannot be found or fails to install, then you may put it in the optionalDependencies object.

optionalDependencies are like dependencies: always downloaded. the if it cannot be found does not seem to be respected. I know it sucks, this makes it pretty useless.

Problem is firefox, chromium and webkit are heavy downloads, specially for a CI (downloaded on each CI run).

Proposed solution: remove optionalDependencies and clearly states in the docs that the user should download either playwright (bundles the 3 browsers) or playwright-firefox or playwright-chromium or playwright-webkit (to be verified, this is my understanding). Maybe move playwright-core to devDependencies?

btw jest-puppeteer does not have any optionalDependencies: https://github.com/smooth-code/jest-puppeteer/blob/949027b1332e270fad78eda10fd1a92c56abe1b5/package.json

btw It's always nice to declare an empty dependencies: {} instead of omitting it: makes it clear when reading the package.json

page is not defined

Hi, I'm on [email protected], when I add your preset and attempt to reference page I get:

    ReferenceError: page is not defined

Do I need any additional configuration in addition to the one in the readme?

I tried to use browser.newPage() but it tells me browser is not defined

feat: access current device and browser inside a test

Maybe just add browser and device to the globals.

cc @mmarkelov

Migrate to new playwright version

Playwright 0.11.0 was released, as I know there were some changes with API, so it can need some changes to current implementation
#30

Add ability to setup playwright through file

jest-dev-server and wait-on

The current jest-dev-server in this package-lock.json (https://github.com/mmarkelov/jest-playwright/blob/master/package-lock.json) is:

    "jest-dev-server": {
      "version": "4.4.0",
      "resolved": "https://registry.npmjs.org/jest-dev-server/-/jest-dev-server-4.4.0.tgz",
      "integrity": "sha512-STEHJ3iPSC8HbrQ3TME0ozGX2KT28lbT4XopPxUm2WimsX3fcB3YOptRh12YphQisMhfqNSNTZUmWyT3HEXS2A==",
      "requires": {
        "chalk": "^3.0.0",
        "cwd": "^0.10.0",
        "find-process": "^1.4.3",
        "prompts": "^2.3.0",
        "spawnd": "^4.4.0",
        "tree-kill": "^1.2.2",
        "wait-on": "^3.3.0"
      }
    },

jest-dev-server version 4.4.0 (https://github.com/smooth-code/jest-puppeteer/blob/master/packages/jest-dev-server/package.json) has this in its package.json:

  "dependencies": {
    "chalk": "^3.0.0",
    "cwd": "^0.10.0",
    "find-process": "^1.4.3",
    "prompts": "^2.3.0",
    "spawnd": "^4.4.0",
    "tree-kill": "^1.2.2",
    "wait-on": "^4.0.0"
  }

How is it that wait-on then is only at version 3.3.0 above? I need to use features in 4.0.0.

Error when using custom Jest runner with selector engine: "shadow" selector engine has been already registered

UPDATED

Getting an error when using custom Jest runner with selector engine: "shadow" selector engine has been already registered. See repro in comments below.

Original problem below. After update, solved by upgrading also Playwright.

Just upgraded to latest jest-playwright (0.1.1) and get this when running:

FAIL   E2E  tests/e2e/suite/login/login.e2e.test.js
  ● Test suite failed to run

    Cannot evaluate a string with arguments

      at assert (../../node_modules/playwright-core/lib/helper.js:196:15)
      at Function.evaluationString (../../node_modules/playwright-core/lib/helper.js:25:13)
      at Selectors.register (../../node_modules/playwright-core/lib/selectors.js:31:40)
      at Selectors.<anonymous> (../../node_modules/playwright-core/lib/helper.js:55:31)
      at ../../node_modules/jest-playwright-preset/lib/utils.js:69:88
          at Array.map (<anonymous>)
      at Object.exports.getPlaywrightInstance (../../node_modules/jest-playwright-preset/lib/utils.js:69:41)
      at PlaywrightEnvironment.setup (../../node_modules/jest-playwright-preset/lib/PlaywrightEnvironment.js:75:36)

My jest-playwright.config.js:

const {
  selectorEngine,
} = require('query-selector-shadow-dom/plugins/playwright');

module.exports = {
  selectors: [{ name: 'shadow', script: selectorEngine }],
  server: {
    command: 'npm run start',
  },
};

Migrate dependency bump bot

See https://greenkeeper.io for more information.

Possible fixes are:
a) migrate to new Synk Greenkeeper bot
b) Use Renovate bot
c) Use Dependabot

I would choose a) or b). You can decide 👍

Development environment

• eslint
• prettier
• travis
• tests

Find out why all tests can't be run in the same Jest run

Currently we run them separately, would be cool to get rid of that and find the cause why we can't run them together.

Add ability to define device

https://github.com/microsoft/playwright/blob/master/docs/api.md#browsertypedevices

Failing tests return exit code 0 instead of 1

I set up a failing jest test. I observed the following behavior:

./node_modules/.bin/jest
--- TEST RUN ---
echo $?
1

When run with jest-playwright:

./node_modules/.bin/jest-playwright
--- TEST RUN ---
echo $?
0

This makes usage of jest-playwright in a CI system difficult.

Make priority for defined viewport from jest-playwright.config.js for devices

#31

Usage along with testing-library (jest-dom, react)

Hello,

I'd like to give jest-playwright a try to make e2e tests inside my project.

Right now I'm using testing-library to unit/integration test my React components.

I tried adding jest-playwright to my project to add e2e tests but now my test suite is failing with this error for every tests (even non e2e tests) :

FAIL  src/components/App/Login.test.tsx
  ● Test suite failed to run

    Protocol error (Target.setAutoAttach): Target closed.

      at node_modules/playwright-core/lib/chromium/crConnection.js:123:63
      at CRSession.send (node_modules/playwright-core/lib/chromium/crConnection.js:122:16)
      at CRSession.<anonymous> (node_modules/playwright-core/lib/helper.js:87:31)
      at Function.connect (node_modules/playwright-core/lib/chromium/crBrowser.js:59:27)
      at Chromium.launch (node_modules/playwright-core/lib/server/chromium.js:44:53)
      at getBrowserPerProcess (node_modules/jest-playwright-preset/lib/PlaywrightEnvironment.js:58:33)
      at PlaywrightEnvironment.setup (node_modules/jest-playwright-preset/lib/PlaywrightEnvironment.js:107:31)
        -- ASYNC --
      at CRSession.<anonymous> (node_modules/playwright-core/lib/helper.js:66:23)
      at Function.connect (node_modules/playwright-core/lib/chromium/crBrowser.js:59:27)
      at Chromium.launch (node_modules/playwright-core/lib/server/chromium.js:44:53)
      at getBrowserPerProcess (node_modules/jest-playwright-preset/lib/PlaywrightEnvironment.js:58:33)
      at PlaywrightEnvironment.setup (node_modules/jest-playwright-preset/lib/PlaywrightEnvironment.js:107:31)

I looked for this error and it seems that it's related to browser.close() : puppeteer/puppeteer#1947 (comment)

But I have no idea how to fix it.

Also, I'm surprised that this error also happens in my integration tests where I don't use playwright at all. Is there any way to run playwright only for a subset of my tests files? In my case all my e2e files have a specific e2e.ts extension.

Here is my repo : https://github.com/jgoux/storyflow
Doing a yarn install && yarn test should repro the error.

Thanks for your help.

EDIT : I was able to fix the separation of the configuration of jest using https://jestjs.io/docs/en/configuration.html#projects-arraystring--projectconfig 🌟

But the error remains on the e2e test I added.

[BUG] jest-dev-server won't be installed by default

Hi!

I just tried to install version 0.0.15 and it installs no requires, see here:

    "jest-playwright-preset": {
      "version": "0.0.15",
      "resolved": "https://registry.npmjs.org/jest-playwright-preset/-/jest-playwright-preset-0.0.15.tgz",
      "integrity": "sha512-DHWu0fNLzmfudVZ+wOKuMTiCpavu+TicMateKNgajH1A5RzDY78PceAGFrlEQIrnDyTsmhgbruMALgxEhprKcQ==",
      "dev": true
    },

This means e.g. jest-dev-server is not installed nor no other packages. Am I missing something?

Fork jest-dev-server?

Hi!

Is there a chance to fork the jest-dev-server into this repo or a separate repo? I just submitted two new issues on jest-dev-server (see argos-ci/jest-puppeteer#341 and argos-ci/jest-puppeteer#342) but the overall repo (https://github.com/smooth-code/jest-puppeteer/commits/master) seems a bit stale, 5 commits in soon 4 months.

Another idea would be to put jest-playwright and all its packages under an organisation instead of under mmarkelov/jest-playwright. E.g. jest-playwright/X or something, so you could bring more packages under there?

I am aware I am going out of my way here for asking for something where you are already donating a great deal of your free time. Which is very much appreciated.

[Bug] Can't connect to WebKit more than once

Cross-referencing microsoft/playwright#830

jest-playwright launches a browser once and then connects to it over webSocket for each environment. That works for Chromium and Firefox since they support multiple web socket clients, but fails for WebKit. I guess we did not do a great job explaining our launcherApp apis and limitations.

What our local harness does instead is running an instance of a browser app per test process. It uses regular playwright.launch for that and then exposes browser.newContext with test environment. @mmarkelov is this something you could do in jest-playwright?

[Feature]: Provide helper function to debug/pause execution

Similar to that: https://github.com/smooth-code/jest-puppeteer#globaljestpuppeteerdebug

No wsEndpoint support

Hi,

It seems this library does not yet have support to connect to a wsEndpoint?
jest-puppeteer does provide this support. We are looking to add documentation to our website HeadlessTesting.com to use jest-playwright with our grid, but we would need wsEndpoint support for this.

If interested, we can help build a PR to add this feature?

[Enhancement]: Ability to use only a selected browser to reduce install duration

Currently we need the full playwright package in order to use jest-playwright. It would be awesome, if we could use only a specific browser or two which would reduce the install duration.

They already provide these packages in:

• plawright-chromium
• playwright-webkit
• playwright-firefox

(ref: https://github.com/microsoft/playwright/tree/master/packages)

Instead of:

const playwright = require("playwright")
playwright["chromium"]...

it would be:

const chromium = require("playwright-chromium")
chromium...

BUG: beforeAll() with a page.goto with incorrect URL generates strange error: UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Target closed.

Promise errors in beforeAll() gives some un-understandable errors, see below. I could not replicate this with just using Playwright standalone.

Code

beforeAll(async () => {
  await page.goto(`http://0.0.0.5:8080/`);
});

it('enter password', async () => {
  await page.type('#password', 'test');
});

Output

(node:102419) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Target closed.
    at /home/userA/code/projectA/node_modules/playwright-core/lib/chromium/crConnection.js:121:63
    at new Promise (<anonymous>)
    at CRSession.send (/home/userA/code/projectA/node_modules/playwright-core/lib/chromium/crConnection.js:120:16)
    at CRExecutionContext.evaluate (/home/userA/code/projectA/node_modules/playwright-core/lib/chromium/crExecutionContext.js:69:50)
    at FrameExecutionContext._evaluate (/home/userA/code/projectA/node_modules/playwright-core/lib/dom.js:35:35)
    at FrameExecutionContext.ExecutionContext.evaluateHandle (/home/userA/code/projectA/node_modules/playwright-core/lib/javascript.js:24:25)
    at RerunnableTask._task (/home/userA/code/projectA/node_modules/playwright-core/lib/dom.js:485:24)
    at RerunnableTask.rerun (/home/userA/code/projectA/node_modules/playwright-core/lib/frames.js:836:23)
  -- ASYNC --
    at Frame.<anonymous> (/home/userA/code/projectA/node_modules/playwright-core/lib/helper.js:54:23)
    at Page.type (/home/userA/code/projectA/node_modules/playwright-core/lib/page.js:374:33)
    at Page.type (/home/userA/code/projectA/node_modules/playwright-core/lib/helper.js:55:31)
    at Object.<anonymous> (/home/userA/code/projectA/tests/sample.e2e.test.js:6:14)
    at Object.asyncJestTest (/home/userA/code/projectA/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
    at /home/userA/code/projectA/node_modules/jest-jasmine2/build/queueRunner.js:43:12
    at new Promise (<anonymous>)
    at mapper (/home/userA/code/projectA/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
    at /home/userA/code/projectA/node_modules/jest-jasmine2/build/queueRunner.js:73:41
(node:102419) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:102419) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
 FAIL   E2E  tests/sample.e2e.test.js (10.538s)
  ✕ enter password (5003ms)

  ● enter password

    While navigating to http://0.0.0.5:8080/: net::ERR_INVALID_ARGUMENT at http://0.0.0.5:8080/

      1 | beforeAll(async () => {
    > 2 |   await page.goto(`http://0.0.0.5:8080/`);
        |   ^
      3 | });
      4 |
      5 | it('enter password', async () => {

      at throwIfError (../node_modules/playwright-core/lib/frames.js:330:19)
      at Frame.goto (../node_modules/playwright-core/lib/frames.js:307:9)
      at Object.<anonymous> (sample.e2e.test.js:2:3)
        -- ASYNC --
      at Frame.<anonymous> (../node_modules/playwright-core/lib/helper.js:54:23)
      at Page.goto (../node_modules/playwright-core/lib/page.js:224:33)
      at Page.goto (../node_modules/playwright-core/lib/helper.js:55:31)
      at Object.<anonymous> (sample.e2e.test.js:2:14)

  ● enter password

    : Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error:

      3 | });
      4 |
    > 5 | it('enter password', async () => {
        | ^
      6 |   await page.type('#password', 'test');
      7 | });
      8 |

      at new Spec (../node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Object.<anonymous> (sample.e2e.test.js:5:1)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        10.836s
Ran all test suites matching /e2e/i.
Jest did not exit one second after the test run has completed.

This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.

Removing the beforeAll() and instead doing this:

it('enter password', async () => {
  await page.goto(`http://0.0.0.5:8080/`);
  await page.type('#password', 'test');
});

Instead gives a better error:

 FAIL   E2E  sample.e2e.test.js (5.565s)
  ✕ enter password (9ms)

  ● enter password

    While navigating to http://0.0.0.5:8080/: net::ERR_INVALID_ARGUMENT at http://0.0.0.5:8080/

Collaborate

@celeryclub asked about using jest-playwright with qawolf, which I thought was a great idea!

There is some overlap and it would be great for us to be able to use jest-playwright under the hood. I am curious if you are planning to maintain this library and if you are you open to collaborators?

jest-playwright.config.js in project root not working

The documentation states:

You can specify a jest-playwright.config.js at the root of the project

This is not true, as with the below configuration, where one use Jest projects, it does not find jest-playwright.config.js by default if it is placed in the project's root dir:

module.exports = {
  verbose: true,
  projects: [
    {
      displayName: 'UNIT:SERVER',
      testEnvironment: 'node',
      rootDir: 'server/',
    },
    {
      displayName: 'E2E',
      preset: 'jest-playwright-preset',
      rootDir: 'tests/',
      setupFilesAfterEnv: [
        '<rootDir>/jest.setup.js',
      ],
    },
  ],
};

This forces me to use JEST_PLAYWRIGHT_CONFIG, but it would be nice if this plugin actually looked at the Jest project root when finding the file.

Testing jest-playwright

#1 (comment)

Missing TypeScript typings for global page

In my source code I've added types/jest-playwright-preset.d.ts:

import { Page } from 'playwright';

declare global {
  const page: Page;
}

Allows page to be typed:

describe('Google', () => {
  beforeAll(async () => {
    await page.goto('https://whatismybrowser.com/')
  })
  ...

More or less related to #26

[BUG] page.route not working properly

Context:

• Jest-Playwright Version: 0.0.9
• Playwright Version: 0.11.1
• Operating System: Mac OS X 10.14.6
• Extra: [email protected] [email protected]

Code Snippet

    /** works **/
    // const { chromium } = require("playwright")
    // const mybrowser = await chromium.launch()
    // const mycontext = await mybrowser.newContext()
    // const mypage = await mycontext.newPage()
    // await mypage.route(/\.(png|jpeg|jpg)$/, request => request.abort());
    // await mypage.goto(baseUrl)

    /** doesn't work, throws error **/
    await page.route(/\.(png|jpeg|jpg)$/, request => request.abort());
    await page.goto(baseUrl)

Describe the bug

page.route throws the error below, works fine when instantiating playwright manually:

  console.error node_modules/jest-jasmine2/build/jasmine/Env.js:248
    Unhandled error

  console.error node_modules/jest-jasmine2/build/jasmine/Env.js:249
    Error: url parameter should be string, RegExp or function
        at Object.assert (/Users/aes/projects/amber/node_modules/playwright-core/lib/helper.js:196:15)
        at Object.urlMatches (/Users/aes/projects/amber/node_modules/playwright-core/lib/platform.js:206:14)
        at Page._requestStarted (/Users/aes/projects/amber/node_modules/playwright-core/lib/page.js:315:26)
        at FrameManager.requestStarted (/Users/aes/projects/amber/node_modules/playwright-core/lib/frames.js:145:24)
        at CRNetworkManager._onRequest (/Users/aes/projects/amber/node_modules/playwright-core/lib/chromium/crNetworkManager.js:169:34)
        at CRNetworkManager._onRequestPaused (/Users/aes/projects/amber/node_modules/playwright-core/lib/chromium/crNetworkManager.js:144:18)
        at CRSession.emit (events.js:193:13)
        at Promise.resolve.then (/Users/aes/projects/amber/node_modules/playwright-core/lib/chromium/crConnection.js:135:47)

jestPlaywright is not defined

After following the basic setup, I am running into following issue:

Environment
node -v 12.15.0
Windows 10

Only the first test case respects setting jest.setTimeout()

For some reason, the second (and third, and fourth and so on) test case does not respect what is set with jest.setTimeout(). Only the first test case does.

I've got a Jest config like this:

module.exports = {
  testSequencer: './tests/e2e/util/jestSequencer.js',
  projects: [
    {
      displayName: 'E2E',
      rootDir: 'tests/e2e',
      preset: 'jest-playwright-preset',
      runner: '<rootDir>/util/serialJestRunner.js',
      setupFilesAfterEnv: [
        '<rootDir>/jest.setup.js',
        '<rootDir>/util/registerAllureReporter',
      ],
      transformIgnorePatterns: [
        '/node_modules/(?!(query-selector-shadow-dom)/)',
      ],
    },
  ],
};

jest.setup.js sets jest.setTimeout(5000);.

When running two test cases where I on purpose delay them (to replicate this), this is what I receive:

 FAIL   E2E  tests/e2e/suite/login/login.e2e.test.js (21.387s)
  \u25cf Login page \u203a has a title

    : Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error:

      22 |   });
      23 | 
    > 24 |   it('has a title', async () => {
         |   ^
      25 |     await sleep(11000);
      26 |     const title = await page.title();
      27 |     expect(title).toBe('Login');

      at new Spec (../../node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.<anonymous> (suite/login/login.e2e.test.js:24:3)

 FAIL   E2E  tests/e2e/suite/login/foo.e2e.test.js (15.55s)
  \u25cf Console

    console.error node_modules/jest-jasmine2/build/jasmine/Env.js:248
      Unhandled error
    console.error node_modules/jest-jasmine2/build/jasmine/Env.js:249
      Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
          at CRSession.send (/home/userA/code/projectA/node_modules/playwright-core/lib/chromium/crConnection.js:121:19)
          at CRSession.<anonymous> (/home/userA/code/projectA/node_modules/playwright-core/lib/helper.js:84:31)
          at CRExecutionContext.evaluate (/home/userA/code/projectA/node_modules/playwright-core/lib/chromium/crExecutionContext.js:70:83)
        -- ASYNC --
          at Frame.<anonymous> (/home/userA/code/projectA/node_modules/playwright-core/lib/helper.js:63:23)
          at Page.title (/home/userA/code/projectA/node_modules/playwright-core/lib/page.js:285:33)
          at Page.title (/home/userA/code/projectA/node_modules/playwright-core/lib/helper.js:84:31)
          at Object.<anonymous> (/home/userA/code/projectA/tests/e2e/suite/login/login.e2e.test.js:26:30)

  \u25cf Login page \u203a has a title

    Timeout - Async callback was not invoked within the 10000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 10000ms timeout specified by jest.setTimeout.

Notice the 5000ms timeout (which is what I configured) vs the 10000ms, where I have no idea where that comes from.

It can be reproduced by having two tests files with this:

const { promisify } = require('util');

const sleep = promisify(setTimeout);

beforeAll(async () => {
  await page.goto(`http://example.com/`);
}, 10000);

it('sleep', async () => {
  await sleep(11000);
  expect(true).toBe(true);
});

Feature: add utility expect methods in Jest

Support features like jest-puppeteer: https://github.com/smooth-code/jest-puppeteer/blob/master/packages/expect-puppeteer/README.md#api

How to pass arguments to the browser executable?

How to pass arguments to the browser executable?

Use case:

• microsoft/playwright#1062
• https://github.com/arjun27/playwright-github-actions/blob/2770e149bdd75408203aa2b6590e11dafec6fdc3/test-project/index.js#L19

For GitHub Actions to work with Playwright, we need to pass --no-sandbox to chromium

I think the arguments should be passed somewhere here: https://github.com/mmarkelov/jest-playwright/blob/v0.0.10/src/PlaywrightEnvironment.js#L48

Migrate to playwright v0.12

Breaking changes here

Add configuration option to run all browsers in parallel

It would be great to be able to run tests in all browsers with just one command.