tnicola/cypress-parallel

Reduce up to 40% your Cypress suite execution time parallelizing the test run on the same machine.

cypress-parallel

Reduce up to 40% your Cypress suite execution time parallelizing the test run on the same machine.

cypress	cypress-parallel

Run your Cypress tests in parallel (locally)

How it works

🔍 - Search for existing Cypress tests
📄 - Read (if exists) a weight file
⚖️ - Split spec files into different threads
⚙️ - For each thread it runs the Cypress command you've passed as argument
📈 - Wait for all threads to finish and collects the result in a single report

How to use

Install

npm i cypress-parallel -D

or

yarn add cypress-parallel -D

Add a new script

In your package.json add a new script:

"scripts" :{
  ...
  "cy:run": "cypress run", // It can be any cypress command with any argument
  "cy:parallel" : "cypress-parallel -s cy:run -t 2 -d '<your-cypress-specs-folder>' -a '\"<your-cypress-cmd-args>\"'"
  ...
}

With Arguments

Sample:

-a '\"--config baseUrl=http://localhost:3000\"'

Launch the new script

npm run cy:parallel

or

Run with npx (no package installation needed)

npx cy:parallel -s cy:run -t 2 -d '<your-cypress-specs-folder>' -a '"<your-cypress-cmd-args>"'

Passing Specs

cypress-parallel -s cy:run -t 2 -a '\"<your-cypress-cmd-args>\"' --spec path/to/spec1.spec.js path/to/spec2.spec.js

Scripts options

Option	Alias	Description	Type
--help		Show help
--version		Show version number
--script	-s	Your npm Cypress command	string
--args	-a	Your npm Cypress command arguments	string
--threads	-t	Number of threads	number
--specsDir	-d	Cypress specs directory	string
--spec		Cypress spec file paths	string
--weightsJson	-w	Parallel weights json file	string
--reporter	-r	Reporter to pass to Cypress.	string
--reporterOptions	-o	Reporter options	string
--reporterModulePath	-n	Specify the reporter module path	string
--bail	-b	Exit on first failing thread	string
--verbose	-v	Some additional logging	string
--strictMode	-m	Add stricter checks after running the tests	boolean

NB: If you use cypress-cucumber-preprocesor, please disable the strictMode to avoid possible errors:

"scripts" :{
  ...
  "cy:parallel" : "cypress-parallel -s cy:run -t 4 -m false"
  ...
}

NB: If your cypress-multi-reporters module is not found on the same level as your Cypress suite (e.g. in a mono-repo) then you can specify the module directory for Cypress to search within.

"scripts" :{
  ...
  "cy:parallel" : "cypress-parallel -s cy:run -t 4 -n .../../../node_modules/cypress-multi-reporters"
  ...
}

Env variables

CYPRESS_THREAD

You can get the current thread index by reading the CYPRESS_THREAD variable.

 const threadIndex = process.env.CYPRESS_THREAD;
 // return 1, 2, 3, 4, ...

Contributors

Looking for contributors.

License

This project is licensed under the MIT license. See LICENSE.

cypress-parallel's Issues

Unhandled 'error' event

cypress-parallel: 0.1.7
cypress: 4.12.1
NPM: 6.14.4
OS: macOS

Creating weight file: cypress/parallel-weights.json
events.js:292
      throw er; // Unhandled 'error' event
      ^

Error: spawn yarn ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
    at onErrorNT (internal/child_process.js:467:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
Emitted 'error' event on ChildProcess instance at:
    at Process.ChildProcess._handle.onexit (internal/child_process.js:273:12)
    at onErrorNT (internal/child_process.js:467:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn yarn',
  path: 'yarn',
  spawnargs: [
    'run',
    'cy:run',
    '--',
    '--reporter',
    'cypress-parallel/json-stream.reporter.js',
    '--spec',
    "'cypress/integration/audit-lighthouse.spec.js,cypress/integration/cookies.spec.js,cypress/integration/notification.spec.js,cypress/integration/visual.spec.js'"
  ]
}

Using a shared node.js for all running agents

Currently I use 4 "agents" to run in parallel with command line script "-t 4"
Is it possible to use shared node.js modules for every agent or will each agent always run separately?

I'd like to try and create a shared queue for dynamically created test users, but at the moment all four agents are overwriting the shared list of used test accounts with the cy.writeFile command

Since the tests run in parallel, when each test creates an account, they write them to the array of users at the same time, resulting in array containing 4-5 users instead of intended 30.

Is there any way to resolve this?

Reporting Attributes not set properly from individual specs

We are using Report portal as reporter,

and we are setting test attributes individually from Spec files, and they are set very randomly when using Cypress parallel

Example

spec 1 -

const attributes = [{
key: "type",
value: "suite1",
}
beforeEach(() => {
cy.addTestAttributes(attributes)
})

spec 2 -

const attributes = [{
key: "type",
value: "suite2",
}
beforeEach(() => {
cy.addTestAttributes(attributes)
})

Report portal test attributes are appearing as:

Spec1 - empty
Spec2 - type:suite1 and type:suite2

The Async calls where you gather the results must be misplacing this

Can you please resolve the issue ?

Error: Cannot find module 'colors/safe'

I'm getting this error in a new clean repo:

internal/modules/cjs/loader.js:883
  throw err;
  ^

Error: Cannot find module 'colors/safe'

With simple config:

{
  "name": "cypress-parallel-running-2",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts" :{
    "cy:run": "cypress run",
    "cy:parallel" : "cypress-parallel -s cy:run -t 2 -d cypress/integration/**/*.spec.js"
  },
  "dependencies": {
    "cypress": "^9.6.0",
    "cypress-parallel": "^0.9.0"
  }
}

Run command:
$ npm run cy:parallel

Threads seem to lose connection sometimes

First of all: great job on this library :)

Every now and then I get the following errors:

cypress.32683  | [80:0228/164642.193174:ERROR:connection.cc(66)] X connection error received.
cypress.32683  | [400:0228/164642.193183:ERROR:connection.cc(66)] X connection error received.
cypress.32683  | [400:0228/164642.193180:ERROR:connection.cc(66)] X connection error received.
cypress.32683  | Gdk-Message: 16:46:42.236: Cypress: Fatal IO error 11 (Resource temporarily unavailable) on X server :99.
cypress.32683  | 
cypress.32683  | Thread 1 likely finished with failure count: 0
cypress.32683  | error Command failed with exit code 1.
cypress.32683  | Thread 0 likely finished with failure count: 1

But when I check the mocha results or run again none of the tests should have failed

Cypress version: cypress/included:9.5.0
Running in: docker-compose
Command:

yarn run cy:parallel \
    --threads 2 \
    --reporter mocha-junit-reporter \
    --reporterOptions mochaFile="cypress/results/${1}/junit/outcome-[hash].xml" \
    --args '"--browser chrome"' \
    --verbose

Anything I can log or do to provide more information?

Set env variable to indicate current thread

I tried cypress-parallel but I ran into an issue where the different threads are filling the same mysql db
It would be great if an env variable is set to indicate which thread is running the current test.
That way, I can possibly seed a different db for each thread

Thank you

"reporter" parameter is ignored

Providing the --reporter path parameter doesn't change the hardcoded --reporter cypress-multi-reporters value when the command is run.

Being able to provide custom reporter path is useful in monorepos, where node_modules is not in the same path as the project.

Current behaviour:

When running cypress-parallel -s test:cypress:headless -t 1 -d tests/e2e/specs/ -r '../../node_modules/cypress-multi-reporters'

The generated command is:

cypress run --config-file tests/e2e/config/local.json --browser chrome '' --spec (...) --reporter cypress-multi-reporters --reporter-options configFile=(...)multi-reporter-config.json

And my expectation would be to see the provided reporter in the generated command:

cypress run --config-file tests/e2e/config/local.json --browser chrome '' --spec (...) --reporter '../../node_modules/cypress-multi-reporters' --reporter-options configFile=(...)multi-reporter-config.json

parallel-weights.json not generated

When cypress-parallel is running in a Docker container that exits immediately after having completed the tests suites, it never writes the file parallel-weights.json because the process terminate before.
I think that the problem it's caused by using asynchronous write API to write the file.

function generateWeightsFile(specWeights, totalDuration, totalWeight) {
  Object.keys(specWeights).forEach((spec) => {
    specWeights[spec].weight = Math.floor(
      (specWeights[spec].time / totalDuration) * totalWeight
    );
  });
  const weightsJson = JSON.stringify(specWeights);
  fs.writeFile(`${settings.weightsJSON}`, weightsJson, 'utf8', (err) => {
    if (err) throw err;
    console.log('Generated file parallel-weights.json.');
  });
}

I have changed this function and apparently works:

function generateWeightsFile(specWeights, totalDuration, totalWeight) {
  Object.keys(specWeights).forEach((spec) => {
    specWeights[spec].weight = Math.floor(
      (specWeights[spec].time / totalDuration) * totalWeight
    );
  });
  const weightsJson = JSON.stringify(specWeights);
  fs.writeFileSync(`${settings.weightsJSON}`, weightsJson, 'utf8');
}function generateWeightsFile(specWeights, totalDuration, totalWeight) {
  Object.keys(specWeights).forEach((spec) => {
    specWeights[spec].weight = Math.floor(
      (specWeights[spec].time / totalDuration) * totalWeight
    );
  });
  const weightsJson = JSON.stringify(specWeights);
  fs.writeFileSync(`${settings.weightsJSON}`, weightsJson, 'utf8');
}

Extend documentation for Scripts options

For better understanding of the possible script options, it would be great to extend this section from README file

1. Add columns to script options table:
   1.1 column to show if option is required or optional
   1.2 column to show what will be the default value (example: strictMode)

2. Add examples for script options:
   2.1 show one example which include all the options without placeholder. Currently only for -m and -a are documented.

Not working with cypress-cucumber-preprocesor

When using cypress cucumber preprocessor in a typical setup the .feature file and stepDefinition folder lives in the integration folder of cypress, in cli.js

https://github.com/tnicola/cypress-parallel/blob/master/lib/cli.js#L90

This line above is not letting the process to be finish because the statement is always falsy. testSuitePaths is returning the files that include .feature files and resultMaps only return the number of test runs which will never going to match.

This check is not fool prove it has made assumption if (timeMap.size !== testSuitePaths.length), it assumes that testSuitePaths will always going to have the number of files equal to files in resultMaps.

This tool has a great potential but this needs to be fix by some way, i am suggesting to optionally ignore the check and let consumer decide if they want to that check or add modes like strict: true and do this check else ignore this check.

While executing parallel tests, it waits for other thread to get completed to start a new tests on a thread where execution already completed

I have a question. This is what my scenarios looks like. I have 10 tests and 1 tests takes around 10 mins (Can't break down the test due to known limitation) and others are taking less than 2 mins individually.
I was thinking to run tests on parallel with two threads where in one thread I can run test which is taking more time and in other I will keep all others test.
But with this tool, after running it with two threads, Second thread gets into a waiting until and unless the first thread execution gets completed.
Can someone help me with the solution? Thanks in advance!

[Feature Request] add parallel-weights.json routing to args

Being able to specify the directory the cypress tests live is invaluable, but it's pointless if we require a cypress folder anyway for the parallel-weights.json.

I tried to push a PR, but I was unable to sadly - good old permission denied!

Here's the changes I suggest to lib/cli.js:

Line 10:

const argv = yargs
  .option('script', {
    alias: 's',
    type: 'string',
    description: 'Your npm Cypress command'
  })
  .option('threads', {
    alias: 't',
    type: 'number',
    description: 'Number of threads'
  })
  .option('specsDir', {
    alias: 'd',
    type: 'string',
    description: 'Cypress specs directory'
  })
  .option('args', {
    alias: 'a',
    type: 'string',
    description: 'Your npm Cypress command arguments'
  })
  .option('reporter', {
    alias: 'r',
    type: 'string',
    description: 'Reporter to pass to Cypress'
  })
  .option('reporterOptions', {
    alias: 'o',
    type: 'string',
    description: 'Reporter options'
  })
  .option('weightsFile', {
    alias: 'w',
    type: 'string',
    description: 'Route to your weights file'
  }).argv;

Line 55:

const WEIGHTS_JSON = argv.weightsFile ? argv.weightsFile : 'cypress/parallel-weights.json';

thrown errors on hidden files e.g. .DS_Store

Currently script scans directory and treats all files in selected directory as spec files.
I got error on .DS_Store, so it might occur with other files e.g. .gitignore.

It would be nice to have one of the options available:

• hardcoded filtering files and include *.js only
• possibility to define filter in -d option, or additional option, which will do filtering out for user

e.g.

# --include option
cypress-parallel -s cy:run -t 3 -d ./cypress/integration/examples --include *.js
# -- exclude option
cypress-parallel -s cy:run -t 3 -d ./cypress/integration/examples --exclude *.docx

Thrown error you can find below.

 npm
  ERR! code ELIFECYCLE
npm ERR! errno 1

 npm
 ERR! [email protected] cy:run: `cypress run "--reporter" "cypress-parallel/json-stream.reporter.js" "--spec" "'cypress/integration/examples/.DS_Store'"`
npm ERR! Exit status 1

 npm ERR!

npm ERR! Failed at the [email protected] cy:run script.

 npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Flag -b not working as intended ?

Hi,
We're using cypress-parallel on our CI and we were expecting the -b flag to stop every tests running but it seems to skip only the next tests inside the currently playing file but still continue with the next ones according to the runner's parameters. Is it correct or is this an issue with the flag ? In case this is working as intended, could you implement a solution to stop absolutely all running tests on the first failing test ?
Thank's.

My System keeps restarting as soon as I run test in parallel

OS: Mac
Version: 11.0.1

Git Repo
https://github.com/kunalashar25/learn-cypress

Executing using command
npm run cy:parallel

Please let me know why am I facing this issue.

[Feature request] Support folder structure

It would be great if the plugin would also support a folder structure. So e.g. -d cypress/interactive/**/ where ** stands for any number of folders.
It seems that currently only one folder with all files on one level is supported?

The command line is too long.

I've tried your extension but end up with having this error message: The command line is too long.

My setup: cypress-parallel -s cypress:run -t 2 -d cypress -a '--config baseUrl=https://localhost'

Using: Cypress 5.3.0 on Windows

Maybe the solution is obvious and if so I am sorry for bothering you.

Error message in latest release : 'Node.js, fs.rmdir(path, { recursive: true }) will be removed'

Getting below error message after installing latest release 0.5.0.

(node:5337) [DEP0147] DeprecationWarning: In future versions of Node.js, fs.rmdir(path, { recursive: true }) will be removed. Use fs.rm(path, { recursive: true }) instead
(Use node --trace-deprecation ... to show where the warning was created)
node:fs:1526
handleErrorFromBinding(ctx);

is anyone facing the same issue?

Question about: Some test suites likely terminated without passing on results

Hi,

First off, thanks for the repo and work :)

So i want to use this package in our tests, but am seeing the following error randomly on my test runs:
Some test suites likely terminated without passing on results

I know that it's a validation, but do we know why this is happening?

Thanks,
-arnon

Facing performance issue in parallel execution

Hi,

I have a smoke suite with 21 scripts which takes 5 mins to execute in single thread chrome headed mode, Where when the same is tried with 4 thread parallel execution mode, The execution takes 12 minutes and below are parallel observations.

1. CPU utilization during execution is above 80%
2. Memory Utilization during execution is above 75%
3. The application hangs

Cypress Version: 8.3.1

Script failed when 'runner-results' folder not exist or if not empty

Hi,

in the start script, there is a line code that remove the 'runner-results' folder but the script throw an exception in 2 cases:
-> when this folder not exist
-> when this folder exist and it contains files

So before run the cmd, i must clean the folder or create a empty 'runner-results' folder. Could you handle this exception?

cypress-parallel -s cy:run -t 4 -d cypress/integration/ -p cypress-report-config.json

(node:30920) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, rmdir 'C:\Personal_Unsaved\Projet\front-end\runner-results'
at Object.rmdirSync (fs.js:684:3)
at cleanResultsPath (C:\Personal_Unsaved\Projet\front-end\node_modules\cypress-parallel\cli.js:19:6)
at start (C:\Personal_Unsaved\Projet\front-end\node_modules\cypress-parallel\cli.js:23:3)
at Object. (C:\Personal_Unsaved\Projet\front-end\node_modules\cypress-parallel\cli.js:124:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
(node:30920) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:30920) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

support for run-ct command

Hello,

Our team is trying to adopt component testing of cypress and wonders if we can run them in parallel locally.
I have tried using cypress-parallel package to run in parallel, hoping that I just need to swap run command to run-ct. But unfortunately it did not work.

Before I start pasting error messages and my attempts, I would like to know if it is officially supported (or it is possible) from this repo, or has anyone successfully got it working.

Cheers

Running cypress-parallel directly from command line (not via scripts)

Is it possible to run cypress-parallel directly from command line i.e npx cypress-parallel cypress run -t 2 -d cypress/integration/*?

Reason I ask is that I'm trying to get this working inside a large mono repo and I cannot use scripts.

Compatibility with cypress-cucumber-preprocessor

Hi,

I just wanted to ask if there could be some compatibility added to the cypress-tags command from the cypress-cucumber-preprocessor plugin.
From what I've noticed running parallel on a cypress-tags command will not actually run anything in parallel (execution time stay the same even with 100+ tests) and also the .json report files don't get generated for any of the specs.
I don't really understand the mechanism behind this parallelization, but since cypress-tags is a wrapper of the cypress run command, I would assume it could also work in parallel.

Cucumber is very popular and the cy-cucumber-preprocessor plugin is used on a lot of projects I've seen, so I think it would be great if there was any support for it in parallel execution.

Cannot find module 'is-npm'

Cypress-parallel fails to start if is-npm is not installed in the project
Installing is-npm manually fixes the issue

Random error when running parallel tests in Cypress

Hi team,

When running my tests with cypress-parallel I encounter about 50% of the time the following error:

This error fails the suite so it is problematic.

My main issue issue is that it does not happen consistently. I can run 6 suites in parallel, 2 random ones will encounter this and thus only 4 will finish. Next run it won't happen; next run a different suite gets the error etc.

Have any of you encountered this? If not, any clue how to debug?

Thanks in advance :)

Files do contain DOS line endings

The Javascript files do contain DOS line endings (CRLF) which causes errors with yarn.

Should be LF as line endings

ERROR: env: node\r: No such file or directory

When I try to run cy:parallelcy:parallel it exits with error:

env: node\r: No such file or directory
error Command failed with exit code 127.

command looks like this:
"cy:parallel" : "cypress-parallel -s cypress:run -t 2 -d cypress",

Video Recording not working in latest release

I just updated to today's new release (https://github.com/tnicola/cypress-parallel/releases/tag/0.5.0) and cypress video recording is no longer working

Error from console:

13:44:46  Warning: We failed to record the video.
13:44:46  
13:44:46  This error will not alter the exit code.
13:44:46  
13:44:46  Error: ffmpeg exited with code 1: /home/ubuntu/workspace/[branch]/cypress/videos/[filepath].mp4: No such file or directory
13:44:46  
13:44:46      at ChildProcess.<anonymous> (/home/ubuntu/.cache/Cypress/8.3.1/Cypress/resources/app/packages/server/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
13:44:46      at ChildProcess.emit (events.js:315:20)
13:44:46      at Process.ChildProcess._handle.onexit (internal/child_process.js:277:12)

I'm not sure if this error is actually being caused by the new changes to this package or is an issue in cypress that was suddenly uncovered by the recent changes. I found a related issue here where other people seem to have a similar issue: cypress-io/cypress#9128 (comment)

reproduction steps:
in cypress.json set "video": true, and then run cypress-parallel via command line

info:
Cypress: v8.3.0
cypress-parallel: v0.5.0
run environment: ci/cypress run/jenkins and my local machine's macOS
browser: electron 91

possibly relevant:
"viewportWidth": 1920,
"viewportHeight": 1080
cypress-io/cypress#3491 (comment)

Error: Cannot find module 'is-npm'

Error while running command:

cypress-parallel -s cy:run -t 3 -d ./cypress/integration -a '\"--config baseUrl=https://domain.com\"'

Error: Cannot find module 'is-npm'
Require stack:
- \cypress\node_modules\cypress-parallel\cli.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:965:15)
    at Function.Module._load (internal/modules/cjs/loader.js:841:27)
    at Module.require (internal/modules/cjs/loader.js:1025:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (\cypress\node_modules\cypress-parallel\cli.js:7:19)
    at Module._compile (internal/modules/cjs/loader.js:1137:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
    at Module.load (internal/modules/cjs/loader.js:985:32)
    at Function.Module._load (internal/modules/cjs/loader.js:878:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '\\cypress\\node_modules\\cypress-parallel\\cli.js'
  ]
}

ENAMETOOLONG in runner-result

Hello!
I have an issue with ENAMETOOLONG
the problem is we have deep hierarchical directory structure. it causes very long name in runner-result, so we face ENAMETOOLONG error.
Looks like we need api to control runner-result names to solve the problem

npm Cypress command arguments via -a results as baseUrl is null

I need to use -a option within the command line, as it will become an input parameter in my GitHub workflow. But when I pass -a option in the command line this results as baseUrl is null.

Is this an issue in the latest version or I'm doing something wrong?

"devDependencies": {
    "cypress": "^9.5.1",
    "cypress-multi-reporters": "^1.5.0",
    "cypress-parallel": "^0.9.0"
  }

"scripts": {
    "cy:run": "cypress run --browser chrome"
  },

Run command:

npx cypress-parallel -s cy:run -t 2 -d cypress/integration -a '\"--config baseUrl=https://example.cypress.io\"'

baseUrl is null

Works fine with regular Cypress run command

$ npx cypress open --config baseUrl=https://example.cypress.io

Also passing the option in script does work:

"scripts": {
    "cy:run": "cypress run --browser chrome --config baseUrl=https://example.cypress.io"
  },

$ npx cypress-parallel -s cy:run -t 2 -d cypress/integration

Summation problem on Pending tests

I think there might be a summation problem. Note the results in the last box for pending tests vs the box right above it.

I know that this reports 14 failing tests, but in the detail of my table, there's only 2 tests that fail. This does not correlate to MaxListener problems while taking test recordings. I see those and have counted them with the failed tests and it doesn't add up to 14.

I do like this package though. It's pretty nifty and reduces my run time by about 45-55% (~1600-1800 in the UI, vs 730-980 in parallel mode). Thanks!

Non 0 output for failed tests? (Usage in CI)

Hi there,

I am running cypress parallel in CI and was wondering if it's possible to return a non-zero output if one test fails?

Here's the command I'm running: cypress-parallel -s cy:run -t 4 -d 'cypress/integration/v2'

Thank you!

Cannot execute just a spec (or group of them) inside the specs folder

Hi,
first of all, thank you for your contribution, this can be very useful for our daily testing activities.

I see a problem, and it is that I cannot find a way to execute just a spec (or group of them) inside the specs folder.

For example, in regular Cypress I'll do:

"cypress run --spec 'cypress/integration/a.spec.js,cypress/integration/b.spec.js'  --reporter cypr...

But if in cypress-parallel I try to do this, it does not work:

"parTest": "cypress-parallel -s cy:run -t 2 -a '\"--spec 'cypress/integration/a.spec.js,cypress/integration/b.spec.js'\"' ",
"cy:run": "cypress run ",

I also tried other alternatives but I can not find a way to do this.
What I am doing wrong? Is it possible to achieve this with current version of cypress-parallel?

Thank you very much in advance!

Question - Avoid multi-reporter-config.json and runner-result creation

Hi, Is there a way to avoid the multi-reporter-config.json file creation and also the runner-result folder creation, I don't want that since I am only interested on the console output.

Use the `integrationFolder` from cypress.json

Hi,

Great repo, looking forward to having a play.

Instead of passing through --specsDir (and then default to integration), what about using the integrationFolder parameter from cypress.json?

Just an idea :)

Missing documentation for weight file and multi-reporter-config.json

I would love to see some documentation with an example of using a weight file correctely.
Furthermore the multi-reporter-config.json file which pops up after execution is not documented at all.

feature-request: Grouping tests

I would appreciate if there is a possibility to group tests. So that all test in a group will be run sequentially in a thread. Maybe grouping by folders could be a first approach.
In my case I have tests competing for the backend.

Error: Cannot find module 'colors/safe'

I hit this today on fresh install of 0.9.0

#101

Sounds like the fix is to add colors as a dependency of cypress-parallel

Support multi-machine execution via weight file

Hi,

Great tool. I'd like to further improve test speed by using multiple machines in the CI and split the tests between them using the generated weight file. I.e. commit weight file, then specify the group to be run: cypress-parallel --executors 5 --group 1

Is this possible or could it be supported?

Thanks!

Tests Skipped in a feature is not displayed in reporting

Let's say, I have 2 features with 2 tests each.
Feature 1 - 2 smoke tests
Feature 2 - 1 smoke & 1 regression tests

If i use this plugin to execute smoke tests, only feature 1 is displayed in the console report. Feature 2 is not displayed.
I tried printing the [test] and found that only for the feature where all the tests executed, suiteEnd is displayed and not for the feature thats partially executed.

Root cause Assumption -
if (test[0] === 'suiteEnd' && test[1].title != null) { timeMap.set(test[1].title, { ...test[1], duration: suiteDuration }); }

Could someone please look into it?

parallel reporting

Hello,

For some reason the parallel run doest work with reporting and it only shows the results in the terminal, and doesn't create normal report if you use it with mochawesome reporting for example.
Best Regards

empty output and empty parallel-weights.json when tests complete

Greetings, @tnicola!

I've set up these commands as per the README:

"cy:run": "cypress run",
"cy:parallel-local": "cypress-parallel -s cy:run -t 3 -d cypress/integration/test"

Relative to package.json, I have three spec files in cypress/integration/test. Running yarn cy:parallel-local starts up cypress-parallel. The tests then run invisibly. (I can see the activity hitting my API.) When the tests conclude, this is printed to the console:

Thread 0 time: 0s
Thread 1 time: 0s
Thread 2 time: 0s
┌─────────────────────────┬────────┬───────┬─────────┬─────────┬─────────┐
│ Spec                    │ Time   │ Tests │ Passing │ Failing │ Pending │
└─────────────────────────┴────────┴───────┴─────────┴─────────┴─────────┘
Generated file parallel-weights.json.

The parallel-weights.json file is just an empty object. I do get an .mp4 generated for each spec, and the results logged to Cypress Dashboard (if use the --record option). However, it seems that the tests are being run in serial and not in parallel. Running with Yarn 2. Let me know if you have any ideas! Thanks!

npm i cypress-parallel installes version 0.1.8

how do you get the latest installation?

Question / Enhancement: Specify Output Folder for runner-results

Hey,

Is there a way to specify where the runner-results folder is generated. I use a reports folder, and would be great to have it in there rather than the root.

while running this command with cucumber project it is showing error as 'error: unknown option: -d'.

while running this command with cucumber project it is showing error as 'error: unknown option: -d'.

My command is :
npx cypress run -t 2 -d 'cypress/integration/step_definitions/ui' -a TAGS='@smoke and not (@Skip or @underfix)'

My step definition files are in the folder cypress/integration/step_definitions/ui

Can i please know where i have gone wrong

/usr/bin/env: ‘node\r’: No such file or directory error

Quick question, does the package run only on Windows so far?

When I try to execute it on a Linux machine (Ubuntu), I got the following error:

/usr/bin/env: ‘node\r’: No such file or directory
error Command failed with exit code 127.

To fix it temporarily, I just have to edit the cli.js file to replace CRLF with LF in the first line and then it works great.

But it means currently I can't easily integrate it in our project for all my developers + I'd like to test it in our CI :-)

.

Create a test report in html format

Is it possible for us to generate report in html format