Giter Site home page Giter Site logo

jaredpalmer / cypress-image-snapshot Goto Github PK

View Code? Open in Web Editor NEW
879.0 18.0 163.0 1.91 MB

Catch visual regressions in Cypress

License: MIT License

JavaScript 100.00%
cypress cypress-plugin image-diff jest-image-snapshot visual-regression-testing visual-regression-tests visual-regression image-diffs

cypress-image-snapshot's Introduction

Cypress Image Snapshot

Cypress Image Snapshot binds jest-image-snapshot's image diffing logic to Cypress.io commands. The goal is to catch visual regressions during integration tests.

Discord

See it in action!

Cypress GUI

When using cypress open, errors are displayed in the GUI.

Cypress Image Snapshot in action

Composite Image Diff

When an image diff fails, a composite image is constructed.

Cypress Image Snapshot diff

Test Reporter

When using cypress run and --reporter cypress-image-snapshot/reporter, diffs are output to your terminal.

Cypress Image Snapshot reporter

Installation

Install from npm

npm install --save-dev cypress-image-snapshot

then add the following in your project's <rootDir>/cypress/plugins/index.js:

const {
  addMatchImageSnapshotPlugin,
} = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on, config);
};

and in <rootDir>/cypress/support/commands.js add:

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

addMatchImageSnapshotCommand();

Syntax

// addMatchImageSnapshotPlugin
addMatchImageSnapshotPlugin(on, config);

// addMatchImageSnapshotCommand
addMatchImageSnapshotCommand();
addMatchImageSnapshotCommand(commandName);
addMatchImageSnapshotCommand(options);
addMatchImageSnapshotCommand(commandName, options);

// matchImageSnapshot
.matchImageSnapshot();
.matchImageSnapshot(name);
.matchImageSnapshot(options);
.matchImageSnapshot(name, options);

// ---or---

cy.matchImageSnapshot();
cy.matchImageSnapshot(name);
cy.matchImageSnapshot(options);
cy.matchImageSnapshot(name, options);

Usage

In your tests

describe('Login', () => {
  it('should be publicly accessible', () => {
    cy.visit('/login');

    // snapshot name will be the test title
    cy.matchImageSnapshot();

    // snapshot name will be the name passed in
    cy.matchImageSnapshot('login');

    // options object passed in
    cy.matchImageSnapshot(options);

    // match element snapshot
    cy.get('#login').matchImageSnapshot();
  });
});

Updating snapshots

Run Cypress with --env updateSnapshots=true in order to update the base image files for all of your tests.

Preventing failures

Run Cypress with --env failOnSnapshotDiff=false in order to prevent test failures when an image diff does not pass.

Reporter

Run Cypress with --reporter cypress-image-snapshot/reporter in order to report snapshot diffs in your test results. This can be helpful to use with --env failOnSnapshotDiff=false in order to quickly view all failing snapshots and their diffs.

If you using iTerm2, the reporter will output any image diffs right in your terminal ๐Ÿ˜Ž.

Multiple reporters

Similar use case to: https://github.com/cypress-io/cypress-example-docker-circle#spec--xml-reports

If you want to report snapshot diffs as well as generate XML junit reports, you can use mocha-multi-reporters.

npm install --save-dev mocha mocha-multi-reporters mocha-junit-reporter

You'll then want to set up a cypress-reporters.json which may look a little like this:

{
  "reporterEnabled": "spec, mocha-junit-reporter, cypress-image-snapshot/reporter",
  "mochaJunitReporterReporterOptions": {
    "mochaFile": "cypress/results/results-[hash].xml"
  }
}

where reporterEnabled is a comma-separated list of reporters.

You can then run cypress like this:

cypress run --reporter mocha-multi-reporters --reporter-options configFile=cypress-reporters.json

or add the following to your cypress.json

{
  ..., //other options
  "reporter": "mocha-multi-reporters",
  "reporterOptions": {
    "configFile": "cypress-reporters.json"
  }
}

Options

  • customSnapshotsDir : Path to the directory that snapshot images will be written to, defaults to <rootDir>/cypress/snapshots.
  • customDiffDir: Path to the directory that diff images will be written to, defaults to a sibling __diff_output__ directory alongside each snapshot.

Additionally, any options for cy.screenshot() and jest-image-snapshot can be passed in the options argument to addMatchImageSnapshotCommand and cy.matchImageSnapshot(). The local options in cy.matchImageSnapshot() will overwrite the default options set in addMatchImageSnapshot.

For example, the default options we use in <rootDir>/cypress/support/commands.js are:

addMatchImageSnapshotCommand({
  failureThreshold: 0.03, // threshold for entire image
  failureThresholdType: 'percent', // percent of image or number of pixels
  customDiffConfig: { threshold: 0.1 }, // threshold for each pixel
  capture: 'viewport', // capture viewport in screenshot
});

How it works

We really enjoy the diffing workflow of jest-image-snapshot and wanted to have a similar workflow when using Cypress. Because of this, under the hood we use some of jest-image-snapshot's internals and simply bind them to Cypress's commands and plugins APIs.

The workflow of cy.matchImageSnapshot() when running Cypress is:

  1. Take a screenshot with cy.screenshot() named according to the current test.
  2. Check if a saved snapshot exists in <rootDir>/cypress/snapshots and if so diff against that snapshot.
  3. If there is a resulting diff, save it to <rootDir>/cypress/snapshots/__diff_output__.

Cypress Version Requirements

Cypress's screenshot functionality has changed significantly across 3.x.x versions. In order to avoid buggy behavior, please use the following version ranges:

  • cypress-image-snapshot@>=1.0.0 <2.0.0 for cypress@>=3.0.0 <3.0.2
  • cypress-image-snapshot@>2.0.0 for cypress@>3.0.2.

cypress-image-snapshot's People

Contributors

asirgado avatar dependabot[bot] avatar github-actions[bot] avatar imgbotapp avatar jackjocross avatar james-e-adams avatar jaredpalmer avatar jhoobergs avatar jhswart avatar lazarljubenovic avatar lmeikle avatar michaeldeboey avatar raag007 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cypress-image-snapshot's Issues

Document multi-reporter setup

Since cypress-image-snapshot/reporter may be used with additional test reporters, we should document how to do so.

False positive on images

Often we see false positives on images used, for example:
element_0_1024x768 diff

To me it feels/looks like there's some compression on the baseline, which is not applied when comparing the screenshots.

CHANGELOG missing

I just saw the package is updated to v2.3.5, but I can't find any CHANGELOG to check out the changes.

Would you guys be so kind to provide us with a decent CHANGELOG please? ๐Ÿ™‚

Thanks for all your hard work! ๐Ÿ™‚

Difference in color is not considered in image snapshot

Problem: Colors should be considered as a part of the snapshot image difference

Issue: A difference in color is not considered as a failing point for Cypress Image Snapshot
PFB Pictures

PFB the images
Expected: Image diff should FAIL as the expected colors are different
Actual: Image difference is PASSED.

Please let me know how do I enable inclusion of Color in image diff?

image

image

I have overridden below Image Diff configuration let me know if this is the issue:

//<cypress-root>/support/commands.js

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

addMatchImageSnapshotCommand({
  failureThreshold: 0.9, // threshold for entire image
  failureThresholdType: 'percent', // percent of image or number of pixels
});

Deprecation warning when installing cypress-image-snapshot in a clean project

After installing cypress-image-snapshot in a clean project, I get the following deprecation warning:

npm WARN deprecated [email protected]: cross-spawn no longer requires a build toolchain, use it instead

This warning is coming from [email protected]'s dependency iterm2-version dependency.

I've added a similar issue upstream, which is already fixed.

Can the term-img dependency be updated please?

Thanks for all your great work @jaredpalmer! ๐Ÿ™‚

Snapshots for running all specs don't use individual spec snapshots

Hi!

I haven't had a chance to investigate this super deeply.

But if you run all the tests specs together from the cypress open ui - cypress-image-snapshot is generating snapshots under an All Specs folder, rather than using the snapshots stored under each respective spec's folder.

From a quick glance in the code this isn't intentional, and I can see a related issue:

#17

Is it possible that the same issue is reoffending in later versions?

Versions:
Cypress: 3.1.2
cypress-image-snapshot: 2.3.5

Screenshot fullpage

Cypress support full page screenshot now. (Implement it by scrolling to the top, then taking a screenshot, scrolling, taking a screenshot, and so on, then stitching them together.)

Is there anyway we are able to compare full page screenshot for regression?
Refer: cypress-io/cypress#1424

Example fails to find image

I just tried the example repo using the latest Cypress (3.0.2) on my MacBook and it fails with the following error:

Error: ENOENT: no such file or directory, open '/Users/rob/Projects/cypress-image-snapshot/examples/cypress/screenshots/app-should-visit-the-app.png'

On my Mac the screenshot is actually located in:

/Users/rob/Projects/cypress-image-snapshot/examples/cypress/screenshots/All Specs/app-should-visit-the-app.png

So it looks like the plugin is expecting the file to be in a different location. To reproduce simply follow the instructions for the example:

git clone https://github.com/palmerhq/cypress-image-snapshot.git
cd cypress-image-snapshot/examples
npm install
npm run examples

BDD-like .not

Hey! I've just realised something that might me interesting.

I'm working in a very specific visual effect so i've decided that i want to test that screenshots actually do not match. Apperantaley when I do this

  it('police', () => {
    cy.get('[data-testid=police]').not.matchImageSnapshot();
  });

i'm getting this error in cypress runner

TypeError: cy.get(...).not.matchImageSnapshot is not a function

Workaround

However, we still can do "not" with this syntax

  it('police', () => {
    cy.get('[data-testid=police]').not('matchImageSnapshot');
  });

Also i've noticed that when i use .not snapshots are not being captured.

Screenshot 2019-06-09 at 20 36 08

Do you want me to dive into this problem and prob. submit a PR?

OOM errors from Electron child process when running screenshot tests

I'm consistently getting out-of-memory errors when trying to run screenshot tests with cypress-image-snapshot.

The stack trace is quite busy, but it looks like something deep within this library's actions on after:screenshot is causing this...

It should be noted that I'm diffing full-page screenshots, which weigh in at 1 and 3.4MB, though this error is happening with the 1MB test case and was not happening when I only had the 3.4MB test case. It's possible that I'm plainly pushing Cypress or this library too hard ๐Ÿ˜„

I'm doing this in an open-source project, so you can see my entire setup here: https://github.com/swagger-api/swagger-ui/compare/regression/schemes-servers-rendering?expand=1


Output:

  Running: screenshot/openapi3-petstore-expanded.js...                                     (5 of 6)


  OpenAPI 3 Petstore Expanded Screenshots

<--- Last few GCs --->

[16880:0x7ffb58032e00]    69441 ms: Scavenge 11.8 (17.0) -> 10.8 (17.0) MB, 0.0 / 0.0 ms  allocation failure
[16880:0x7ffb58032e00]    69445 ms: Scavenge 11.8 (17.0) -> 10.8 (17.0) MB, 0.0 / 0.0 ms  allocation failure
[16880:0x7ffb58032e00]    69852 ms: Mark-sweep 139.2 (144.6) -> 136.6 (143.1) MB, 0.7 / 0.0 ms  (+ 6.5 ms in 7 steps since start of marking, biggest step 2.1 ms, walltime since start of marking 406 ms) finalize incremental marking via stack guard GC in ol

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1d50276ad681 <JSObject>
    1: toJSON [buffer.js:~931] [pc=0x3c6d7b026bdd](this=0x473c10823b9 <Uint8Array map = 0x224d9534ffc1>)
    2: arguments adaptor frame: 1->0
    5: builtin exit frame: stringify(this=0x1d5027693ec9 <JSON map = 0x224d95306249>,0x29d5fd282311 <undefined>,0x473c1083cc1 <JSFunction JSON.stringify (sfi = 0x3f0a3617dd19)>,0x473c1083c99 <Object map = 0x2f41d02c34f1>)

    6: arguments adaptor frame: 2...

FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 2: node::FatalError(char const*, char const*) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 3: v8::internal::FatalProcessOutOfMemory(char const*) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 4: v8::internal::FatalProcessOutOfMemory(char const*) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 5: v8::internal::JSArrayBuffer::Setup(v8::internal::Handle<v8::internal::JSArrayBuffer>, v8::internal::Isolate*, bool, void*, unsigned long, v8::internal::SharedFlag) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 6: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 7: v8::internal::EhFrameIterator::DecodeSLeb128(unsigned char const*, int*) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 8: v8::internal::HeapNumber::HeapNumberPrint(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 9: v8::internal::RegisterConfiguration::AreAliases(v8::internal::MachineRepresentation, int, v8::internal::MachineRepresentation, int) const [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
10: v8::internal::OldSpaces::next() [/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
11: 0x3c6d7ae843fd

Error: The following error was thrown by a plugin. We've stopped running your tests because a plugin crashed.

Error [ERR_IPC_CHANNEL_CLOSED]: channel closed
    at ChildProcess.target.send (internal/child_process.js:584:16)
    at Object.send (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:34:27)
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:74:28
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:77:16
    at Promise.cancellationExecute [as _execute] (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/debuggability.js:321:9)
    at Promise._resolveFromExecutor (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:481:18)
    at new Promise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:77:14)
    at Object.wrapParentPromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:61:14)
    at Object.after:screenshot (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:67:27)
    at Object.execute (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:115:38)
    at Object.afterScreenshot (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/screenshots.js:411:22)
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/automation/screenshot.js:16:32
    at tryCatcher (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:636:18)
    at Promise._resolveCallback (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:431:57)
    at Promise._settlePromiseFromHandler (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:522:17)
    at Promise._settlePromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:636:18)
    at Promise._resolveCallback (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:431:57)
    at Promise._settlePromiseFromHandler (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:522:17)
    at Promise._settlePromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:636:18)
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/nodeback.js:42:21
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/graceful-fs/polyfills.js:287:18
    at FSReqWrap.oncomplete (fs.js:153:5)

    at Object.get (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/errors.js:220:11)
    at ChildProcess.handleError (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:92:24)
    at emitOne (events.js:115:13)
    at ChildProcess.emit (events.js:210:7)
    at ChildProcess.target.send (internal/child_process.js:588:12)
    at Object.send (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:34:27)
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:74:28
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:77:16
    at Promise.cancellationExecute [as _execute] (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/debuggability.js:321:9)
    at Promise._resolveFromExecutor (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:481:18)
    at new Promise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:77:14)
    at Object.wrapParentPromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:61:14)
    at Object.after:screenshot (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:67:27)
    at Object.execute (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:115:38)
    at Object.afterScreenshot (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/screenshots.js:411:22)
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/automation/screenshot.js:16:32
    at tryCatcher (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:636:18)
    at Promise._resolveCallback (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:431:57)
    at Promise._settlePromiseFromHandler (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:522:17)
    at Promise._settlePromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:636:18)
    at Promise._resolveCallback (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:431:57)
    at Promise._settlePromiseFromHandler (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:522:17)
    at Promise._settlePromise (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:636:18)
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/nodeback.js:42:21
    at /Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/graceful-fs/polyfills.js:287:18
    at FSReqWrap.oncomplete (fs.js:153:5)


  (Results)

  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚ Tests:        0                                        โ”‚
  โ”‚ Passing:      0                                        โ”‚
  โ”‚ Failing:      1                                        โ”‚
  โ”‚ Pending:      0                                        โ”‚
  โ”‚ Skipped:      0                                        โ”‚
  โ”‚ Screenshots:  0                                        โ”‚
  โ”‚ Video:        false                                    โ”‚
  โ”‚ Duration:     0 seconds                                โ”‚
  โ”‚ Spec Ran:     screenshot/openapi3-petstore-expanded.js โ”‚
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜


โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€

  Running: screenshot/swagger2-petstore.js...                                              (6 of 6)

Error: The following error was thrown by a plugin. We've stopped running your tests because a plugin crashed.

Error [ERR_IPC_CHANNEL_CLOSED]: channel closed
    at ChildProcess.target.send (internal/child_process.js:584:16)
    at Object.send (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:34:27)
    at EventEmitter.<anonymous> (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/preprocessor.js:60:18)
    at emitOne (events.js:120:20)
    at EventEmitter.emit (events.js:210:7)
    at Object.removeFile (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/preprocessor.js:108:19)
    at Object.options.onBrowserClose (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/open_project.js:98:30)
    at EventEmitter.<anonymous> (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/browsers/index.js:111:21)
    at Object.onceWrapper (events.js:314:30)
    at emitNone (events.js:110:20)
    at EventEmitter.emit (events.js:207:7)
    at BrowserWindow.<anonymous> (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/browsers/electron.js:247:29)
    at Object.onceWrapper (events.js:316:30)
    at emitOne (events.js:120:20)
    at BrowserWindow.emit (events.js:210:7)

    at Object.get (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/errors.js:220:11)
    at ChildProcess.handleError (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/index.js:92:24)
    at emitOne (events.js:115:13)
    at ChildProcess.emit (events.js:210:7)
    at ChildProcess.target.send (internal/child_process.js:588:12)
    at Object.send (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:34:27)
    at EventEmitter.<anonymous> (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/preprocessor.js:60:18)
    at emitOne (events.js:120:20)
    at EventEmitter.emit (events.js:210:7)
    at Object.removeFile (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/preprocessor.js:108:19)
    at Object.options.onBrowserClose (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/open_project.js:98:30)
    at EventEmitter.<anonymous> (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/browsers/index.js:111:21)
    at Object.onceWrapper (events.js:314:30)
    at emitNone (events.js:110:20)
    at EventEmitter.emit (events.js:207:7)
    at BrowserWindow.<anonymous> (/Users/kyle/Library/Caches/Cypress/3.1.0/Cypress.app/Contents/Resources/app/packages/server/lib/browsers/electron.js:247:29)
    at Object.onceWrapper (events.js:316:30)
    at emitOne (events.js:120:20)
    at BrowserWindow.emit (events.js:210:7)


  (Results)

  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚ Tests:        0                               โ”‚
  โ”‚ Passing:      0                               โ”‚
  โ”‚ Failing:      1                               โ”‚
  โ”‚ Pending:      0                               โ”‚
  โ”‚ Skipped:      0                               โ”‚
  โ”‚ Screenshots:  0                               โ”‚
  โ”‚ Video:        false                           โ”‚
  โ”‚ Duration:     0 seconds                       โ”‚
  โ”‚ Spec Ran:     screenshot/swagger2-petstore.js โ”‚
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜


====================================================================================================

  (Run Finished)


      Spec                                                Tests  Passing  Failing  Pending  Skipped
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
  โ”‚ โœ” bugs/4838.js                              00:13        1        1        -        -        - โ”‚
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
  โ”‚ โœ” bugs/4867.js                              00:02        1        1        -        -        - โ”‚
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
  โ”‚ โœ” bugs/editor-1868.js                       00:03        1        1        -        -        - โ”‚
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
  โ”‚ โœ” deep-linking.js                           00:18       20       20        -        -        - โ”‚
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
  โ”‚ โœ– screenshot/openapi3-petstore-expandeโ€ฆ       0ms        -        -        1        -        - โ”‚
  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
  โ”‚ โœ– screenshot/swagger2-petstore.js             0ms        -        -        1        -        - โ”‚
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
    2 of 6 failed (33%)                         00:37       23       23        2        -        -

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] test-e2e-cypress: `cypress run`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the [email protected] test-e2e-cypress script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/kyle/.npm/_logs/2018-10-02T01_16_23_812Z-debug.log
ERROR: "test-e2e-cypress" exited with 2.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] e2e-cypress: `run-p -r hot-e2e-cypress-server mock-api test-e2e-cypress`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] e2e-cypress script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/kyle/.npm/_logs/2018-10-02T01_16_23_967Z-debug.log

Duplicated bottom of image

Hi!

When taking snapshots, it looks like the bottom of the image of the screenshots are duplicated. Here an example:
image
There should be just 3 buttons at the bottom of the page, not 6: the other 3 are duplicated. The same is happening to me in all screenshots I take.

Why is this happening?

Thanks!

False positives by font rendering differences

Thanks for this great tool. I really enjoy using it. However over time we noticed that font rendering is causing a lot of false positives on our end.

Is there a known way to solve this?

Thanks!

Image size different when browser dev tools are open

I take my snapshot with a fixed viewport size:

cy.viewport(1200, 800);
cy.get('[data-cy=canvas-preview]').matchImageSnapshot();

With the dev tools closed it's fine. But when I run the test with them open it fails

Seems the dev tools is eating up some of the space...

(docked to the right)
Error: Image size (1150x1009) different than saved snapshot size (1070x1009).

(docked to the bottom)
Error: Image size (1150x1009) different than saved snapshot size (1150x655).

snapshots recorded during `cypress run` mismatch `cypress open`

To reproduce:

  1. write in a test cy.matchImageSnapshot()
  2. run $ node_modules/.bin/cypress run
  3. run again $ node_modules/.bin/cypress run - test passes
  4. run $ node_modules/.bin/cypress open
  5. click on the aforementioned test - test fails

Starting with cypress open and then following with cypress run also gives an error.

This might be a bug/inconsistency on cypress's end, so let me know if I should open it there as well.

Many thanks for this awesome tool!

TS and Cypress compatibility issues

Having difficulty getting this working with the typescript variant of cypress. Also looks like the new version of cypress doesn't like you wrapping commands as the example errors out on run. Main problem I am getting is that there are no type defs and cypress is saying the snapshotmatching commands are not chainable off the cy namespace.

Error when specify customSnapshotsDir outside cypress folder

When i run my test i dont want to save any screenshots on the agent machine i want to save these images to custom directory when i try to do this i am getting this error RangeError: Maximum call stack size exceeded. this works fine if i give a folder path inside cypress directory like this cypress\my folder

Can we actually save screenshot files to a different folder path than the cypress directory?
Please this may be the format issue - can please help me on this -Thanks

addMatchImageSnapshotCommand(

{
 failureThreshold: 0.00, // threshold for entire image
 failureThresholdType: 'percent', // percent of image or number of pixels
 customDiffConfig: { threshold: 0.0 }, // threshold for each pixel
 capture: 'viewport', // capture viewport in screenshot
 customSnapshotsDir:'C:\foldername',
 customDiffDir:'C:\foldername'
 }
);

screenshot problem under linux

I run the test under MacOS and took a screenshot. However, when I run the same test on Linux(cypress docker), the screenshot comparison failed. I see the comparison result, probably because of the problem of screen resolution.

Explore using Resemble.js for image diffing

I recently came across Resemble.js (https://github.com/rsmbl/Resemble.js) and noticed it offers a few advantages over pixelmatch, namely:

  • Can run image diffing in the browser, removing the need for Cypress browser <-> server communication overhead
  • Tuned for image diffing rather than pixel matching, allowing for less false positives

I think we should explore using Resemble.js for image diffing and see if it would be worth the breaking change.

Can't get the plugin working with Typescript

Hi,
I already worked through the suggestions in #13 however I still can't get the plugin working.

We are using TypeScript with WebPack.
I also installed @types/jest-image-snapshot

I've added a file index.d.ts in my <"root dir">/types/cypress-image-snapshot folder:

index.d.ts:

import Cypress from 'cypress';
import { MatchImageSnapshotOptions } from 'jest-image-snapshot';

declare global {
  namespace Cypress {
    interface MatchImageSnapshotOptions extends MatchImageSnapshotOptions, Partial<ScreenshotOptions> {}

    interface Chainable {
      matchImageSnapshot(options?: MatchImageSnapshotOptions): void;
      matchImageSnapshot(fileName: string, options?: MatchImageSnapshotOptions): void;
    }
  }
}

Then pointing to the file in tsconfig.json:

tsconfig.json

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es5",
    "module": "commonjs",
    "strict": true,
    "types": [
      "cypress"
    ]
  },
  "include": [
    // spec files
    "cypress/*/*.ts",
    "tests/**/*.ts"
  ],
  "files": [
    "types/cypress-image-snapshot/index.d.ts"
  ]
}

In <"rootDir">/cypress/plugins/index.js I added the required lines for this plugin:

const wp = require('@cypress/webpack-preprocessor');
const {
  addMatchImageSnapshotPlugin,
} = require('cypress-image-snapshot/plugin');


module.exports = (on, config) => {
    const options = {
        webpackOptions: require('../../webpack.config'),
    };
    on('file:preprocessor', wp(options));
    addMatchImageSnapshotPlugin(on, config);

    return getConfigurationByEnvironment(config)
};

So far so good, but when I add in /cypress/support/commands.ts (not commands.js) :


 import { addMatchImageSnapshotCommand } from '../../node_modules/cypress-image-snapshot/command';

addMatchImageSnapshotCommand();

I get an error saying that no declaration for the module "../../node_modules/cypress-image-snapshot/command" could be found and I should try to install @types/cypress-image-snapshot (which does not exist) or that I should add a declaration file (.d.ts), which contains "declare module 'cypress-image-snapshot';"

And in my test specs I can add a cy.matchImageSnapshot command and I do not get a syntax-error in the spec, but when I run cypress I get an error:

TypeError: cy.matchImageSnapshot is not a function

Invalid plugins file error

Cypress logs TypeError: Path must be a string. Received null and is unable to start the server because The plugins file is missing or invalid.

I have:

plugins/index.js:

const {
  addMatchImageSnapshotPlugin
} = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on);
}

(If I comment out the above, Cypress at least displays the available integration tests, so the problem must be with something above.)

support/commands.js:

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

addMatchImageSnapshotCommand();

node_packages/ and package.json is in cypress/, is there anything else I need to do to set up the plugin?

I ran:

cd cypress/
npm init
npm install --save-dev cypress-image-snapshot
npm install cypress jest # because of unmet peer dependencies

cachePath does not exists

cachePath uses static path cypress/.snapshot-report.
it's worked good if you run tests from root of project.
But we run tests from subproject in cypress folder,
path cypress/cypress... does not exists and tests fails.
This is a reason for disable reporter =(

Can you make this path more configurable from options?

Don't fail until the end

It'd be great to have an option to not immediately throw an error when the snapshot is different.

It's a slow development cycle to run a bunch of tests and then hit one issue, fix it, then repeat.

Cannot find module "cypress-image-snapshot/command" error.

I am getting this current issue whenever I try to run any of my tests in Cypress.
image

This is my current configuration in "cypress/plugins/index.js"

const {
  addMatchImageSnapshotPlugin,
} = require('cypress-image-snapshot/plugin');

const browserify = require('@cypress/browserify-preprocessor');

module.exports = on => {
  addMatchImageSnapshotPlugin(on);
};

module.exports = (on, config) => {
  // `on` is used to hook into various events Cypress emits
  // `config` is the resolved Cypress config
  const options = browserify.defaultOptions;
  options.browserifyOptions.bundleExternal = false;
  // options.browserifyOptions.basedir = __dirname;
  // options.browserifyOptions.paths = ["./node_modules/", path.resolve(__dirname, "../../Bundles/")];
  // options.browserifyOptions.noParse = ["C:\\Users\\MWalker\\Workspace\\SStream-Client\\lib\\3.0\\Bundles\\Core.js"];
  on("file:preprocessor", browserify(options));
};

I have added import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command'; in commands.js and still encountering this issue.

Inconsistent snapshot directory for `cypress run` vs. `cypress open`

Hi!

I found out that snapshots are saved in different directories depending on if they are created using cypress run or cypress open.

Steps to reproduce

  1. Delete the examples/snapshot directory
  2. Run cypress run
  3. Run crypress open and click on Run all specs

What happens is that the same call to matchImageSnapshot produces two different snapshot files:

screen shot 2018-07-19 at 17 37 38

I stumbled open this because I made a change and expected the tests to fail but since there was a new file created everything was still green. I would expect that the previous snapshot should have been used to match against the new screenshot.

Is there something we can do here?

cy.screenshot() timeout error while running cypress with `cypress-image-snapshot`

The package looks great and it helped in automating my testing process well. Thanks for this ๐Ÿ™‚
I'm using this package with vuejs.

Few errors I encounter often are,

  • CypressError: cy.screenshot() timed out waiting '30000ms' to complete. while running cypress test with cy.matchImageSnapshot('home') even though if there is no diff in image.
  • Inconsistent screenshot folder (<rootFolder>/screenshots, <rootFolder>/tests/e2e/screenshots)
  • Inconsistent file names (home.snap.png, home-snap.png)
  • When there is a actual diff in output, cypress doesn't throw the specific error (Image mismatch). Instead, it throws the above said timeout error.

Can't find existing snapshots

Hi! Me again.

I haven't had a chance to dig in as deeply as I'd like to yet but for some reason it can't find existing snapshots and thinks it's doing a new run every time. It doesn't override the existing snapshot -it just creates a new one and never does any diff.

I.e
cy.matchImageSnapshot('potato')

for a spec named cheese.spec.js

results in

screenshots/cheese.spec.js/potato.png
snapshots/cheese.spec.js/potato.snap.png

on the first go round.

And then second go round it generates:

screenshots/cheese.spec.js/potato (1).png
snapshots/cheese.spec.js/potato (1).snap.png

and does no diffing.

I've also set up mocha-multi-reporters but no luck with the reporting.
(Maybe because it's not doing any diffing so it has nothing to report)

Options are standard (from README) - and commands/plugins are set up as discussed earlier.

Element screenshot not working if inside cy.get().within()

Hi, I have this code.

it('Favorite icon button gets filled on click', function() {
  cy.get(`[data-testid="example-variant-buttons"]`).within(() => {
    cy.get(`[title="Add to Favorites"]`).click();
    cy.get(`[title="Added to Favorites"]`).matchImageSnapshot();
  });
});

Currently instead of taking the screenshot of just the Added to Favorites button, it taskes the screenshot of the entire DOM inside example-variant-buttons.

Custom directories

I couldn't find the required info so opening this issue. Some questions, Is there a way to achieve this:

  1. Specify a custom path where the .snap.png files are stored?
  2. Specify a custom path to store the __diff_output__ ?

Basically move from <rootDir>/cypress/snapshots to some custom path. And the path for the __diff_output__ can/should be different than the snapshots directory.

Error: Screenshot was 0% different from saved snapshot with 0 different pixels.

In CI: https://travis-ci.org/OperationCode/front-end/builds/531865502#L1708

My custom commands:

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

Cypress.Commands.add('visitAndWaitFor', path => {
  cy.visit(path);
  cy.wait(3000); // eslint-disable-line cypress/no-unnecessary-waiting
  cy.url().should('contain', path);
});

addMatchImageSnapshotCommand({
  failureThreshold: 0.1, // and tried with `0.05`
  failureThresholdType: 'percent',
  customDiffConfig: { threshold: 2.25 },
  capture: 'fullPage',
});

Cypress.Commands.add('setResolution', size => {
  if (Cypress._.isArray(size)) {
    cy.viewport(size[0], size[1]);
  } else {
    cy.viewport(size);
  }
});

// I use this when I want to make a snapshot test
Cypress.Commands.add('createVisualRegressionTests', () => {
  cy.get('[data-testid="Not Mobile Nav"'); // wait for resize listener
  cy.matchImageSnapshot('desktop');
});

Element screenshot not working

Hey!

First of all, thanks a lot for this project! โค๏ธ I'm very new to cypress so maybe I missed something but I think element snapshots are broken with the latest version.

I quickly made some changes to the example folder to illustrate what I mean (I can also submit a PR if you like):

With the following element in App.js:

<div
  style={{ display: 'inline-block', backgroundColor: 'orange' }}
  data-cy="foo"
>
  Hello, world!
</div>

and the following test file:

describe('App', () => {
  it('should visit the app', () => {
    cy.visit('http://localhost:3000');

    cy.get("[data-cy='foo']").screenshot();

    cy.get("[data-cy='foo']").matchImageSnapshot();
  });
});

the call to .screenshot() produces this image:

app -- should visit the app

but .matchImageSnapshot() produces this:

should-visit-the-app

Is this intended behavior?

Unexpected exit code when running `cypress run` (0, even with test failures).

Before I open a bug in Cypress itself, could someone verify if the exit code for failed snapshot tests return the correct exit code?

Should be greater than 0, but it's being set as 0, which makes tests in CI appear as passed, when they really failed, since CI tools rely on exit codes to determine the success of a given step.

image

Has someone else experienced this issue? I don't know if this plugin uses the Cypress Module API or not, but could it be possible that the exit code isn't being set correctly, as it happened to this person? cypress-io/cypress#2816

cy.task is not a function

I've followed the instructions from the readme, but I'm getting the following.

cypress_runner.js:139349 TypeError: cy.task is not a function
    at Context.matchImageSnapshot (http://localhost:8081/__cypress/tests?p=tests/e2e/support/index.js-677:26:8)
From previous event:
    at runCommand (http://localhost:8081/__cypress/runner/cypress_runner.js:63043:14)
    at next (http://localhost:8081/__cypress/runner/cypress_runner.js:63125:14)
From previous event:
    at http://localhost:8081/__cypress/runner/cypress_runner.js:63144:37
From previous event:
    at run (http://localhost:8081/__cypress/runner/cypress_runner.js:63142:15)
    at Object.cy.(anonymous function) [as matchImageSnapshot] (http://localhost:8081/__cypress/runner/cypress_runner.js:63362:11)
    at Context.runnable.fn (http://localhost:8081/__cypress/runner/cypress_runner.js:63493:20)
    at callFn (http://localhost:8081/__cypress/runner/cypress_runner.js:33042:21)
    at Test.Runnable.run (http://localhost:8081/__cypress/runner/cypress_runner.js:33035:7)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:66382:28
From previous event:
    at Object.onRunnableRun (http://localhost:8081/__cypress/runner/cypress_runner.js:66381:20)
    at $Cypress.action (http://localhost:8081/__cypress/runner/cypress_runner.js:61845:51)
    at Test.Runnable.run (http://localhost:8081/__cypress/runner/cypress_runner.js:65521:20)
    at Runner.runTest (http://localhost:8081/__cypress/runner/cypress_runner.js:33505:10)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:33611:12
    at next (http://localhost:8081/__cypress/runner/cypress_runner.js:33425:14)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:33435:7
    at next (http://localhost:8081/__cypress/runner/cypress_runner.js:33367:14)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:33403:5
    at timeslice (http://localhost:8081/__cypress/runner/cypress_runner.js:28644:27)

I'm using Vue's CLI and its plugin for cypress, if that's of any help.

Here are my complete files which I think are relevant. The project is private so I cannot share it, but I can try reproducing later if needed -- thought it might be obvious from the stack trace to the maintainers, so wanted to avoid setting everything up.

plugins/index.js

const { addMatchImageSnapshotPlugin } = require('cypress-image-snapshot/plugin')
module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on)
  return Object.assign({}, config, {
    fixturesFolder: 'tests/e2e/fixtures',
    integrationFolder: 'tests/e2e/specs',
    screenshotsFolder: 'tests/e2e/screenshots',
    videosFolder: 'tests/e2e/videos',
    supportFile: 'tests/e2e/support/index.js'
  })
}

(The return and Object.assign were there form Vue's scaffolding).

support/commands.js

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command'
addMatchImageSnapshotCommand()

(This file was previously empty from Vue's scaffolding)

specs/public-pages.js

describe(`Public pages`, () => {
    before(() => {
        cy.visit('/') })

    describe(`Login page`, () => {
        it(`is the first page user sees when not logged in`, () => {
            cy.url().should('contain', '/login') })
        it(`looks as per design`, () => {
            cy.matchImageSnapshot() })
        it(`should have a link to Forgot password page`, () => {
            cy.get('a[href="/forgot-password"]')
                .should('exist')
                .contains(`Forgot Password?`)
                .click() }) })

    describe(`Forgot Password page`, () => {
        it(`opens after clicking the link from Login`, () => {
            cy.url().should('contain', '/forgot-password') })
        it(`looks as per design`, () => {
            cy.matchImageSnapshot() }) }) })

And here are the results:

image

image

Unexpected image size differences

Hi! I posted an issue earlier this morning #79 which you resolved by having a more helpful error message.

Error: Image size (1280x1162) different than saved snapshot size (1280x1161).

I saw #67, but I'm unsure if it's related, as I'm using cypress run both locally and in CI. Feel to close, and I'll copy-paste my input there.

I have no idea how I'm getting a 1 pixel difference on a static page, especially since the other static pages DONT have this difference. Do you have any ideas?

Screenshot was NaN% different from saved snapshot with undefined different pixels.

Hey!

This is very very cool. Nice work :)

I just gave it a crack in our project but getting the following:

Screenshot was NaN% different from saved snapshot with undefined different pixels.
  See diff for details: undefined

screen shot 2018-11-15 at 12 12 41 pm

It's saving the screenshot correctly but haven't got a run to pass yet with/without a saved image available in the screenshots dir.

Is there anything special I have to do on first run? I've tried with both suggested env options

updateSnapshots=true failOnSnapshotDiff=false but no luck getting the test to pass.

I also followed the suggested instructions,

and have tried the following options in my commands:

addMatchImageSnapshotCommand({
  capture: 'viewport', // capture viewport in screenshot
})
addMatchImageSnapshotCommand({
  failureThreshold: 0.03, // threshold for entire image
  failureThresholdType: 'percent', // percent of image or number of pixels
  customDiffConfig: { threshold: 0.1 }, // threshold for each pixel
  capture: 'viewport', // capture viewport in screenshot
})

Thanks for any help you can provide.

Snapshots sometimes captured at smaller size

I'm sometimes seeing failed tests where the snapshots look to be the same, but at different resolutions.

Has anyone else experienced this?

playback at time 0 5 diff

index.html

    <canvas id="canvas" width="1920" height="1080" style="width: 500px;"></canvas>

test.spec.js

// ...  code to render image on canvas. 
cy.get("#canvas").matchImageSnapshot()

Match cy.screenshot() filename behaviour

https://docs.cypress.io/api/commands/screenshot.html#Naming-conventions

Newest cypress versions (3.0.2+) allow defining a nested screenshot filename like cy.screenshot('dir1/dir2/foobar) which automatically creates the dir1 directory and its dir2 subdirectory, storing foobar.png inside it. This pretty convenient e.g. when doing screenshots for mobile/dashboard and desktop/dashboard.

This plugin has its own naming scheme (test.title + name in kebab-case) and does not support this syntax for nested folders, it would be nice to match the default cypress behaviour :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.