argos-ci/argos-javascript

Official Argos SDKs for JavaScript

Argos is a visual testing solution that fits in your workflow to avoid visual regression. Takes screenshots on each commit and be notified if something changes.

Official Argos SDKs for JavaScript

All Argos JavaScript SDK packages are centralized in that mono-repo, usually available on @argos-ci/ namespace. It provides a convenient interface and improves consistency between various JavaScript environments.

Links

• Official SDK Docs
• Discord

Supported environments

For each major JavaScript platform, there is a specific high-level SDK that provides all the tools you need in a single package. Please refer to the README and instructions of those SDKs for more detailed information:

• @argos-ci/cli: interact with and upload screenshots to argos-ci.com via command line
• @argos-ci/core: SDK for Node.js, the base to create other integrations

argos-javascript's Issues

Send pull-request number in @argos-ci/core

Send pull-request number to the server in @argos-ci/core package. It should be collected automatically from environment variables for all CI.

We will use prNumber.

Support GitLab CI

Iframe stabilization

import type { Page } from '@playwright/test';

export async function waitForIFrames(page: Page) {
  await page.evaluate(() => {
    return Promise.all(
      Array.from(document.querySelectorAll('iframe')).map(
        (iframe) =>
          new Promise((resolve) => {
            const checkIframeContent = () => {
              const iframeContent =
                iframe.contentDocument || iframe.contentWindow?.document;

              if (iframeContent?.body?.childNodes?.length) {
                resolve(null);
              } else {
                setTimeout(checkIframeContent, 50); // Poll every 50ms
              }
            };

            checkIframeContent();
          }),
      ),
    );
  });
}

Enhance hover stabilization

The idea is to move the cursor to a specific position to remove any hover effect.

// When an element disappears after we interacted with it (like a save button), an element displayed below can catch a hover event.
// We explicitly hover on the top-left corner of the page to avoid any hover effects in the Argos screenshots.
const removeHoverEffect = async (page: Page) => {
  await page.mouse.move(0, 0);
};

export const stabilizedArgosScreenshot = async (
  page: Page,
  name: string,
  options?: ArgosScreenshotOptions,
) => {
  await removeHoverEffect(page);
  return argosScreenshot(page, name, { fullPage: false, ...options });
};

Stabilization options should be configurable because users may not want that.

Grep option in Playwright breaks Argos upload dir

Ignore zone option

A user need to ignore an area for mobile testing.
As Odiff has a build-in feature for it, it should not be hard to implement.

TO DO

• Add the option in the CLI
• Add a visual mark in Argos SaaS when a screenshot area is ignored
• Document the new option
• Reach out to the client when it's done

ODIFF doc

Send prHeadCommit in `@argos-ci/core`

Problem is: GitHub uses merge commit by default to run CI tests. Argos uses the branch commit to find the reference branch.

What we want is:

Argos uses the merge commit for everything but it displays the branch commit in UI.

The goal is to set the commit from $GITHUB_SHA and set another variable : prHeadCommit from the head of branch commit.

Use GitLab with a Repository Access Token

Connect GitLab projects using repository access tokens.

This will provide users with enhanced control over repository permissions, allowing selective access for Argos rather than using broader personal access tokens.

Create a Github Action to take screenshots from URL

Allow to run `argos upload` without directory (zero-config)

The command argos upload in most cases can be smart and does not require to specify a screenshot folder. Our clients mostly uses Argos integrations to take screenshots. When Argos takes the screenshot, Argos knows where the screenshot is, so argos upload is enough.

To achieve that, the goal is to create a package named : @argos-ci/config in argos-javascript repository.

This package will be used by all Argos integrations (puppeteer, playwright, cypress) to know where to store screenshots.

And it will be used by our CLI to know where to find screenshots to upload.

For the initial release, it is only possible to specify a directory in a file argos.config.js, argos.config.mjs, argos.config.cjs.

/** @type {import('@argos-ci/config').ArgosConfig} */
const argosConfig = {
  screenshotDir: "screenshots"
}
 
export default argosConfig

Modernize suite of tests

Put in common all tests in E2E frameworks and ensure that it covers every aspects of stabilization and every features.

Attempts in the name of the screenshot on Cypress SDK

A client have screenshots with "(attempt n)" in their name, so it creates flakiness (remove / added) and does not compare screenshot. We have to:

• Investigate when does Cypress do that
• Be sure that the screenshot name is consistent if taken with argosScreenshot method

      on('after:screenshot', (details) => {
        // Get the base filename without extension
        const baseName = path.basename(
          details.path,
          path.extname(details.path),
        );

        // Remove attempt from the filename
        const newBaseName = baseName.replace(/ \(attempt \d+\)/, '');

        // Construct a new path with the original file extension
        const newPath = path.join(
          path.dirname(details.path),
          newBaseName + path.extname(details.path),
        );

        return new Promise((resolve, reject) => {
          fs.rename(details.path, newPath, (err) => {
            if (err) {
              return reject(err);
            }

            resolve({ path: newPath });
          });
        });
      });

Remove sticky before fullpage screenshot

Issue

A lot a our users have a issue with sticky navbar that happen when they try to screenshot a fullpage.

1. I suggest that we enforce display: relative on all sticly items before taking a fullpage screenshot by default.

2. I recommend to allow to deactive this behavior with a --allow-sticky option.

ENOENT: no such file or directory

Hi I got this error when trying to run argos cli on Windows 10.

$ npx agos
node:internal/process/esm_loader:94
    internalBinding('errors').triggerUncaughtException(
                              ^

[Error: ENOENT: no such file or directory, open 'C:\C:\Users\user\test-project\node_modules\@argos-ci\cli\package.json'] {  errno: -4058,
  code: 'ENOENT',
  syscall: 'open',
  path: 'C:\\C:\\Users\\user\\test-project\\node_modules\\@argos-ci\\cli\\package.json'
}

Support --token from command line

The message is the same if the token has wrong format or if it is missing, let's be clearer on the missing vs wrong format message.

Build failed: Cannot read property 'map' of undefined

I am running cypress to generate screenshots with cy.argosScreenshot("file name"); on a react project (running on node 14) and when I try to run argos upload cypress/screenshots it fails with an error:

✖ Build failed: Cannot read property 'map' of undefined

can you help please? thanks

Bug: Cypress screenshot does not work

Bug description

Timed out retrying after 4000ms: cy.readFile("node_modules/@argos-ci/browser/dist/global.js") failed because the file does not exist at the following path:
<CODE_PATH>/node_modules/@argos-ci/browser/dist/global.js

Correct behavior expected

cy.argosScreenshot() should work and find any dependent files.

Wrong behavior observed

File dependency is missing or there is an installation step missing from documentation.

Screenshot

Reproduce

Follow https://argos-ci.com/docs/cypress

Setup

Browser: Chrome
Package manager: pnpm 8.9.2
Node: 20.8.0
Cypress: 13.3.2

Add type on argos-cypress argosScreenshot command

Automatic parallel for Playwright

https://playwright.dev/docs/test-sharding

Fix external in argos-javascript

Dependencies are bundled together, this is not good! external in rollup does not work well.

Configurable comparison threshold

We will allow to specify a custom preconfigured threshold value for a specific screenshot in the UI.

• Add a page about what is Argos diffing strategy.

Error: Could not load the "sharp" module using the linux-x64 runtime

Bug description

On my local, I get the following error after taking the screenshot with Argos 2.0.1:

Error: Could not load the "sharp" module using the linux-arm64 runtime
Possible solutions:
- Ensure optional dependencies can be installed:
    npm install --include=optional sharp
    yarn add sharp --ignore-engines
- Ensure your package manager supports multi-platform installation:
    See https://sharp.pixelplumbing.com/install#cross-platform
- Add platform-specific dependencies:
    npm install --os=linux --cpu=arm64 sharp
- Consult the installation documentation:
    See https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (/app/node_modules/sharp/lib/sharp.js:114:9)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Object.require.extensions.<computed> [as .js] (/root/.cache/Cypress/13.8.1/Cypress/resources/app/node_modules/ts-node/dist/index.js:851:20)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1022:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/app/node_modules/sharp/lib/constructor.js:10:1)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Object.require.extensions.<computed> [as .js] (/root/.cache/Cypress/13.8.1/Cypress/resources/app/node_modules/ts-node/dist/index.js:851:20)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1022:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/app/node_modules/sharp/lib/index.js:6:15)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Object.require.extensions.<computed> [as .js] (/root/.cache/Cypress/13.8.1/Cypress/resources/app/node_modules/ts-node/dist/index.js:851:20)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1022:12)
    at cjsLoader (node:internal/modules/esm/translators:366:17)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:315:7)
    at ModuleJob.run (node:internal/modules/esm/module_job:222:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:323:24)
    at async Object.exports.upload (/app/node_modules/@argos-ci/core/dist/index.cjs:3:24)
    at async Object.handler (/app/node_modules/@argos-ci/cypress/dist/task.cjs:27:21)

On CI, I'm getting the following error after taking the screenshot with Argos 2.0.1:

Error: Could not load the "sharp" module using the linux-x64 runtime
Possible solutions:
- Ensure optional dependencies can be installed:
    npm install --include=optional sharp
    yarn add sharp --ignore-engines
- Ensure your package manager supports multi-platform installation:
    See https://sharp.pixelplumbing.com/install#cross-platform
- Add platform-specific dependencies:
    npm install --os=linux --cpu=x64 sharp
- Consult the installation documentation:
    See https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (/app/node_modules/sharp/lib/sharp.js:114:9)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Object.require.extensions.<computed> [as .js] (/root/.cache/Cypress/13.8.1/Cypress/resources/app/node_modules/ts-node/dist/index.js:851:20)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1022:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/app/node_modules/sharp/lib/constructor.js:10:1)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Object.require.extensions.<computed> [as .js] (/root/.cache/Cypress/13.8.1/Cypress/resources/app/node_modules/ts-node/dist/index.js:851:20)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1022:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/app/node_modules/sharp/lib/index.js:6:15)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Object.require.extensions.<computed> [as .js] (/root/.cache/Cypress/13.8.1/Cypress/resources/app/node_modules/ts-node/dist/index.js:851:20)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1022:12)

Correct behavior expected

The sharp library should not fail.

Wrong behavior observed

The sharp library failed to be loaded apparently because of the OS, however, everything looks fine, we are using the official Cypress image and I tried with the official Node Image either.

Reproduce

I'm not sure, but I have assumptions this is related to the incompatibility with Cypress or Node docker images

Setup

OS: MacOS M2 Pro
Docker Desktop Version: 4.29.0
Cypress Official Image: 13.8.1
Argos Version: 2.0.1
Yarn Version: 4.1.1
Node Version: v20.12.2

On CI everything is the same, except the OS, we are using CircleCI and the OS is ubuntu-2204:2024.01.1

Bug: bun install fails on Docker (bun:latest as of 4/5/2024)

Bug description

As a user, when I attempt to bun install (@argos-ci/[email protected] - which is pulled in as part of a shared test automation library) I get the following error due to the "sharp" npm library:

@videate/[email protected]
  └─┬ @argos-ci/[email protected]
    └─┬ @argos-ci/[email protected]
      └── [email protected]

Here is the failure:

bun install v1.1.1 (ca1dbb4e)
0.183  Resolving dependencies
1.240  Resolved, downloaded and extracted [2]
6.449 sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.5/libvips-8.14.5-linux-arm64v8.tar.br
6.449 
6.449 104 |   };
6.449 105 | 
6.449 106 |   stream.pipeline(
6.449 107 |     fs.createReadStream(tarPath),
6.449 108 |     verifyIntegrity(platformAndArch),
6.449 109 |     new zlib.BrotliDecompress(),
6.449           ^
6.449 error: zlib.BrotliDecompress is not implemented

Correct behavior expected

It should install without an error as it's supposedly fixed in bun 1.1

Wrong behavior observed

Docker build failure

Detail

My Dockerfile has: FROM oven/bun:latest as of 4/5/2024

This is part of a company wide move to bun in our ci and across our shared libraries. It would be great if argos could upgrade to the latest version of sharp which I believe includes fixes for this issue (lovell/sharp#3511)

Reproduce

Just include the argos playwright library in package.json and then use bun:latest Dockerfile and attempt a bun install

FROM oven/bun:latest
WORKDIR /usr/src/app

COPY package.json bun.lockb ./
RUN --mount=type=secret,id=bunfig,mode=0444,dst=bunfig.toml bun install

Setup

Additional information

We include argos playwright into a shared automation lib that is imported by a number of repos, all the bun related repos started failing in CI.

Allow to specify custom reference branch & reference commit

• Update core
• Update cli
• Update documentation
• Add a link to reference branch documentation into reference branch section on Settings page

https://docs.percy.io/docs/baseline-picking-logic#overriding-the-default-base-branch