cypress-io/testing-workshop-cypress

End-to-end testing workshop with Cypress

testing-workshop-cypress

A full day testing workshop complete with application, exercise tests and speaker slides for Cypress.io

Requirements

• Any computer: Mac, Windows, Linux
• Node 12.0.0+ (LTS), check GH workflow file
• git

In order to get the code and install dependencies

git clone [email protected]:cypress-io/testing-workshop-cypress.git
cd testing-workshop-cypress
npm install

If necessary, install dependencies inside TodoMVC folder

cd todomvc
npm install

Quick check ✅

You can test the installation by starting TodoMVC in the first terminal window

cd todomvc
npm start

and you should see in the terminal

> json-server --static . data.json --middlewares ./node_modules/json-server-reset


  \{^_^}/ hi!

  Loading data.json
  Loading ./node_modules/json-server-reset
  Done

  Resources
  http://localhost:3000/todos

  Home
  http://localhost:3000

Note for Windows users: if npm start throws an error, it is probably due to cd todomvc; ... first command in the npm start script. In this case change the working folder to "todomvc" and run npm start from there.

From the second terminal window you should be able to open Cypress in the root of the project with

$ npm run cy:open

> [email protected] cy:open /git/testing-workshop-cypress
> cypress open

Tip

You can use the installed start-server-and-test utility to start the app, open Cypress and then shutdown the app when you exit Cypress.

$ npm run dev

Application 💾

Vue.js + Vuex + REST server application that we are going to test is in the folder todomvc. This application and its full testing is described in this blog post. The application should run fine without network access.

Slides 🖥

See the presentation at https://testing-workshop-cypress.netlify.app/. Every section of the presentation has a subfolder in the slides folder with a Markdown file. The Markdown is rendered into HTML using Vite and Reveal.js combination. You can open the presentation by clicking on "link" in the table below.

Content 🗂

Beginner

	topic	folder	contents	slides
🔗	Introduction, TodoMVC application	todomvc	intro.md	link
🔗	Loading page	00-start	00-start	link
🔗	cypress open vs cypress run	01-basic	01-basic	link
🔗	Adding items test, cypress.json file	02-adding-items	02-adding-items	link
🔗	Selector Playground	03-selector-playground	03-selector-playground	link
🔗	Reset database using cy.request	04-reset-state	04-reset-state	link
🔗	Spy and stub XHR requests, fixtures	05-xhr	05-xhr	link
🔗	Access application code and data	06-app-data-store	06-app-data-store	link

Intermediate

	topic	folder	contents	slides
🔗	Setting up E2E tests on CI	07-ci	07-ci	link
🔗	Setting up Cypress Dashboard	07-ci	08-dashboard	link
🔗	Test reporters	-	09-reporters	link
🔗	Configuration	-	10-configuration	link
🔗	Retry-ability	11-retry-ability	11-retry-ability	link
🔗	Custom commands	12-custom-commands	12-custom-commands	link

Advanced

	topic	folder	contents	slides
🔗	Page Objects vs App Actions	13-app-actions	13-app-actions	link
🔗	Fixtures	14-fixtures	14-fixtures	link
🔗	Debugging	02-adding-items/demo.js	15-debugging	link
🔗	Preprocessors	16-preprocessors	16-preprocessors	link
🔗	Component testing	17-component-testing	17-component-testing	link
🔗	Backend code	18-backend	18-backend	link
🔗	Code coverage	19-code-coverage	19-code-coverage	link
🔗	Stubbing methods	20-stubbing	20-stubbing	link
	The end	-	end	link

For speakers 🎙

This workshop can take all day, but you can pick the sections you are interested in teaching at will and customize it into any time duration. Everyone is coding for the most part, except for CI and the Cypress Dashboard sections, where the usage was shown via slides and actual sites.

During the workshop, keep the todomvc app running in one shell, while each section 01-basic, 02-..., 03-... etc. has its own Cypress and specs subfolders cypress/integration/.... Usually a spec has several tests with placeholder comments. The workshop attendees are expected to make the tests pass using the knowledge from the slides and hints (and Cypress documentation). Note that most folders have a prepared spec.js file and an answer.js file. The answer.js file is ignored by Cypress using a setting in cypress.json.

The only exception is the folder 00-start. This is a folder for students to see how Cypress scaffolds example specs when you open Cypress for the very first time. In this folder students should execute...

cd 00-start
npm run cy:open

...and see the list of created example specs.

The slides are generated using Reveal.js from Markdown sources in the slides folder. You can show the slides locally by running

npm run slides:dev

The slides are deployed to Netlify automatically, see https://testing-workshop-cypress.netlify.app/

Additional information 🗃

• https://www.cypress.io/
• https://docs.cypress.io/
• https://docs.cypress.io/api/api/table-of-contents.html

testing-workshop-cypress's Issues

Add watching data file to show how to reset database using cy.task

json-server --watch ... and to reset the file we can probably just overwrite it using a task

Add section about reporters

• how to use Mocha's built-in reporters
• how to use mocha awesome reporters
• combine multiple reports into one using a separate tool

Add large application assignment

Users from the workshop expressed a desire to include a larger application as a test case and let them figure out how to test it. This would include planning tests, writing them, etc. Probably would be 1-2 hour activity.

can we use correct relative paths in slides?

Currently slides are included from the root pitchme.md file and all image paths are relative. But when GitHub shows each markdown file in slides folder the image links are broken. Maybe there is a way to specify folder for relative paths to have images appear in github and in gitpitch correctly

Make app fully offline

Separate some exercises to have clear objections

go through slides and make sure objectives are clear. Also each spec file should have comments with what to do right in the comments

Add slide on code organization and custom commands

and remove Cypress custom commands from the index slide

Add slide with CircleCI Orb example

In addition to explaining "normal" CI workflow add a slide with CircleCI orb example to show how easy it is to configure. https://github.com/cypress-io/circleci-orb

Update dark theme CSS

On Mac OS Catalina current dark theme looks pretty bad

add visual testing chapter

use Percy, hmm, what kind of exercises would there be?

add code coverage guide

Using https://github.com/cypress-io/cypress-istanbul

Find a better local presentation workflow

Currently we cannot see local slide presentation preview without jumping through lots of hoops. Maybe it would be better to switch from Gitpitch to something friendlier

Ideas

• Vuepress
  • would be nice to generate static site where each page was coming from separate subfolder
  • and from the same static content it would be nice to generate slide presentations
  • some tools to show Vuepress site as a slide show:
    • https://keynote.sh/guide/
    • https://github.com/Xanderite/vue-decks

Full day Cypress workshop ideas

first half - same as before

second half - new content

• retry-ability and should(cb) in #29
  • these are core concepts and we have spent time adding more examples to the kitchensink and documentation sites
  • https://on.cypress.io/should
  • bringing additional assertion libraries
• Cypress & Jest (unit testing & component testing & e2e testing)
  • configuring Cypress when Jest is installed (clashing global methods like expect)
  • react-testing-library and cypress-testing-library
• Custom commands in #30
  • how to write them, why to write the
  • including TypeScript definitions like example https://github.com/cypress-io/cypress-example-todomvc#cypress-intellisense
• reporters
  • how to use reporters, save results #25
• Cypress module API
• Cypress configuration: cypress.json, command line flags, environment variables, plugins file #32
• Page objects vs app actions #40
  • based on https://www.cypress.io/blog/2019/01/03/stop-using-page-objects-and-start-using-app-actions/ and https://github.com/bahmutov/test-todomvc-using-app-actions
• GraphQL testing
  • this is becoming a must for E2E
  • probably will need to add GraphQL endpoint
  • https://glebbahmutov.com/blog/e2e-testing-graphql-using-cypress/
• debugging #42
• fixtures and backend with cy.task #43
• bundling with your own browserify or webpack options #47
• unit testing components #48

need to add top level vs child commands, and different types of the parent element

Add to custom commands section

Add GA token to GitPitch config

Add GA tracking to slides to better understand its usage by the community.
Relevant docs: https://gitpitch.com/docs/settings/google-analytics

Split sections into introduction, intermediate, advanced

Mention VS Code in intellisense slide

Might be obvious that intellisense is only available for VS code if you're in a workshop, but not super clear from slides alone.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• Update dependency @cypress/browserify-preprocessor to v3.0.2
• Update dependency cypress-xpath to v1.8.0
• Update dependency eslint-plugin-cypress to v2.12.1
• Update cypress orb to v2
• Update cypress/base Docker tag to v18
• Update dependency cypress to v11
• Update dependency cypress-xpath to v2
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update cypress orb to v1.29.0
• Update cypress/base Docker tag to v12.22.8
• Update dependency cypress to v7.7.0
• Update dependency cypress-dark to v1.8.3
• Update dependency cypress-failed-log to v2.10.0
• Update dependency start-server-and-test to v1.14.0
• Update dependency typescript to v4.9.3
• Update dependency @cypress/code-coverage to v3
• Update dependency cypress-pipe to v2
• Update dependency cypress-react-unit-test to v4
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

circleci

.circleci/config.yml

• cypress 1.28.0
• cypress/base 12.14.0
• cypress/base 12.14.0
• cypress/base 12.14.0
• cypress/base 12.14.0
• cypress/base 12.14.0

github-actions

.github/workflows/main.yml

• cypress-io/github-action v2
• cypress-io/github-action v2

.github/workflows/min-node-version.yml

• cypress-io/github-action v2
• cypress/base 12.0.0-libgbm

.github/workflows/windows.yml

• cypress-io/github-action v2

npm

00-start/package.json

package.json

• @cypress/browserify-preprocessor 3.0.1
• @cypress/code-coverage 1.14.0
• cypress 7.6.0
• cypress-dark 1.7.14
• cypress-failed-log 2.9.1
• cypress-pipe 1.7.0
• cypress-plugin-snapshots 1.4.4
• cypress-react-unit-test 2.10.0
• cypress-xpath 1.6.2
• eslint-plugin-cypress 2.11.3
• start-server-and-test 1.12.1
• typescript 4.3.4

todomvc-redux/package.json

• start-server-and-test 1.12.1

todomvc/package.json

---

• Check this box to trigger a request for Renovate to run again on this repository

check if there is an exercise testing the "loading" state

Slow down network request, maybe with a stub, check what the app is showing while the data is loading

Add section about testing components

Probably show React unit testing

Use full path names when saying "open file ... spec.js"

Put full pathnames to spec files to open in the slides content

rework to have single npm install

• install all dependencies from the root package
• have all specs as subfolders for clarity

Add section on configuration

How to set Cypress config variables: cypress.json, cypress.env.json, command line, environment variables, in code

note: this might change in the future versions of Cypress.

For each workshop presentation start with "what you will learn"

Similar to how our docs start with it

test on Windows

Probably on AppVeyor CI to confirm that paths are all working for Windows users

Add CLA

because outside users are submitting pull requests

Show different between then(cb) and should(cb)

And how if you need result from should(cb) you want to do

command
.should(cb)
.then(... return value)

Add section with assertions and retry-ability

• which commands are retried
• different types of assertions
• writing custom assertions

Matching documentation page cypress-io/cypress-documentation#1314

Section on debugging

From http://on.cypress.io/debugging

Split combined presentation - just have its start

and then let people select particular sections

Writing custom commands

• top-level commands
• child commands
• retries
• our TodoMVC examples
• adding types for custom commands to cy object
• using external commands like cypress-pipe etc
• documentation page http://on.cypress.io/custom-commands

split fixtures from backend

Part "Fixtures and backend" should be separated

TDD approach in workshop tests: make them fail by default

@bahmutov About the discussion we add on Full day Cypress workshop, here is what I would suggest so that workshop tests fail by default, which I find is a better TDD approach:

// create a specific command for failures
Cypress.Commands.add("fail", description => {
  throw new Error(description);
});

// in .spec.js files 
it('starts with zero items (stubbed response)', () => {
  cy.fail('start Cypress network server')
  cy.fail(`stub 'GET /todos' with []
    save the stub as an alias`)

  // THEN visit the page
  cy.visit('/')

  cy.fail(`wait for the route alias
    grab its response body
    and make sure the body is an empty list`)
})

The trainee will be reminded step-by-step what he has to do.
I am not sure about the multi-fail approach, which could make him think he didn't do the expected thing he just jumped to the next fail ; any remark about it ?

Tell me If you consider it's a good idea, so that I extend it to all tests and make a PR.

Make each section individually addressable

so we can point at a particular section like #fixtures

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Preset name not found within published preset config (monorepo:babel6). Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Consider merging example applications

We have 3 TodoMVC applications

• Vue + Vuex + REST api without bundling used for most of the workshop
• TodoMVC React with bundling for showing testing components
• TodoMVC Redux for showing code coverage

We could just update the main Vue application and use it for everything

• split into components, bundle using parcel-json-server to serve the app and have the REST api (@amirrustam )
• use cypress-vue-unit-test to demo component testing (@amirrustam )
• measure code coverage(@bahmutov )

Update the command for starting the app - for Windows users

Current command npm start does cd todomvc; ... which does not work well on Windows. Fix this command to work without cd todomvc part

add example overwriting built-in command

to the section on custom commands

Fix missing images

@bahmutov commented on Sat Jun 30 2018

Some images were in subfolders, that were now merged

like ![First test run](04-reset-state/img/passing-test.png)

Draw attention to how to use selector playground

An arrow or some emphasis on the selector playground button would make selector playground slides clearer, especially when not with a speaker.

Make sure all answers pass

Using command

npx cypress run --spec 'cypress/integration/*/answer.js' \
 --config ignoreTestFiles="",supportFile=cypress/support/reset.js

Currently only the following pass / fail

Go over retry-ability markdown

small fixes are needed

Add section about custom bundlers

Custom browserify
Custom webpack
Writing your own

Show iframe example

Show how to work with an iframe, example https://github.com/forallabeautifulearth/fabe-browser-tests/blob/a0880fdd76a11b3275a4a9885e0584ba9fb28845/cypress/support/utils.js#L5

Add section on fixtures and backend code

• how to use fixtures
• scope and order of execution when loading fixtures
• how to call backend code using cy.task

Consider paying for GitPitch

Since my promotional GitPitch Pro subscription expired after 1 year, each slide deck shows a last slide asking to pay. We could transition to something else for showing slides, or just pay for the GitPitch Pro license.

suggestion: move lesson 4 up

First off I love this repo and this DIY workshop idea. Thank you for putting this together!

I would like to take this opportunity to make a small suggestion I think would improve the comprehension of newcomers like me. Yesterday during lessons 1-3 I was struggling to understand what was happening at first, then realized my data wasn't being reset. This issue came up because I ended up having to run the tests a few times while I experimented with the syntax. This left some tests adding data and others failing due to leftover state. I found the cy.request('POST', '/reset') call and then calling that in a beforeEach (just like lesson 4). Today I got to lesson 4 and I now see that you're assuming in lessons 1-3 people either won't need to run the test more than once because they input everything perfect.

I'd like to suggest moving lesson 4 up to the front and explaining that right away. Otherwise it leaves a bad taste when you have tests failing inexplicably. Thanks for your time Gleb. I really appreciate your efforts here. I hope this ticket helps improve this workshop.

PS: Great docs!

section on plugins

we have a lot of good plugins, would be nice to show off some of them

Page objects vs app actions

Based on the Cypress blog post https://www.cypress.io/blog/2019/01/03/stop-using-page-objects-and-start-using-app-actions/ and code repo https://github.com/bahmutov/test-todomvc-using-app-actions

Test answers

Test most of the answers on CI by overwriting the ignoreTestFiles config from CLI or via an environment variable

Blocked by cypress-io/cypress#1696 that should be out in 3.3.2