clement-joye/kubernetes-ui-tests

Automation scripting solution for running UI tests in parallel in Kubernetes with Cypress and AKS

Kubernetes-UI-Tests

• Kubernetes-UI-Tests
  • Introduction
  • Requirements
  • Installation
  • Run the project
  • Configuration
  • General information
  • Azure Pipeline
  • Test distribution strategy
  • Other information
  • Contributing
  • License

Introduction

Solution for running UI Tests suite in Kubernetes in order to reduce test execution time. Following framework / platforms are currently used in this project:

• Azure Kubernetes Service (AKS)
• PowerShell Core (Azure CLI)
• Cypress
• Azure DevOps pipeline

Requirements

• Docker >= 18 (kubernetes enabled for local debug)
• Azure subscription
• PowerShell / PowerShell Core (cross-platform) >= 5
• Azure CLI >= 2.16
• Kubectl >= 1.19
• Cypress >= 6.0

Installation

git clone https://github.com/clement-joye/Kubernetes-UI-Tests.git

• Your cypress tests and all necessary fixtures, plugins and support must be in cypress folder.
• If you are using external libraries for your cypress tests, you need to build your own docker image. You can take help from the existing Dockerfile.

For DEBUG purpose:

• Open config/config.DEBUG.json and replace imageName and baseUrl accordingly with your own values.
• Make sure that your kubectl context points to your local kubernetes.

For running in AKS:

• Open config/config.TEST.json and replace accordingly with your own values
  • resourceGroup,
  • name,
  • nodeCount,
  • vmSize,
  • imageName
  • baseUrl

Run the project

$ cd ./powershell
$ cd ./Invoke-K8sTests.ps1 -Mode "All" -Environment "DEBUG"

This will run the all 3 main stages of the PowerShell script: Create, Run, Dispose, with config.DEBUG.json configuration file.

For running specific stage:

$ cd ./Invoke-K8sTests.ps1 -Mode "Create" -Environment "DEBUG"
or
$ cd ./Invoke-K8sTests.ps1 -Mode "Run" -Environment "DEBUG"
or
$ cd ./Invoke-K8sTests.ps1 -Mode "Dispose" -Environment "DEBUG"

Configuration

As mentioned above:

The config.DEBUG.json is meant to be used for your local Kubernetes installation, therefore it requires no resourceGroup, name, nodeCount or vmSize.

The config.TEST.json is meant to be used with your cloud Kubernetes service (AKS currently), therefore resourceGroup, name, nodeCount or vmSize values must be provided.

• isLocal specifies if the PowerShell script is used agains your local environment or in AKS.
• wait specifies if Create or Dispose stages are meant to be run asynchronously or not. This is useful for configuring the way the script will be run in your build or release pipeline.

The rest of the json file is not meant to be edited, and is only there for leaving it open to future developement.

Example:

{
"ClusterParameters": 
    {
        "resourceGroup": "my-resourcegroup-name",
        "name": "my-aks-cluster-name",
        "nodeCount": 1,
        "vmSize": "Standard_DS2_v2",
        "wait": true,
        "isLocal": false
    },
}

General information

When running the PowerShell script, three different stages will be executed:

Create stage:

• Retrieves the configuration,
• Instantiates data in memory,
• Creates the necessary yaml templates to be used for deploying the UI tests,
• Creates the cluster (or use an existing one if already created).

Run stage:

• Gets an existing cluster,
• Initializes the necessary resources to run the UI tests properly such as cypress configuration and cypress folder to expose to the cluster)
• Deploys the ui tests in individual pods,
• Monitors the tests until one fails or all succeeded,
• Export the reports generated by the tests back to the local host in /reports folder.

Dispose stage:

• Clear all resources in the cluster
• Dispose the cluster (if AKS cluster is used).

Azure Pipeline

The pipeline azure-pipelines.yml provided is a minimal DevOps Azure pipeline that simply runs all three stages sequentially. It's up to you to adapt it to your needs, and make calls to the different stages separately to optimize the execution time of the pipeline as you see fit.

Test distribution strategy

The current strategy taken by the PowerShell script is to instantiate one pod per test file. Another strategy can be adopted, but this requires to modify the script accordingly.

The test execution time will decrease depending on different factors:

• The size of your VM. The more powerful the VM the faster your tests can execute to some extent.
• The node count in your cluster. Locally you are obviously limited by your own computer setup.
• The actual possibility for your tests to run in parallel.

Other information

• A publication about the project is available here on Medium

Contributing

Bug reports and pull requests are welcome on GitHub. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

License

The code within this repository is available as open source under the terms of the GNU GPL v3 License.