Giter Site home page Giter Site logo

redcap_cypress's Introduction

CircleCI

REDCap Cypress Test Framework

The REDcap Cypress Test Framework enables automated feature testing for REDCap.

This repository serves as a starting point for running and writing your own automated feature and regression tests, which can be used to validate the features and functionality of your REDCap environment.

Features are written using the Gherkin domain-specific language.

Core REDCap features, written in Gherkin, are posted to a repository guided by the Regulatory & Software Validation committee: https://github.com/aldefouw/redcap_rsvc

Overview

How to Install

If you are a developer, please start by visiting the REDCap Cypress Developer Toolkit, located here: https://github.com/aldefouw/redcap_cypress_docker

The Developer Toolkit is the best way to get Cypress up and running on your developer machine.


Defining Your Test Environment

Environment Variables

Cypress will understand your environment only if you describe it accurately.

Your description will live inside an environment variable definition file.

You will need to set the variables appropriately in cypress.env.json in order for your test suite to function.

cypress.env.json

To get you started, an example file named cypress.env.json.example is included within this repository.

Here is an example environment variable setup:

{
  "users": {
    "admin": {
      "user": "admin_user",
      "pass": "Testing123"
    },
    "standard": {
      "user": "test_user",
      "pass": "Testing123"
    }
  },
  "redcap_version": "13.1.7",
  "redcap_source_path": "../redcap-source",
  "mysql": {
    "host": "docker exec -i redcap_docker-app-1 mysql",
    "path": "mysql",
    "port": "3306",
    "db_name": "redcap",
    "db_user": "root",
    "db_pass": "root"
  }
}

Below are descriptions of the configuration variables shown above.


redcap_version

The version of REDCap that you are testing against. This is a critical value to set so that Cypress knows the correct URLs to use when testing. Note that the version of REDCap you specify here MUST be available on your machine in order for tests to work.


mysql

The JSON array that contains several keys, which are critical for your database structure and seeds to be populated correctly before each and every test spec.


mysq['host']

The hostname or IP address of your MySQL database host.

For many of us, this will likely be either localhost, 127.0.0.1, or a reference to a Docker container via db. Keep in mind that there are subtle nuances between localhost and 127.0.0.1. Thus, you need to choose the option best-suited to your environment.


mysql['path']

The path to your mysql binary.

For many of us, this will probably be mysql, but you could also use a full path like /usr/local/opt/[email protected]/bin/mysql if necessary. If you are on a Unix-like environment, you can often determine your full path by entering which mysql at the terminal window. You can also reference a binary within a Docker container like this docker exec -i redcap_docker-app-1 mysql


mysql['port']

The port to your MySQL instance.

This is usually 3306 on standard setups, but for many of us running Docker instances we may wish to use an alternative port so we can differentiate between the standard MySQL instance that is installed on a local operating system and the Docker instance itself.


mysql['db_name']

The name of your MySQL REDCap database.

This is typically redcap but not always. You'll want to check your database.php file on your test instance of your REDCap installation to determine this value.


mysql['db_user']

The username of your MySQL REDCap database user.

This is typically root on local instances of MySQL or local Docker containers. You'll want to check your database.php file on your test instance of your REDCap installation to determine this value.


mysql['db_pass']

The password of your MySQL REDCap database user.

This is typically root on local instances of MySQL or local Docker containers. You'll want to check your database.php file on your test instance of your REDCap installation to determine this value.


Database Strategy

To create non-deterministic tests, the REDCap Cypress Test Framework resets the database to a known state before each feature is run.

Before the test suite is run, the appropriate tables for your specified REDCap version are installed into your MySQL database. To achieve this, the framework needs to know about where your REDCap source code is located.

You will need the following environment variables configured in your cypress.env.json file:

  • "redcap_source_path": "../path/to/redcap/source/here": Contains the relative or absolute path to your REDCap source folder root (files from Vanderbilt). Must contain the version-specific files for the version you wish to tests against.

  • "redcap_version": "13.1.37": Contains the string version of REDCap you want to test against.

The seeds file in this template repository also include a both an admin user and a standard user.

Which user you use to login to REDCap is dependent upon what kind of feature you are intending to test.

Running Your Tests

Opening Cypress

To run the tests in the Cypress debug environment, issue the following command at the root of your test folder:

npx cypress open

A Cypress window will open and you can select which specs you'd like to run.

Headless Mode

After your test suite is mature, it will be faster to run your tests in headless mode. This is how you would run your tests on a CI server. To do so, issue the following comand:

npx cypress run


redcap_cypress's People

Contributors

aldefouw avatar andrewpoppe avatar cdebacker avatar krishchant avatar mmpeterson24 avatar mukerji2 avatar rabramley avatar rushi79 avatar tintinn avatar wibeasley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redcap_cypress's Issues

Krishna Upadhyay

Operating System: Windows Server 2016

Test Environment: IIS10 / PHP7 / MySQL8

Handling security-related bugs

If a test uncovers a security vulnerability, it shouldn't be available to the public. Maybe we see if vandy would fund a private instance of circle ci (or whatever could be web-scraped/mined but a malicious user looking to exploit a system with phi).

Adam De Fouw

Operating System: CentOS 7 / PHP 7

Test Environment:

MacOSX Catalina 10.15 host operating system running Docker Container running CentOS 7 / PHP 7

Testing External Dependencies (like Mail)

@dr01d3r brought up a good point about external dependencies like mail.

For instance, Manage User tests will likely look at approving an account's email address.

In our next meeting we should discuss how we want to handle this external dependency.

For REDCap, we could do something like this:

Apache or IIS with virtual hosts:

  1. REDCap server (whatever port) mapped to http://redcap.domain.com
  2. MailHog Server (port 1080) mapped to http://mail.domain.com

Question is ... should we?

Implement Dynamic User Seeds

Users are configured within the project like this right now:

"users": {
    "admin": {
      "user": "test_admin",
      "pass": "Testing123"
    },
    "standard": {
      "user": "test_user",
      "pass": "Testing123"
    },
    "standard2": {
      "user": "test_user2",
      "pass": "Testing123"
    }
  }

We need to use to cy.mysql_db() to automatically seed the users as the ones defined within this environment variable setup.

Should not be too hard since we can use REGEX replacement in the second "replace" parameter of cy.mysq_db(). Something like this:

REDCAP_USER/username

Where REDCAP_USER is the magic constant and username is the name passed in from a Cypress environment variable.

Could/should this be integrated into REDCap Core, the module framework, or specific modules?

I must admit I forgot this library existed... I'm afraid I might do the same again if I don't stumble upon it again for another year or so, and possibly re-implement parts of it as future needs arise. Could/should any existing tests people have written using this framework be incorporated into REDCap Core, the module framework, and/or specific modules so they are automatically available to all REDCap users (and perhaps even automatically run during updates)? I know none of the above are trivial, but I'm curious to hear your thoughts.

Shefali Mukerji

Operating System: CentOS 7 / PHP 7

Test Environment:
MacOSX 10.13 host operating system running Docker Container running CentOS 7 / PHP 7

Approach for feasibly covering different versions

As we discussed this week with Kyle and others, there are several possible approaches for covering multiple versions of REDCap. It sounds like you'd appreciate the thoughts and feedback from various people.

REDCap versions affect the framework in at least two places:

  1. Installing a specific version by cypress, especially the sql files that seed the data.
  2. The tests themselves. For instance, if a feature was added or modified in 8.2.3, the test should by different (or possibly skipped) when testing 7.4.4.

Cc: @123andy, @mullen2, @pbchase, & @kcmcg

Brian Funaro

Operating System: Linux ???

Test Environment: Docker ???

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.