Meteor Cucumber

This package gives you Cucumber Velocity framework so you can do BDD with Meteor.

What is Cucumber?

Cucumber is an open-source Behaviour-Driven Development (BDD) tool popular in Agile circles. It allows you to define the behaviour of your app using plain text. See below for more details and examples.

#Get the Book To learn more about testing with Meteor, consider purchasing our book The Meteor Testing Manual.

Your support helps us continue our work on Velocity and related frameworks.

Features

CucumberJS with Promise support (mostly ready)
Includes Chai & Chai-as-promised promise based assertions by default
Auto-configured WebdriverIO with PhantomJS
Auto-configured a promise-based DDP connection to the mirror
Auto-downloads and configures Selenium to gives you real browser testing locally
Supports SauceLabs / Selenium Grid out of the box
Reduces CucumberJS stack trace noise by only showing you relevant lines (can be disabled)
Experimental Parallel testing support (see the super fast e2e testing talk)

Installation

meteor add xolvio:cucumber

Usage

Basic

After adding the package, click the 'Add cucumber sample tests' button in the HTML reporter. This will create these files in your project:

tests
└── cucumber
    ├── features
    │   ├── sample.feature
    │   └── step_definitions
    │       └── sample_steps.js
    └── fixtures
        └── my_fixture.js

And you should also see this in the reporter:

Don't worry, the failure is by design to encourage you to always start with a failing specification. If you go ahead and change the file /tests/cucumber/features/sample.feature and replace "intentional failure" with the actual title of your site, you should see this:

See a more detailed Example of BDD with Meteor below.

Step Sugar

Meteor Cucumber uses Chimp which gives you some sugar inside the step definitions.

Chai / Chai-as-Promised

Chai and Chai-as-Promised have been wired in both into the steps (globally) and integrated with WebdriverIO. This means you should be able to perform assert from any context.

WebdriverIO

Chimp, and therefore Meteor Cucumber, use the excellent WebdriverIO and this comes pre-configured with PhantomJS by default and is available for you to use like this:

this.client.
  getTitle().should.become(expectedTitle).and.notify(callback);

Notice the prose uses chai-as-promised instead of dealing with callback nesting.

IMPORTANT: A gotcha to look out for is to be sure to use and.notify(callback) instead of using WebdriverIO's .call(callback). This is because Cucumber doesn't support Promises/A+ (yet). This is something Xolv.io will be adding to Meteor Cucumber soon.

If you want to call a Meteor method from the browser, you can do so like this:

// From browser, call server-side method and test response with `assert`.
this.client.executeAsync(function (done) {
  Meteor.call('tester', function (err, res) {
    done(res); // Don't pass `err` into this as the first argument!
  });
}, function (err, res) {
  assert.equal(res.value, 'yes');
  callback();
});

The same mechanism above can be used to perform tasks like Meteor.loginWithPassword etc. For more details on how to use the executeAsync command, refer to the WebdriverIO docs.

Note you can also access the browser instance on the global object like this: global.browser. This is useful if you are trying to connect from a context where the world object may have been destroyed, like a hook.

If you wish to use real browsers, see the WebdriverIO Options below.

DDP

You have a DDP connection pre-connected to the mirror server that you can access like this:

this.server.call('yourMethod', arg1, arg2, ..., callback);
// or
this.server.apply('yourMethod', [arg1, arg2, ...], callback);

Note that you can use this.server or this.mirror. They are both an alias to this.ddp.

The signature for the DDP call is the same as Meteor.call, and you can also use the Meteor.apply method too. Meteor Cucumber uses a forked version of the oortcloud node-ddp-client. The Xolv.io fork matches the method signatures of call and apply of the Meteor API, and also promisifies them using Bluebird. This means you can't use the usual asynchronous method callbacks, but you can achieve the same thing by calling this.server.call('myMethod').then(callback).

You can use this connection to either perform API-level end-to-end testing, or to use in conjunction with fixtures to clear the mirror database or to setup test-data (see Fixtures).

Note you can also access the DDP client on the global object like this: global.ddp. This is useful if you are trying to connect from a context where the world object may have been destroyed, like a hook.

Be mindful that the DDP connection to the server. If you want to call a Meteor method from the client, you should do so using the this.client. See WebdriverIO for more details.

Fixtures

If you include any source files under the /tests/cucumber/fixtures directory, these will be included on the mirror only. This is very useful as it allows you to have test-only code such as fixtures. Here's an example that shows you how you can reset your system prior to every scenario:

// /tests/cucumber/fixtures
Meteor.methods({
  'reset' : function() {
    MyCollection.remove({});
    MyOtherCollection.remove({});
    // ...
  }
});

// /tests/cucumber/features/step_definitions/hooks.js
this.Before(function (event, callback) {
  global.server.call('reset', callback);
}

The hook will call the fixture on the mirror and ensure your collections are cleared before each scenario is run. Magic!

Another pattern for creating fixtures is to do the following inside your app:

meteor create --package fixtures

Then modify the package.js file to set the debugOnly flag to true like this:

Package.describe({
  name: 'fixtures',
  version: '0.0.1',
  debugOnly: true,
  // ...
});

The debugOnly flag instruct Meteor not to bundle this package when building, which is how you ensure this package does not make it to production. You can now define all your fixtures in this package.

Logging

Logs from the mirror and cucumber can be seen in the log file here:

<your_project_dir>/.meteor/local/log/cucumber.log

It's advised that you monitor this log file with a command like

tail -f .meteor/local/log/cucumber.log

Adding NPM Modules

You might want to use npm packages inside your steps, like underscore for instance. To do this, you can add a package.json file inside your /tests/cucumber directory and include npm modules like you would in any normal node app. Here's an example:

{
  "name": "cucumber-tests",
  "version": "1.0.0",
  "description": "Dependencies for our Cucumber automation layer",
  "private": true,
  "dependencies": {
    "fs-extra": "0.18.0",
    "underscore": "^1.8.3"
  }
}

Note you still need to manually run npm install yourself currently. This may change in future versions of this package.

Continuous Integration

On the CI server, just run: VELOCITY_CI=1 meteor --test

When VELOCITY_CI is set xolvio:cucumber runs all tags and not just the @dev tags like in dev mode. See the cucumber options below if you'd like to use custom tags.

To run your tests for Cucumber you just need to be sure any npm dependencies are installed on the CI server. So if you have created an npm package file under tests/cucumber/package.json, then you need to run npm install prior to running meteor --test

Here's an example CI script:

cd tests/cucumber
npm install
cd ../..
meteor --test

You may want to collect the raw json report from Cucumber. This can be done by setting the CUCUMBER_JSON_OUTPUT variable to the path you'd like the json report to be written to.

See the Letterpress circle.yml file for an example of using meteor-cucumber on CircleCI.

Configuration

You can configure settings using environment variables. These are available:

Chimp Options

This package is somewhat opinionated in its outputs and options, however you have complete control over how you want the underlying tools to behave. The tool that is doing all the work is Chimp. You can directly pass command line switches to Chimp as follows:

CHIMP_OPTIONS='--format=progress --browser=chrome ...'

See the Chimp docs for details.

Be mindful that using this is an advanced option that bypasses the defaults in xolvio:cucumber.

Cucumber Options

CUCUMBER_FORMAT=summary | json | progress | pretty (default)

CUCUMBER_COFFEE_SNIPPETS=1

CUCUMBER_TAGS=@mytag,@myOtherTag,~@notThisTag

CUCUMBER_SCREENSHOTS_DIR=./tests/cucumber/.screenshots (default)

Experimental: Parallel testing

To enable this mode, you need to checkout the parallel branch and set this environment variable: CUCUMBER_NODES=4

This will run 4 separate nodes, each with their own mirror, Cucumber, WebDriverIO and PhantomJS instances. These nodes will run through a feature at a time until all the features are complete. For this more to work best, it's advised that you keep your features short, which is generally good practise anyway.

WARNING: This mode is under development and it may or may not work for you! When it does work for everyone, it will be awesome!

WebdriverIO Options

WD_LOG=command/debug/silent (default)

WD_TIMEOUT_ASYNC_SCRIPT=10000 (default)

By default, WebdriverIO is wired up with PhantomJS. You can however run it locally with any other locally installed browser using Selenium or remotely on SauceLabs / Selenium Grid like this:

Selenium

SELENIUM_BROWSER=firefox | chrome | phantomjs (default)

Note: Selenium comes bundled with a driver for firefox. For other browsers you will have to download those drivers. See the full list of 3rd party bindings.

SauceLabs / Selenium Grid

HUB_HOST=ondemand.saucelabs.com | localhost (default)

HUB_PORT=xxxx | 80 | 4444 (default)

HUB_USER=dude

HUB_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx

HUB_PLATFORM='Windows 7' | 'OS X 10.8' | ANY (default)

HUB_VERSION=35

Example of BDD with Meteor

Here's an example of doing BDD with Meteor + Cucumber:

Describe your feature in human readable format using Gherkin syntax:

Feature: Author a Website

  As a web page author
  I want to set the title of my page
  So that I can create the simplest website in the world

  Scenario: Author using the Meteor settings file
    Given I have created a document with the title "Meteor Cucumber by Xolv.io"
    When  I navigate to "/"
    Then  I see the title "Meteor Cucumber by Xolv.io"

Upon saving the file you will see this in the log:

You then take these conveniently generated step definition snippets write the the code to automate the scenario as concrete actions:

this.Given(/^I have authored the site title as "([^"]*)"$/, function (title, callback) {
  // this.server is a connection to the mirror available to you in all steps
  this.server.call('updateTitle', [title], callback);
});

this.When(/^I navigate to "([^"]*)"$/, function (relativePath, callback) {
  // this.client is a pre-configured WebdriverIO + PhantomJS instance
  this.client.
    url(url.resolve(process.env.ROOT_URL, relativePath)). // process.env.ROOT_URL points to the app
    call(callback);
});

this.Then(/^I should see the heading "([^"]*)"$/, function (expectedTitle, callback) {
  // you can use chai-as-promised in step definitions also
  this.client.
    waitForVisible('h1'). // WebdriverIO chain-able promise magic
    getText('h1').should.become(expectedTitle).and.notify(callback);
});

Upon saving you should see a failing test indicator either in the Velocity HTML reporter or the console:

You then write the actualizing code to make the above steps work:

<body>
  <h1>Title Has Not Been Set</h1>
</body>

if (Meteor.isClient) {
  Meteor.call('getTitle', function(err, res) {
    $('h1').text(res).show();
  });
} else {
  Meteor.methods({
    'updateTitle' : function(title) {
      Meteor.settings.pageTitle = title;
    },
    'getTitle' : function() {
      return Meteor.settings.pageTitle;
    }
  });
}

Now you'll see a passing test indicator:

You now write another scenario or feature by going back to step 1.

Example Meteor Projects

Breaking Changes

No more test-proxy package

The latest Velocity 0.6.0 release removed the test-proxy package. After you update, please be sure to remove this package from your /packages folder.

No more xolvio:webdriver

You need to delete the xolvio:webdriver package as this is now built-in using Chimp

Cukes now runs outside the Meteor context

You no longer have access to the main Meteor app from within your step definitions. You should never need the main app anyway. Typically users were using Meteor.DDP, but you can now use this.ddp instead which is pre-connected to the mirror. See the DDP section above for details.

No World object / world.browser / world.ddp

Chimp already creates and initializes a simple world object. This means no more helper.world.browser, instead you can just replace all those calls with this.client.

Same goes for DDP, you now use this.server and be sure read the notes above about DDP

If you need a World object, please get in touch by reporting an issue and letting us know what you're trying to do.

Remove your hooks

If you are using the boilerplate hooks that the previous version of meteor-cucumber used, you should get rid of the hooks file. If you are doing any custom hooks, you should extract those outside the boilerplate file and use separately.

mhurwi / meteor-cucumber Goto Github PK

meteor-cucumber's Introduction