spion/blue-tape

substack's tape test runner with promise support

blue-tape's Issues

`instanceof Function` isn't reliable cross-realm

If you have a promise from an iframe or a worker or something, Function is different, and instanceof won't work.

A simple test is typeof foo === 'function', but https://www.npmjs.com/package/is-callable is a fully reliable test that works across all engines. Would you accept a PR using that?

Don't try to detect promises. Just use `Promise.resolve`.

https://github.com/spion/blue-tape/blob/master/blue-tape.js#L14

P/A+ Promises already have a carefully spec'd promise resolution procedure for detecting whether something may be a promise, without potentially throwing errors. The proper way to handle things that might or might not be actual promises is to coerce them into being promises using Promise.resolve, which takes care of unwrapping thenables for you. More importantly, resolve yields rejected promises when weird edge cases (e.g. accessing a .then which is actually a getter which throws only on the second access) could possibly come into play.

Blue tape currently goes partway, manually, towards trying to handle these nuances. It would be simpler and more foolproof to just use the existing behavior of ES2015-compliant native promises, IMHO.

---

PS: Ah, I see part of the difficulty is in supporting older environments, plus some concerns that native promises don't go far enough: #9. Not sure I agree with the latter point, but if you prefer to use manual inspection then at least I can see an argument for it.

Consider loosening Tape version requirement to 2.x

That way you won't need to update manually so often.

Substack is generally good about semver in my experience.

Using with t.plan and t.end

I was wondering if it was possible to use blue-tape with t.plan and t.end as not all my tests return promises. I've tried t.plan but getting an error:

not ok 2 plan != count
  ---
    operator: fail
    expected: 5
    actual:   0
  ...

Assume sync if no promise is returned?

If the test return value is not a promise, would it be better to assume that the test is synchronous?

Error output stack trace could be more useful

Note: I don't know enough about Bluebird's handling of stack traces to know if what I'm bringing up is unique to my setup or a common problem.

Currently my error output looks like this:

✖ TypeError: Cannot read property 'jobId' of undefined
    -------------------------------------------------------
      operator: error
      expected: |-
        undefined
      actual: |-
        [TypeError: Cannot read property 'jobId' of undefined]
      at: tryCatcher (/d/node_modules/bluebird/js/release/util.js:16:23)
      stack: |-

Unfortunately, the error stack trace line there isn't very helpful for me.

But if I catch the error myself at the end of the test and console.log(err.stack), the start of my output looks like this:

    TypeError: Cannot read property 'jobId' of undefined
        at waitForJobToFinish (mozQueue.js:59:49)
        at mozQueue.js:49:40

Which is exactly where the error is.

Adding Generator Support?

@spion just a thought to throw your way to get your thoughts. If a user passes a generator function for a test, wrap it in a Promise.coroutine call (or something like that) in order to yield promises, etc. Here is an implementation I'm using currently that wraps blue-tape.

import test from 'blue-tape'
import Promise from 'bluebird'

export default function genTest (...args) {
  let testArgs = args.map( arg => {
    if (arg && arg.constructor.name === 'GeneratorFunction') {
      return Promise.coroutine(arg)
    }

    return arg
  })
  test.apply(null, testArgs)
}

Allowing tests like so:

test('User#save', function* (t) {
  let user = common.initUser()
  user.password = 'foo'
  yield user.save()
  console.log(user.password)
})

Thoughts?

Publish a new version to npm

Test that promise rejects/throws

I can't figure out how I should test that a function that returns a promise (async function) rejects/throws.

I feel I have tried everything up my sleeve: async-await vs then-catch, t.throws(), catch inside test and t.ok()/t.end(), transpiling with babel vs babel-node, etc.

How?

Provide support for generator functions

Hi, I am currently using blue-tape with babel-tape-runner and co.js to use generator functions. Is it possible to pass a generator function instead of a normal one returning a promise?
Here is my test:

test('generators work', t =>
    co(function* () {
        const fixture = setup();

        const response = yield supertest(fixture.server)
            .post('/')
            .end();

        t.equal(response.headers['content-type'], 'application/json; charset=utf-8',
            'type should be JSON and UTF-8');

        teardown(fixture);
    })
);

Here is what I would like to do:

test('generators work', function* (t) {
        const fixture = setup();

        const response = yield supertest(fixture.server)
            .post('/')
            .end();

        t.equal(response.headers['content-type'], 'application/json; charset=utf-8',
            'type should be JSON and UTF-8');

        teardown(fixture);
});

I read that blue-tape somehow supports generators, but I could not get this to work.

Output shows 0 tests when promise is resolved

Hello,

I have a tests directory with this single file (test.spec.js) in it.

var tape = require('blue-tape');

tape('test', function(assert) {
    return Promise.resolve();
});

When I run node node_modules/.bin/tape tests/*.spec.js here is the output I get:

TAP version 13
# test

1..0
# tests 0
# pass  0

# ok

However, when I have this as my test:

var tape = require('blue-tape');

tape('test', function(assert) {
    return Promise.reject();
});

I get the following output:

TAP version 13
# test
not ok 1 (unnamed assert)
  ---
    operator: fail
  ...

1..1
# tests 1
# pass  0
# fail  1

Why does a test with a resolved promise incorrectly tell me that I have 0 tests and 0 passing? But a rejected promise correctly tells me that I have 1 test and 1 failing?

Thanks!

Explain how/if this differs from .catch(t.error)

Looks like it obviates the need for t.plan/t.end, which is pretty cool. But the advantages could be called out better in the readme.

Errors don't crash test runner

This is more of a feature than an issue - normal tape now crashes the test runner on thrown errors, but blue-tape seems to retain the old behavior, and that of tape-catch.

This should be called out in the readme, as both tape-catch and blue-tape are wrappers, and mutually exclusive. Accordingly, it would seem like a developer would need to choose between exceptions as failing tests, or promises, but they can really have both with blue-tape.

No custom message for functions ok, true, assert

Sending a custom message to ok, true, assert-function doesn't produce correct result in the spec reporter. Could be an issue with tape instead. Couldn't say. However it's probably not due to tap-spec since pure TAP output is: ok 6 should be truthy. The definitions according to tape is https://github.com/substack/tape#tokvalue-msg.

t.ok(dataStr.includes('/redirectToLogin', 'contains some url part'));

Spec reporter
Output contains only the following:

should be truthy

package-lock.json excerpt

    "blue-tape": {
      "version": "1.0.0",
...
   "tap-spec": {
      "version": "5.0.0",

Expected

contains some url part

Move tape to peer dependencies

I am trying to run blue-tape tests in gulp with gulp-tape plugin. But unfortunately this plugin assumes that tape is installed. I see two solutions for this issue. One is just to fork gulp-tape and make it work with blue-tape. But it doesn't look very nice. The other way is modify blue-tape deps - move tape from dependencies to peerDependencies. I am not sure if this is a right way but it will give an opportunity to use blue-tape in conjunction with other tape-related libraries without forcing them to know about blue-tape.
I want to ask your opinion if it makes any sense. Or maybe there are better solutions?

Improve shouldFail() similar to throws()?

That supports shouldFail(p: Promise, expected: RegExp | Function, msg?: string)

🌹

Can't assert within a 'then' block

Maybe I'm using this library wrong, but I can't seem to assert anything from within a then block, as so:

require('babel-core/register')({
  presets: ['es2015']
});
const test = require('blue-tape');

...

test('setup', (assert) => {
  return ddpClient.connect().then(() => {
    assert.pass(); // this can be assert.end() and it will still fail
  });
});
test('basic test', (assert) => {
  assert.equal(1, 1);
  assert.end();
});

With this example, both tests fail and give me the following output:

TAP version 13
# setup
not ok 1 test exited without ending
  ---
    operator: fail
  ...
not ok 2 test exited without ending
  ---
    operator: fail
  ...

1..2

If I simplify this example by making the test's callback return a Promise, I can assert anything up until I try returning ddpClient.connect() which returns a Promise.
Side note: if I do assert.end() before that line, by 'basic test' doesn't run, and it says I only passed 1 test.

One thing that may be affecting the output is my use of babel to transpile the ddpClient module and all node_modules I'm using, but I need it (b/c I'm testing React Native, pseudo-ES7 code and trying to mock out a WebSocket server) and it's also hard for me to see why that may be the source of these issues. Can someone point me in the right direction?

Passing tests are not reported

Version info

npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
[email protected] /src
`-- [email protected]
  `-- [email protected] 

npm info ok 

test.js

const test = require('blue-tape');

function delay(seconds) {
    return new Promise(function (resolve) {
        setTimeout(resolve, seconds * 1000);
    });
}

test('simple delay', function () {
    return delay(1);
});

test('should fail', function () {
    return delay(1).then(function () {
        throw new Error('Failed!');
    });
});

Output of node test.js

TAP version 13
# simple delay
# should fail
not ok 1 Error: Failed!
  ---
    operator: error
    expected: |-
      undefined
    actual: |-
      [Error: Failed!]
    stack: |-
      Error: Failed!
          at /src/dumb.js:15:15
          at <anonymous>
  ...

1..1
# tests 1
# pass  0
# fail  1

Shouldn't it say

# tests 2
# pass 1
# fail 1

?