duojs / test Goto Github PK

View Code? Open in Web Editor NEW

15.0 15.0 8.0 221 KB

duo's testing utility

Makefile 1.47% JavaScript 97.89% HTML 0.64%

test's People

Contributors

Stargazers

Watchers

Forkers

alex-tuz beverlycodes ndhoule pigojigoo kvnneff chrissrogers iluxa-com jsdelivrbot

test's Issues

--middleware

edge-case when you need special rendering of files etc..

Example:

duo test <cmd> -m ./render-something.js

module.exports = function(app){
  app.use('/foo.js', function(){});
  app.use('/baz.html', function(){});
};

rename mocha-send.js to saucelabs.js

is there anyway to exclude timeout saucelabs test

Duo-test is really cool, but like the pic below, my saucelabs always fail because of timeout.

Is there anyway to ignore the timeout error or retry the test, or make the timeout optional? Thanks!

example advanced error

npm: [email protected]

duo-test/examples/advanced $ duo-test phantomjs

TypeError: 'undefined' is not a function (evaluating 'mocha.setup({ ui: 'bdd', timeout: 100 })')

when I try custom index.html

and examples/simple is totally ok

debug log

    duo-test emit fatal {"message":"TypeError: 'undefined' is not a function (evaluating 'mocha.setup({ ui: 'bdd', timeout: 100 })')","url":"http://localhost:56865/test/?__id__=phantomjs","lineno":11} +2s
TypeError: 'undefined' is not a function (evaluating 'mocha.setup({ ui: 'bdd', timeout: 100 })')
  url: http://localhost:56865/test/?__id__=phantomjs
  lineno: 11

saucelabs doesn't work on ie6..8

question: how does --pathname work?

Playing around with duo-test(1). Really cool, still think it needs more docs though. Maybe just a getting started section.

Question about how you use pathname. It seems like you build your test files into build.js, so something like:

$ duo test/*.js && duo test browser

So how does --pathname apply?

test duration should be correct

not sure but i think it's because we miss mocha start event sometimes, need to hack around it.

--cover

using duo api.

.use(cover)
if it's an entry instrument it
if it's a local dep of an entry like ./local.js instrument it.
profit

JS errors cause PhantomJS runner to hang

Javascript errors that disrupt the flow of a describe scope cause the entire test suite to stop running, but do not cause PhantomJS to exit. This results in duo-test hanging without reporting any errors.

As of 1.5, PhantomJS supposedly creates a window.phantom object that can be used to catch errors and issue an exit command to PhantomJS. I tried putting an error handler in my tests, but it doesn't seem to ever get hit. I'm thinking registration of an onError handler in Phantom might need to occur before my tests are loaded, and I'm not sure how to get that added to the default.html, or how to test it.

add me (and others) to npm

@lancejpollard @yields @ndhoule

allow specifying the port?

When you're iterating and making changes to components, then testing in the browser, it takes a second or two for the server to start and open a browser window. On top of that, it opens a new browser window.

Is there a way we could specify the port or something? This way it will always use the same URL at least, so we can just refresh the page.

Ideally though, we could just "start the test server" and it will rebuild every time we refresh if any changes are made.

Any recommendations on how to do this?

Error: timeout of "2000" reached when there is an index.html

When index.html is removed, all test cases passed

When there is an index.html, timeout error

➜  particle-by-resize git:(master) ✗ DEBUG=* make test
  koa-static static "/Users/zhangyujun/Workspace/particle-by-resize" {"defer":true} +0ms
  koa:application use serve +2ms
  koa-route GET /duotest -> /^\/duotest\/?$/i +2ms
  koa:application use - +0ms
  koa-route GET /duotest.js -> /^\/duotest\.js\/?$/i +0ms
  koa:application use - +0ms
  duo-test started localhost:62820 +6ms
  duo-test test on 1 browsers +0ms
  duo-test:browser phantomjs: webdriver running on 8910 +1s
  duo-test:browser phantomjs: connected +32ms
  duo-test:browser phantomjs: get http://localhost:62820/test/?__id__=phantomjs +1ms
  koa-send send "/test/" {"defer":true,"root":"/Users/zhangyujun/Workspace/particle-by-resize","index":"index.html"} +18ms
  koa-send send "/node_modules/duo-test/node_modules/mocha/mocha.css" {"defer":true,"root":"/Users/zhangyujun/Workspace/particle-by-resize","index":"index.html"} +19ms
  koa-send send "/node_modules/duo-test/node_modules/mocha/mocha.js" {"defer":true,"root":"/Users/zhangyujun/Workspace/particle-by-resize","index":"index.html"} +2ms
  koa-send send "/build.js" {"defer":true,"root":"/Users/zhangyujun/Workspace/particle-by-resize","index":"index.html"} +0ms

  Error: timeout of "2000" reached
    at null._onTimeout (/Users/zhangyujun/.nvm/versions/node/v0.12.3/lib/node_modules/duo-test/node_modules/co-timeout/index.js:28:17)
    at Timer.listOnTimeout (timers.js:110:15)

content of index.html

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>tests</title>
  <link rel="stylesheet" media="all" href="../node_modules/duo-test/node_modules/mocha/mocha.css">
</head>
<body>
  <div id="mocha"><p><a href=".">Index</a></p></div>
  <div id="messages"></div>
  <div id="fixtures"></div>
  <script src="../node_modules/duo-test/node_modules/mocha/mocha.js"></script>
  <script>mocha.setup('bdd')</script>
  <script src="../build.js"></script>
  <script>mocha.run();</script>
</body>
</html>

needs a gif :-)

I think people would see how amazing this was if it had a gif to go along with it.

Saucelabs times out

Saucelabs reports Browser request was cancelled before a Sauce Labs virtual machine was found.

On my 4th attempt I made it through all the tests but up until then it looks like co-timeout kills the connection if Saucelabs isn't responding fast enough.

river$ make test-saucelabs


  firefox:31:Windows 2012 R2

  ․․․․․․

  6 passing (4s)



  firefox:29:Mac 10.9

  ․․․․․․

  6 passing (499ms)



  chrome:35:Mac 10.9

  ․․․․․․

  6 passing (4s)



  internet explorer:11:Windows 2012 R2

  ․․․․․․

  6 passing (4s)



  internet explorer:9:Windows 2008

  ․․․․
  Error: timeout of "2s" reached
    at null._onTimeout (/Users/river/Projects/sort-by/node_modules/duo-test/node_modules/co-timeout/index.js:28:17)
    at Timer.listOnTimeout (timers.js:133:15)

make: *** [test-saucelabs] Error 1
river$ make test-saucelabs


  chrome:36:Windows 2012 R2

  ․․․․․․

  6 passing (4s)



  internet explorer:9:Windows 2008

  ․․․․․․

  6 passing (4s)



  firefox:29:Mac 10.9

  ․․․․․․

  6 passing (4s)



  safari:7:Mac 10.9

  ․․․․․․

  6 passing (4s)



  safari:6:Mac 10.8

  ․․․․․․

  6 passing (4s)



  internet explorer:11:Windows 2012 R2

  ․․․․․․

  6 passing (4s)


  Error: timeout of "2s" reached
    at null._onTimeout (/Users/river/Projects/sort-by/node_modules/duo-test/node_modules/co-timeout/index.js:28:17)
    at Timer.listOnTimeout (timers.js:133:15)

make: *** [test-saucelabs] Error 1

xvfb

didn't try this out yet, but local headless testing would be cool.

How to duo-test

Hi guys,

I'm just messing around with trying to get a simple test working.
I installed the node_module $ npm install duo-test (assuming that's what needs to happen).

I then created and built the following files:

index.js
build.js
  test/
    index.js

Added following test to the test file:

var assert = require('component/assert');
describe('Totez testin', function(){
  it('should do something', function(){
    var arr = [1,2,3];
    assert(arr.length = 3);
  });
});

I then tried the following commands:

$ duo-test : nothing happens

$ duo-test phantomjs: throws this error in the command line:

 Error: spawn ENOENT
    at exports._errnoException (util.js:742:11)
    at Process.ChildProcess._handle.onexit (child_process.js:1028:32)
    at child_process.js:1109:20
    at process._tickCallback (node.js:343:11)

$ duo-test browser chrome: opens chrome but throws a console error as it creates a script tag with the following url

http://localhost:60838/duotest?id=undefined&data=%7B%22event%22%3A%22ping%22%2C%22data%22%3A%7B%7D%7D&callback=__jp0

What am I doing wrong ?

http://localhost:50495/duotest?id=undefined

Full Error:

GET http://localhost:50495/duotest?id=undefined&data=%7B%22event%22%3A%22ping%22%2C%22data%22%3A%7B%7D%7D&callback=__jp0 404 (Not Found

Not sure what's going on here....

Building to build.js
Tests are in test/io.js
Running: duo test browser

doesn't support node 0.10

looks like mz/thenify broke our node 0.10 support:

stephenmathieson at MBP in ~/repos/github.com/somebody/something
$ tree . -L 2
.
├── Makefile
├── index.js
├── node_modules
│   ├── duo
│   └── duo-test
├── package.json
└── test
    └── index.js

4 directories, 4 files
stephenmathieson at MBP in ~/repos/github.com/somebody/something
$ node -v
v0.10.31
stephenmathieson at MBP in ~/repos/github.com/somebody/something
$ make
node_modules/.bin/duo --type js --root . < test/index.js > build.js

     building : from stdin
        using : component_compat
        using : stoj
    installed : [email protected]
    installed : [email protected]
    installed : [email protected]
    installed : [email protected]
    installed : [email protected]
        built : from stdin

node_modules/.bin/duo-test phantomjs --reporter spec
The file "/Users/stephenmathieson/repos/github.com/somebody/something/node_modules/duo-test/node_modules/koa-static/node_modules/koa-send/node_modules/mz/node_modules/thenify/index.js" requires `Promise`,
but neither `bluebird` nor the native `Promise` implementation were found.
Please install `bluebird` yourself.
make: *** [test] Error 1
stephenmathieson at MBP in ~/repos/github.com/somebody/something
$ nvm use 0.11
Now using node v0.11.14
stephenmathieson at MBP in ~/repos/github.com/somebody/something
$ make clean test
rm -rf build.js components
node_modules/.bin/duo --type js --root . < test/index.js > build.js

     building : from stdin
        using : component_compat
        using : stoj
    installed : [email protected]
    installed : [email protected]
    installed : [email protected]
    installed : [email protected]
    installed : [email protected]
        built : from stdin

node_modules/.bin/duo-test phantomjs --reporter spec

thing
  ✓ should do stuff 

  1 passing (11ms)

stephenmathieson at MBP in ~/repos/github.com/somebody/something

where my Makefile looks like:

BIN := node_modules/.bin
DUO := $(BIN)/duo
TEST := $(BIN)/duo-test

test: build.js node_modules
    $(TEST) phantomjs --reporter spec

build.js: test/index.js index.js node_modules
    $(DUO) --type js --root . < $< > $@

node_modules: package.json
    npm install
    @touch $@

clean:
    rm -rf build.js components

allow console.log

with my latest removal of mocha-phantomjs (to simplify and speed things up) you can't console.log() anymore, just adding this here to see if you guys think it should be added back.

command not found

On my Fedora 20 laptop (and presumably my Ubuntu 14.04 desktop, but I'm not at home atm) duo-test does not work very well.

$ duo test browser

  Error: Command failed: /bin/sh -c google chrome "http://localhost:60883/test/"
/bin/sh: google: command not found

    at ChildProcess.exithandler (child_process.js:729:12)
    at ChildProcess.EventEmitter.emit (events.js:110:17)
    at maybeClose (child_process.js:992:16)
    at Socket.<anonymous> (child_process.js:1142:11)
    at Socket.EventEmitter.emit (events.js:107:17)
    at Pipe.close (net.js:461:12)

On my machine at least, the command is called google-chrome, but I'm guessing this is not the case on Mac?

The name function is returning "google chrome", which after changing it to "google-chrome" it worked. (also noticed "internet explorer" should probably be "iexplorer".

pass all args after `--` to phantomjs / gravy etc..

figure out why phantomjs takes 1s to start and 2s to run tests.

support stdin

Would be cool so we don't have all that redirection:

duo test/index.js | duo test

Add support for https

Would be useful to be able to run tests on either:

http://localhost:port/test
https://localhost:port/test

Maybe we can just pass that in as an option to the Runner? Haven't figured out yet how you guys want to do this kind of stuff.

how do i use this?

duo builds don't expose require(), so how can i test my module?

fix travis

https://travis-ci.org/duojs/test/builds/42224295

... i'd do it, but don't have push access

Proposal: Change CLI to accept entry files (similar to the duo itself)

I don't know if I'm the only one, but I find the current CLI a little confusing to use at first. So much so, I've actually created my own duo-based test-runner at dominicbarnes/duo-test, and I think the interface is a lot more intuitive, especially in comparison to the duo core CLI.

I figured I would at least propose my idea, but I'm totally fine continuing to just use my own runner at this point. (the only real gap it has is that it doesn't do saucelabs testing)

Basically, my CLI takes a variable number of arguments, where each is an entry file that is added to the resulting HTML. (or test/*.js as the default) It uses dominicbarnes/duo-serve under the hood to auto-rebuild. (rather than incorporating a build command into the CLI)

# run phantomjs testing against test/*.js
$ duo-test phantomjs

# start server for browser testing of test/a.js and test/b.js
$ duo-test browser test/a.js test/b.js

My main issue with the current API is the complexity of adding custom build commands, as the default is seldom usable for me. Plus, making the command mirror duo itself should make it easier to start using.

sauce-connect

maybe it will solve domains issue for some scripts: segmentio/analytics.js-integrations#357 (comment)

browsers

android error
iphone timeout
ie7 hangs
ie6 hangs

Building the library example

All the duo-test(1) examples rely on a build.js file being located in the root.

I'm looking at the Library example in particular and wondering if it's possible to have the tests reference build/build.js instead of build.js.

Im a noob to Makefiles and duo-test, so apologies if this is obvious.

T = ./node_modules/duo-test/bin/duo-test
TESTS = $(filter-out test/tests.js, $(wildcard test/*.js))
SRC = $(wildcard index.js lib/**/*.js)

build.js: test/tests.js
    duo --root . --type js < $< > $@

test/tests.js: $(SRC) $(TESTS)
    @echo '// GENERATED FILE: DO NOT EDIT!' > $@
    @$(foreach test, $(TESTS), echo 'require("./$(test)");' >> $@;)

test: test-phantomjs

test-phantomjs: build.js
    @$(T) phantomjs --reporter spec

test-browser: build.js
    @$(T) browser

test-saucelabs: build.js
    @$(T) saucelabs -b safari:6..7

clean:
    rm -rf test/tests.js build.js components

.PHONY: clean test test-phantomjs test-browser

flag to not open a browser

e.g. if you just want to create a server/rebuild/etc.

in the vm:

somebody app (xxx) $ node_modules/.bin/duo-test --build build/tests.js --port 3000 browser

  Error: Command failed: /bin/sh -c google chrome "http://localhost:3000/test/"
/bin/sh: 1: google: not found

    at ChildProcess.exithandler (child_process.js:203:12)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at maybeClose (internal/child_process.js:818:16)
    at Socket.<anonymous> (internal/child_process.js:319:11)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at Pipe._onclose (net.js:469:12)

somebody app (xxx) $

Client ID always returns `undefined` in browser tests

Here browser.js calls DuoTest.url() without an id so the url does not get formatted with one. That means that on this line window.location.search seems like it's always going to be an empty string so function id() will always return undefined.

So function event() will always return a bad query string here.

Admittedly I'm unsure of the expected functionality so I don't know how to fix it. My tests seems to all run great, its just annoying to see repeated calls to an invalid url in the console.

better error messages

can't figure out how to use this

I can't figure out how to get this module working, I have a pretty simple setup I'd like to test:

index.js
test/
  index.js

If I add a make target of:

test: node_modules
    @$(duo-test) browser

Then I get a "Not found" when the page opens. And phantomjs also fails. Not sure what I'm doing wrong here though..

Am I supposed to build the tests myself? Or am I supposed to make the index.html test serving file myself? I assumed duo-test would be doing that for me?

saucelabs

Unable to test in safari?

$ BROWSER=safari make test-browser
node_modules/.bin/duo-test browser safari \
      --reporter spec \
      --build tests.js \
      --port 5678 \
      --commands "make tests.js"

  Error: "safari" is not supported, supported browsers: firefox,chrome,safari,opera,ie
    at callee$0$0$ (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/bin/browser.js:39:10)
    at Empty.invoke (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/gnode/node_modules/regenerator/runtime.js:180:31)
    at next (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/co/index.js:74:21)
    at Command.<anonymous> (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/co/index.js:45:5)
    at next (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/co/index.js:90:21)
    at Command.<anonymous> (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/co/index.js:45:5)
    at Command.listener (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/commander/index.js:282:8)
    at Command.emit (events.js:98:17)
    at Command.parseArgs (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/commander/index.js:524:12)
    at Command.parse (/Users/stephenmathieson/repos/github.com/stephenmathieson/temporarily-insert/node_modules/duo-test/node_modules/commander/index.js:410:21)

make: *** [test-browser] Error 1

mocha opts

creating index.html just to add options is annoying.

get rid of mocha-phantomjs

wtfs:

--setting webSecurityEnabled=true
logs phantomjs errors
coffeescript
js shims
hacky

better browser syntax

parse('chrome'); // => latest chrome on it's "default" os
parse('ie:11'); // => ie11 on it's default os.
parse('chrome:*') => // all chrome versions
parse('chrome:*:*'); => // all chrome versions on all oses
parse('safari:5:*'); // => // safari 5 on all oses
parse('lynx'); // => the latest lynx on it's default os
parse('iphone'); // => latest on ios
parse('ie:6..9'); // => ie 6 to 9
parse('ie:6..'); // => ie 6 to the latest version
parse('chrome:..beta'); // => oldest chrome to beta
parse('chrome:30:win,osx,linux'); // => chrome 30 on win,OS X and linux

Testing React components

Hi guys, this might not be an issue specifically with duo-test but perhaps someone might be able to shed some light.

I’d like to test react components, but the react script itself is making the test runner fail.

I have my component…

// hello.js
/** @jsx React.DOM */
var React = require('reactjs/react-bower:react.js');

var SayHello = React.createClass({
  render: function() {
    return (
      <h1>Hello, world!</h1>
    );
  }
});

module.exports = SayHello;

and my test

//test/hello.js
var assert = require('component/assert');
var sayHello = require('../hello.js');

describe(‘sayHello’, function(){
  it('should say hello’, function(){
    // just a placeholder for now
    assert.equals(2, 2);
  })
})

I build my test ( using the duo-jsx plugin ).

duo --use duo-jsx --root . --type js < test/tests.js > build.js

Run my tests

./node_modules/duo-test/bin/duo-test phantomjs --reporter spec

and I get the following error relating to react’s code.

TypeError: 'undefined' is not a function (evaluating 'ReactElementValidator.createElement
.bind(                                                                                   
      null,                                                                              
      type                                                                               
    )')                                                                                  
  url: http://localhost:62272/build.js                                                   
  lineno: 10343

which relates to :

  // build.js

  //  react shiz
  createFactory: function(type) {
    var validatedFactory = ReactElementValidator.createElement.bind(
      null,
      type
    );
    validatedFactory.type = type;
    return validatedFactory;
  }
  …

Has anyone had any experience testing react components with duo-test ?

Is this more of a ‘if react was a proper component this possibly wouldn’t happen’ situation, or is there a way around this?

Separate tests for local components

Im really liking duo-test. I can see how it'll save so much time in creating and running tests.

Can you tell me if running tests for local components is supported?.

Example structure:

package.json
index.js
  local
    some-component
      index.js
      tests
        build.js
        test.js
    some-other-component
      index.js
      tests
        build.js
        test.js

After building a tests inside a local component, I'm running $ duo test phantomjs which returns:

$ error : duo-test(1) does not exist

This is how I currently work in a component(1) env - but I'm open to other suggestions/approaches if you guys have already thought about this.

browsers not up-to-date

For example: iPhone 8 and 8.1 should be available

./node_modules/.bin/duo-test saucelabs -b iphone:stable.. -B ./build/testsBuild.js


  iphone:7.1:Mac 10.9

  ․․․․․․․․․․․․․․․․․․․․․

  21 passing (9s)

see yields/wd-browser#3

move phantomjs stuff into its own repo

the only alternative (mocha-phantomjs) is slow and buggy. it'd be nice to be able to run tests in phantom without riding on top of duo-test.

thoughts?

unable to specify reporter

duo-test dies when attempting to use a built-in reporter:

$ node_modules/.bin/duo-test phantomjs --reporter tap

  AssertionError: reporter "Tap" was not found
    at duotest (/some/path/node_modules/duo-test/bin/_duo-test:115:3)
    at Command.<anonymous> (/some/path/node_modules/duo-test/bin/_duo-test:96:26)
    at GeneratorFunctionPrototype.next (native)
    at next (/some/path/node_modules/duo-test/node_modules/co/index.js:74:21)
    at Command.<anonymous> (/some/path/node_modules/duo-test/node_modules/co/index.js:45:5)
    at Command.listener (/some/path/node_modules/duo-test/node_modules/commander/index.js:282:8)
    at Command.emit (events.js:110:17)
    at Command.parseArgs (/some/path/node_modules/duo-test/node_modules/commander/index.js:524:12)
    at Command.parse (/some/path/node_modules/duo-test/node_modules/commander/index.js:410:21)
    at Object.<anonymous> (/some/path/node_modules/duo-test/bin/_duo-test:72:9)

is there any reason we're pascal-casing the reporter name?

I think default build path should be build/*

That allows you to have multiple files in test/ and just run:

$ duo test/*.js && duo test browser

source map support in testing

It seems currently the test command is locating the error in build.js. Is it possible to make it to the original source code?

  1) grid should sample the canvas to grid:
     TypeError: 'undefined' is not an object (evaluating 'this.options.scale')
      at http://localhost:51934/build.js:5399
      at http://localhost:51934/build.js:119
      at callFn (http://localhost:51934/mocha.js:4452)

commands to be run as middleware

super useful if we can do: duo-test --commands "make build, make test" and just have those run whenever the server loads.