Generates an HTML view of the Nightwatch.js test reports by either parsing the XML files generated by Nightwatch or by using the Nightwatch reporter options.
License: MIT License
Is it possible to generate static html without jquery?
I wanted to generate HTML report as a last step of jenkins job. Currently it only attaches collapsed view.
Actual:
Expected:
I get the following error when running the basic test on the nightwatch home page
Reading reports directory...
/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1023
throw err;
^
SyntaxError: The keyword 'package' is reserved (160:10)
at Parser.pp.raise (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1021:13)
at Parser.pp.parseIdent (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:732:240)
at Parser.pp.parseBindingAtom (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1136:19)
at Parser.pp.parseVar (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1987:20)
at Parser.pp.parseVarStatement (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1883:8)
at Parser.pp.parseStatement (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1683:19)
at Parser.pp.parseBlock (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1938:21)
at Parser.pp.parseStatement (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1689:19)
at Parser.pp.parseFor (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1961:20)
at Parser.pp.parseForStatement (/usr/local/lib/node_modules/nightwatch-html-reporter/node_modules/jade/node_modules/with/node_modules/acorn/dist/acorn.js:1770:17)
I have installed nightwatch by running npm install nightwatch -g which installs nightwatch v0.5.36
A completely nonessential detail for the new design,
indeed I must apologise for mentioning it at all! ^_^
The new grey color is really depressing compared to the
wonderful green color that was used previously to mark
passing tests.
I've just changed it locally, but it does make the screenshots
on the gitpage look much less exciting!
hi,
I am looking for HTML reporter which can append test cases not to override the previous test cases.
presently I'm working on jest with Frisby's scripts and want HTML reporter of my scripts which can append all test result. So it is configurable with jest or not?
thanks
I want to print a custom message before an assertion to understand what the assertion is for. Can you please tell me how can I get that custom message also displayed in the report?
Hey guys, I followed the steps as described on the readme but im having the following error when trying to read the report folder here is an output of the error:
Reading reports directory...
events.js:160
throw er; // Unhandled 'error' event
^
TypeError: Cannot read property 'name' of undefined
these are the libraries im using for nightwatch:
"nightwatch": "0.9.14", "nightwatch-cucumber": "8.2.6",
I have a test that fails on purpose, but the array of screen shots shown me empty.
Why does this happen?
{ message: 'H2 [Present - H2 element 'Opciones de personalización' is present]',
stacktrace: ' at Object. (/var/www/testing/tests/CorporateData/Domains/domain.js:104:19)\n at HttpRequest. (/var/www/testing/node_modules/nightwatch/lib/index.js:337:20)\n at HttpRequest.emit (events.js:110:17)\n at HttpRequest. (/var/www/testing/node_modules/nightwatch/lib/index.js:365:15)\n at HttpRequest.emit (events.js:118:17)\n at IncomingMessage. (/var/www/testing/node_modules/nightwatch/lib/http/request.js:155:16)\n at IncomingMessage.emit (events.js:129:20)\n at _stream_readable.js:908:16\n at process._tickCallback (node.js:355:11)',
failure: 'Expected "found" but got: "not found"',
screenshots: [],
isFailure: 'Expected "found" but got: "not found"' } ],
isFailure: true }
the path is set in the gruntfile.js:
test_settings : {
default : {
launch_url : "www",
selenium_host : "127.0.0.1",
selenium_port : 4444,
silent: true,
screenshots : {
enabled : true,
on_failure : true,
on_error : true,
path : "./"
},
desiredCapabilities: {
browserName: "chrome",
"platform": "LINUX",
javascriptEnabled: true,
acceptSslCerts: true
}
},
firefox : {
desiredCapabilities: {
browserName: "firefox",
"platform": "LINUX",
javascriptEnabled: true,
acceptSslCerts: true
}
},
},
Thanks!!
Should that be .../index.pug instead?
With the following in the nightwatch config:
"test_workers": {
"enabled": true,
"workers": "auto"
}The html report is created when the first worker finishes, and thus misses a lot of data.
Without workers, the report is created for all tests, but does not include the steps (as they're not in the XML)
Hi James,
if I execute nightwatch tests in parallel the report html will be overwritten, and the report html contains only result of the last testsuite. Is this a configuration issue, or it is not implemented in the nightwatch-html-reporter.
I would introduce a parameter as a constructor of HtmlReporter, for example: separatedReport, which can be true or false. If it is true, every testsuite would had its own report file. For example: the reporter.js would contain something like this:
logger.setLevel(opts.logLevel);
this.fn = function(results, done) {
var testName = '';
console.log(results.modules);
var oneObject = Object.keys(results.modules);
console.log(oneObject);
oneObject.forEach(function (nextLevel) {
testName = nextLevel;
})
if (opts.separateReportPerSuite) {
opts.reportFilename = opts.reportFilename.replace(/\.html/,'') + '-' + testName.replace(/\\/g,'-');
}
opts.reportFilename = opts.reportFilename + '.html'
Would you approve this? Do you have any other idea?
Thank you
if any one of assertion fail then reporter don't generate report file instead print [object Object] in bash. This issue occur in nightwatch latest 0.6.5 and 0.6.4
I just installed the reporter module and wanted to test it out. I noticed that every time the script is run, at time of completion when it generates the HTML report, the script hasn't ended correctly. I have to press Control + C to end it. Not sure what info you need but let me know and I'll send it.
node --version = 10.13.0
npm --version = 6.4.1
nightwatch --version = 0.9.21
Also of note, I am passing other parameters in as well, such --env, --tag, --reporter (in that order if it matters).
it would be useful if there's option to see screenshot taken when test failed.
When running tests with nightwatch using multiple browsers simultaneously, the exported report doesn't combine all the output reports into one. Currently only one of the reports gets provided in html.
There should be a way that this nightwatch-html-reporter automatically fetches all reports for each of these browsers and combines it into one report.
When we run tests in parallel we may get report in two ways
Can we have a flag to have single file even when we run tests in parallel (we executed total 20 tests.js and able to generate one html report containing result for all 20 tests.js
I'm using this reporter in our continuous integration build server, but the report only works when opened from file system. So when the build server grabs the report to make it available after a build, the report is unable to load the script and css as they are referenced by file protocol instead of http. As it is, the reports are unusable for us.
I am using nightwatch 0.6.4 & latest nightwatch-html-reporter. When running test cases if one of test fail reporter don't create report file instead it show [object Object] on nightwatch 0.6.4 and on 0.6.5 it print some error.
Would be fun if we could put our own jade theme file anywhere we want and specify it in the opts, don"t you think ?
For example i am running the tests on chrome,ie and mozilla simultaniously and in the report i see the test suite name for 3 times and i cannot diferantiate which one is IE,Chrome or Firefox.
Can we add the environment were the tests were runned?
Thanks!
Example below :
Nightwatch Results
Show/Hide Successes
✓ pom
pom.tests
✓ Open the website
✓ Go to site and login
✓ Go and logout
✓ pom
pom.tests
✓ Open the website
✓ Go to site and login
✓ Go and logout
Without the callback the plugin does not work as intended.
... Error while running [Test / reporter]: TypeError: done is not a function ...
My XML file has the following content:
<testcase name="Login works" time="3.389" assertions="4">
</testcase>
<testcase name="Login fails" time="3.394" assertions="4">
</testcase>
but the generatedReport.html file shows it has completed some test and second it skipped.
Created in error
HTML report is not correct if I use nightwatch 0.6.4. Assertions always appear failed on report and if i click on test case it show me completed and skip instead of test case description.
Hi,
How can I customize the html report to add more information? for example the screenshots..
Thanks
Some of the themes refer to script or style files from the web, see below. Is it possible to add them as npm dependencies to the reporter, and themes will refer to local path in node_modules? Internet may not be available to all build environments.
https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js
https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css
Thanks
If yes, how. I am not able to see any option regarding it.
I am using cli to generate html report.
Directory structure
-reports
-testname folder
- screenshot saved
The xml output of nightwatch support two kinds of test failures, errors, and failures. An error is thrown if for example you use click on an element that does not exist. The current behavior of the reporter is to generate the report and display the testsuite as a success - which seems misleading.
browser.click('some element that does not exist')The xml output looks like this
<testsuite errors="1" failures="0" hostname="" id=""
name="mytest" package="tests" skipped="0"
tests="6" time="" timestamp="">
...
<system-err>
Unable to locate element: "foo" using: css selector
</system-err>
</testsuite>> nightwatch-html-reporter -d .\myfolder\results
results in
Error: Command failed: File not found - '*.xml'
at ChildProcess.exithandler (child_process.js:648:15)
at ChildProcess.emit (events.js:98:17)
at maybeClose (child_process.js:756:16)
at Socket.<anonymous> (child_process.js:969:11)
at Socket.emit (events.js:95:17)
at Pipe.close (net.js:465:12)
though the folder contains e.g. test.xml file looking like this
<?xml version="1.0" encoding="UTF-8" ?>
<testsuites name="tests"
errors="0"
failures="1"
tests="1">
<testsuite errors="0" failures="1" hostname="" id=""
name="Items should be filterable and first matches rendered in bold - test 2" package="tests" skipped="0"
tests="1" time="" timestamp="">
<testcase name="Assertion failed: 2 == 3">
<failure message="Expected "3" but got: "2""></failure>
</testcase>
</testsuite>
</testsuites>
Can we have symbol "!" for skipped test cases ,instead of "V"?
Hi,
could you please change screenshots links as a relative paths to output file
(e.g. in cover theme) it would be valid after executing in jenkins.
Thanks a lot!
Viewing on Chrome, nothing happens.
Is this like an expand/collapse all function or will it keep the failures collapsed? If it's the latter, can you add an expand all function?
A nightwatch test (version 5.32) crashes the reporter on a test containing
"browser.assert.equal(2,3)"
Everything works as expected if no assertions fails. Fails on waitForElement works fine. The xml file is generated as expected regardless.
I expect it might be a but related to a newer version of Nightwatch?
The stacktrace:
An error occurred while running the tests:
TypeError: ...\node_modules\nightwatch-html-reporter\lib/themes/default/index.jade:22
20| if testCase.isFailure
21| div.name= testCase.name + " " + testCase.failureMessage
> 22| each sshot in testCase.screenshots
23| div.screenshot
24| img(src= sshot)
25| else
Cannot read property 'length' of undefined
at jade_debug.unshift.lineno (eval at <anonymous> (...\node_modules\nightwatch-html-reporter\node_modules\jade\lib\jade.js:179:8), <anonymous>:125:31)
at $$l (eval at <anonymous> (...\node_modules\nightwatch-html-reporter\node_modules\jade\lib\jade.js:179:8), <anonymous>:162:4)
at eval (eval at <anonymous> (...\node_modules\nightwatch-html-reporter\node_modules\jade\lib\jade.js:179:8), <anonymous>:269:4)
at eval (eval at <anonymous> (...\node_modules\nightwatch-html-reporter\node_modules\jade\lib\jade.js:179:8), <anonymous>:489:4)
at eval (eval at <anonymous> (...\node_modules\nightwatch-html-reporter\node_modules\jade\lib\jade.js:179:8), <anonymous>:501:22)
at res (...\node_modules\nightwatch-html-reporter\node_modules\jade\lib\jade.js:180:38)
at Object.exports.writeReport (...\node_modules\nightwatch-html-reporter\lib\reportGenerator.js:15:13)
at Object.fn [as reporter] (...\node_modules\nightwatch-html-reporter\lib\nightwatch-html-reporter.js:41:19)
at ...\node_modules\nightwatch\lib\runner\run.js:611:36
at ...\node_modules\nightwatch\lib\runner\reporters\junit.js:70:11
When I run some tests and they fail, nightwatch-html-reporter shows the assertions that passed but does not display the failed assertion.
Here's the output from nightwatch-html-reporter
Here's the console output from nightwatch
Outlook doesn't display the styles correctly when embedding this report html in the body of an email. This is because of the way how the styles are done which isn't supported by Outlook and some email apps.
Automatic emailing of these reports isn't necessary used by most people but we are using it in that way.
I'm using the following to generate reports from the command line:
nightwatch-html-reporter -d ./reports/junit -o ../html/SummaryReport.html -t compact
The report was being create in the "html" folder. This morning, I ran "npm update" on my project and now the report gets created in the "junit" folder instead of the "html" folder.
What happened and how can I fix it? Thanks.
I like to group my tests in groups and use the folder hiearachy to organize them.
This is supported by nighwatch through the --group flag which me to run one or
specific test folder or group of tests.
I imagine the benefit of this is a better understanding of what fails,
e.g. if the group network-connection-tests is the only group with failure
it would be easier to determine the problem.
It would be nice if this was included in one of the themes as an additional hiearachy.
The open package last update was 5 years ago O.o
I suggest using opn instead for the workaround. The main reason is this critical vulnerability is failing the production build.
I have used default and compact themes and there I can't even see that there are System errors, but in cover theme it states that there are System errors but I can't see the inner text of the error in the report.
@jls is there a way to use it with another reporter such as slack ? Or it is a nightwatch limitation itself ? For now, if I specify both in globals the second one is not getting called.
Hi awesome reporter for nightwatch.
I like the 'Cover' theme you have done. but it dosen't include screenshots of errors, like it does in your default theme.
I'd like to know if it's possible to use this with the built-in "--report" option of Nightwatch. If yes, what is the file I should refer to?
I'm using grunt-nightwatch 0.5.2, with nightwatch 0.8.14.
The XML file generated by my tests doesn't seem to include any information relating to successful assertions, only failures. Looking at the jUnit XML schema, this seems intentional.
However, when I run:
nightwatch-html-reporter -d /path/to/test/reports
The HTML file also doesn't include any entries relating to successful assertions - there is no .package.success div.
What am I not understanding here? Where does nightwatch-html-reporter get information about successful assertions from? Should it be in the XML? It's not, and I'd expect it to be, but the schema seems to suggest otherwise.
This issue is exclusive to 0.2.4 - when I reverted to 0.2.3, everything was fine.
Even when all the constituent test cases in a test suite pass, that test suite is still red, not green as it is supposed to be.
I am using test functions [say login()], which has some assertions and wait functions, inside /pages/utils.js, and using these function inside various tests under the tests folder
'login' : function(Browser){
utils(browser).login();
}
While running the test the assertions are ran and the test is passed when i generate a report, it just shows 'login' passed, there is no info about the assertions passed.
In the current themes, e.g. compact, it would be really nice overview to have a hiearachy of collapsable groups, files, steps, and assertions like follows.
> navigation [success]
> filtering [failure]
>> filter.foo [success]
>> filter.bar [failure]
>> >> step.1 [sucess]
>> >> step.2 [sucess]
>> >> step.3 [failure]
>> >> >> assert 1 [success]
>> >> >> assert 2 [failure]
>> filter.bas [success]
>> testfile.4 [success]
>> testfile.5 [success]
> options [success]
> group.4 [success]
> group.5 [success]
What do you think? ^_^
I do not see an option to include the total test suite run time in the html reporter file. Will this feature be added? If not, are there any docs that I can refer to do create a new theme that will include this?
A Combined HTML report for all the suite runs, after the execution of the parallel tests would be of great help, especially while integrating reports to CI/CD pipeline. As one report per suite is created right now, we need to build a hack to combine them all. I see this has been referenced in #21 . Reopening for another look.
Can anyone guide or share code to customise the nightwatch report generated from XML in the form of tabular html report having the columns as Test suite,Test cases ,Assertions and execution status adding one row for each test suite.
Nightwatch, and the generated xml files support three kind of test "levels" which would be nice to recognize in the html-output. First tests are grouped in files, then tests are grouped in named keys (called steps in nightwatch) and then assertions. At the moment the reporter only supports the file, and assertion levels.
I suggest the addition of a new html "class/level" such that all the three test levels are distinquished.
Mind at the moment steps are not handled correctly in the reporter as each step overwrites the previous in TestSuite.js.
Sample case
MyTestSuite1.js
module.exports = { "step one" : function (browser) { browser .url("http://www.google.com") .waitForElementVisible('body', 1000) .setValue('input[type=text]', 'nightwatch') .waitForElementVisible('button[name=btnG]', 1000) }, "step two" : function (browser) { browser .click('button[name=btnG]') .pause(1000) .assert.containsText('#main', 'The Night Watch') .end(); } };
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.