tapjs/tap-mocha-reporter

Format a TAP stream using Mocha's set of reporters

@tapjs

Workspace for node-tap development.

Dev Commands

Do this at least once to get everything set up and ready to go:

npm run bootstrap

(Note: npm install will not work until you do this, because the generated TypeScript eats its own tail.)

---

Build the test class (required after any plugin or core changes):

npm run build

---

Any other builds:

npm run prepare -w src/{whatever}

---

After adding or removing workspaces:

npm i

---

Run all tests in all workspaces:

npm test

---

Run all tests, saving snapshots:

npm run snap

---

Build and serve docs:

npm start

Contents

• tap The main entry point module, which sets up the root test runner and exposes an alias to the cli runner.
• tap-parser The module that parses TAP
• @tapjs/core Most of the basic moving parts of tap
• tap-yaml Thin wrapper around YAML and yaml-types for consistent handling of JavaScript values in YAML diagnostics.
• @tapjs/test The plugin-ified Test class.
• @tapjs/config Handling config files, command line interface parsing, environment variables, and validation
• @tapjs/run The command line runner
• tcompare The library that does comparison and object formatting (use heavily by @tapjs/asserts methods).
• @tapjs/stack Library for capturing stack frames, the descendant of stack-utils.
• @tapjs/processinfo The library that tracks process information and code coverage (hosted outside the monorepo, because it can't be tested by a version of tap that uses itself without bootstrap paradoxes)
• default plugins:
  • @tapjs/typescript Adds typescript support, and the --typecheck config option
  • @tapjs/before Adds t.before()
  • @tapjs/before-each Adds t.beforeEach()
  • @tapjs/after Adds t.after() and t.teardown() (which are the same thing now)
  • @tapjs/after-each Adds t.afterEach()
  • @tapjs/spawn Adds t.spawn()
  • @tapjs/stdin Adds t.stdin()
  • @tapjs/asserts Adds all the various assertion methods, like t.equal(), t.match(), etc.
  • @tapjs/snapshot Adds t.matchSnapshot()
  • @tapjs/fixture Adds t.testdir() features
  • @tapjs/mock Adds t.mockRequire() and t.mockImport()
  • @tapjs/intercept Adds t.intercept() and t.capture() functionality (sort of like a very scaled-down minimal form of Sinon. If you want more, consider using the @tapjs/sinon plugin.)
  • @tapjs/filter Adds t.only() and support for the --grep and --only cli options.
• optional plugins:
  • @tapjs/nock Optional plugin providing t.nock() method (descendent of @npmjs/tap-nock, and a significant inspiration for tap's plugin architecture)
  • @tapjs/clock Optional plugin providing a clock-mock object on t.clock
  • @tapjs/sinon Optional plugin providing a Sinon sandbox at t.sinon that automatically restores at the end of the test.
  • @tapjs/esbuild-kit Optional plugin that loads TypeScript using @esbuild-kit/cjs-loader and @esbuild-kit/esm-loader instead of ts-node.
• other stuff:
  • npm-init-template A library for more easily creating npm init packages. This will move out as soon as this version of tap is published.
  • @tapjs/create-plugin An npm init library facilitating npm init @tapjs/plugin to create new plugins.

Bootstrap and skipLibCheck

Run npm run bootstrap to build the @tapjs/test module with the default set of plugins, so that the other libraries can build properly. (This only has to be done once, unless the build script or set of default plugins are changed, of course.)

Because there's a bootstrapping cycle between @tapjs/core, @tapjs/test, and all of the plugins, they MUST use skipLibCheck: true in their tsconfigs. It should not be used in other packages.

tap-mocha-reporter's Issues

Update `diff` dependency to >= 3.5.0

There is a GitHub warning for this dependency. Can it be updated? I realize 3.5.0 is a couple major versions different from what is currently used by this package.

Faster diff or ability to disable it in the classic reporter

When Tap compares relatively big objects and fails, outputting a diff can take a crazy amount of time, sometimes freezing Tap completely (e.g. when comparing buffers of files that take more than several megabytes).

Options I see here:

• Heuristic that automatically disables diff when comparing big objects
• Finding out why diff is so slow and making it faster
• Ability to disable diff in the classic reporter completely
• Even better, ability to disable diff per assertion

Please add a help option

bastien@portable2015-bastien:~/Documents/Personnel/soft/debian/js/node-tap-mocha-reporter/node-tap-mocha-reporter/debian/patches$ cat 0003-Support-a-help-option.patch 
From 94f8c101f7a55e7ad7d7df7a77db9277ffdc280d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Bastien=20ROUCARI=C3=88S?= <[email protected]>
Date: Tue, 22 Aug 2017 14:46:56 +0200
Subject: Support a --help option

forwarded: yes, https://github.com/lyroyce/mocha-tap-reporter/issues/5
bug: https://github.com/lyroyce/mocha-tap-reporter/issues/5
---
 index.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/index.js b/index.js
index 46c5663..8b2fc07 100755
--- a/index.js
+++ b/index.js
@@ -99,7 +99,7 @@ reporter.  (Note that some reporters write to files instead of stdout.)
 
 if (require.main === module) {
   var type = process.argv[2]
-  if (!type)
+  if (!type || type == '--help' || type == '-h')
     return usage()
 
   process.stdin.pipe(new Formatter(type))

xunit timestamp should be ISO8601

JUnit schema requires ISO8601 timestamps (https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd#L165).

Current implementation of the xunit reporter uses RFC1123 date stamps via toUTCString() (https://github.com/tapjs/tap-mocha-reporter/blob/master/lib/reporters/xunit.js#L65).

This could be fixed by using toISOString() instead.

Note: this bug exists in the mocha reporter as well
Note 2: this would be a breaking change

It fails if the current path includes spaces

The reporter does not like paths with spaces (which are quite common on macOS):

ilg@wksi xpm-js.git % pwd
/Users/ilg/My Files/WKS Projects/xpack.github/npm-modules/xpm-js.git

> tap --timeout 300 --disable-coverage --allow-empty-coverage -R tap-mocha-reporter -r classic run "tests/tap/*.js"

/bin/sh: /Users/ilg/My: No such file or directory
node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: write EPIPE
    at __node_internal_captureLargerStackTrace (node:internal/errors:478:5)
    at __node_internal_errnoException (node:internal/errors:608:12)
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:905:11)
    at Socket._write (node:net:917:8)
    at writeOrBuffer (node:internal/streams/writable:391:12)
    at _write (node:internal/streams/writable:332:10)
    at Socket.Writable.write (node:internal/streams/writable:336:10)
    at TAP.[emitData] (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/minipass/dist/mjs/index.js:790:24)
    at TAP.emit (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/minipass/dist/mjs/index.js:750:37)
    at TAP.emit (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/base.js:542:27)
    at TAP.write (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/minipass/dist/mjs/index.js:367:18)
    at TAP.write (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/base.js:443:22)
    at TAP.write (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/tap.js:184:22)
    at TAP.#online (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/base.js:463:21)
    at Parser.<anonymous> (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/base.js:292:49)
    at Parser.emit (node:events:513:28)
    at Parser.version (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/tap-parser/dist/esm/index.js:408:18)
    at Parser.parse (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/tap-parser/dist/esm/index.js:783:18)
    at Parser.write (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/tap-parser/dist/esm/index.js:316:18)
    at TAP.#process (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/test-base.js:652:29)
    at TAP.plan (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/core/dist/esm/test-base.js:279:26)
    at TAP.plan (file:///Users/ilg/My%20Files/WKS%20Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/test/test-built/dist/esm/index.js:55:23)
    at executeTestSuite (/Users/ilg/My Files/WKS Projects/xpack.github/npm-modules/xpm-js.git/node_modules/@tapjs/run/src/execute-test-suite.ts:137:7)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
}

 *  The terminal process "/bin/zsh '-l', '-c', 'npm run test-tap'" terminated with exit code: 1. 
 *  Terminal will be reused by tasks, press any key to close it. 

Error when used with global document provided by jsdom

We are setting up a global document, as recommended by enzyme to support some of our tests - using nyc's require configuration to load it before any of the tests start. Unfortunately, this causes tests errors to be reported as

ReferenceError: location is not defined
    at Object.exports.stackTraceFilter ({project root}/node_modules/tap-mocha-reporter/lib/utils.js:651:9)
...

This happens because lib/utils.js tests to see if it is in node or a browser by looking for a defined global document.

is = typeof document === 'undefined'
      ? { node: true }
      : { browser: true }

This is reasonable, but can lead to errors like the one we are experiencing where a global document has been mocked.

HTML reporter: document is not defined

specifying using the HTML reporter either by:
tap -R html tests/*.js
or
tap -- tests/*.js | ./node_modules/.bin/tap-mocha-reporter html

causes:
ReferenceError: document is not defined

Although if using the -R option in the first example it is a silent error.

I realize this is bc the node environment doesn't have a DOM.

logging for posterity to the make mochalike stuff much more Mocha-like

edit: submitted before i was ready.. so lots of formatting and additions

No version 3.0.7

You forget to add new version

README is incorrect about programmatic use case

The README seems to be incorrect to me about how to use tap-mocha-reporter programmatically.

Instead of

fs.createReadStream('saved-test-output.tap')
  .pipe(TSR('dot'))

it should be

fs.createReadStream('saved-test-output.tap')
  .pipe(new TSR('dot'))

Not supplying format type throws EPIPE

The README states that you can pipe to tap-mocha-reporter without specifying an argument, but this displays usage and throws EPIPE for me.

I'm using the same files as coverage-example from tap to test this.

Without an argument to tap-mocha-reporter:

$ ./node_modules/.bin/tap test/*.js | ./node_modules/.bin/tap-mocha-reporter
Usage:
  tap-mocha-reporter <type>

Reads TAP data on stdin, and formats to stdout using the specified
reporter.  (Note that some reporters write to files instead of stdout.)

Available format types:

classic doc dot dump html htmlcov json
jsoncov jsonstream landing list markdown
min nyan progress silent spec tap xunit

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: write EPIPE
    at errnoException (net.js:905:11)
    at Object.afterWrite (net.js:721:19)

And as soon as I supply an argument, it works:

$ ./node_modules/.bin/tap test/*.js | ./node_modules/.bin/tap-mocha-reporter classic
total ................................................. 14/14
total ................................................. 4/4

  4 passing (188ms)

  ok

Compatibility with Node.js native TAP output

It would be amazing to have a good reporter that has high compatibility with Node.js native TAP output via the new Node.js test core module.

This package gets the closest I've found so far, but with a couple of limitations:

• Error stacks do not get displayed when piping the tap stream.
• Skipped tests are not picked up
• Test "TODO"s are not picked up

I understand node-tap is the target, but I can't seem to find an alternative, and it would be great to have a high quality formatter for these use cases.

Process exiting with non-zero even though all tests pass

Using the coverage-example tests from tap – I'm getting npm ERR! Test failed which I assume means that tap-mocha-reporter has exited with a non-zero code, even though the tests all pass fine in tap and no (specific) error is displayed in the console.

$ npm test

> [email protected] test /Users/michael/github/test-project
> tap test/*.js | tap-mocha-reporter classic

total ................................................. 14/14
total ................................................. 4/4

  4 passing (182ms)

  ok
npm ERR! Test failed.  See above for more details.

Just running the tests without piping to tap-mocha-reporter does not have the same issue.

$ npm test

> [email protected] test /Users/michael/github/test-project
> tap test/*.js

ok test/bar.test.js ..................................... 4/4
ok test/baz.test.js ..................................... 6/6
ok test/foo.test.js ..................................... 4/4
total ................................................. 14/14

ok

Perhaps this is why #1 is acting weirdly? tap-mocha-reporter is encountering some sort of error that it's not displaying here and exiting with non-zero?

xunit reporter escaping is broken

While working on node-snmpjs I upgraded tap (from v0.4!) to gain support for reporters and coverage. Unfortunately it looks like some tests generate non-ascii bytes in test names, which causes the xunit reporter to emit invalid XML.

The fix for the mocha xunit reporter is here: mochajs/mocha@9f403bf#diff-50e3aa130a4f97a42ee2cf111c7b1d9d

Here's the test in question:
https://github.com/joyent/node-snmpjs/blob/master/test/protocol/data.test.js#L359-L385

And the invalid XML output:

<testcase classname="./test/protocol/data.test.js octet string encode" name="encode foobar" time="0"/>
<testcase classname="./test/protocol/data.test.js octet string encode" name="encode ��AL� M" time="0"/>
<testcase classname="./test/protocol/data.test.js octet string encode" name="encode foo bar baz quux" time="0"/>
<testcase classname="./test/protocol/data.test.js octet string encode" name="encode �$Y�����" time="0"/>
<testcase classname="./test/protocol/data.test.js octet string encode" name="encode fööbå®" time="0"/>

Furthermore, the reserved-XML character escaping is even broken. It only replaces the first instance of each char:

// currently implemented:
exports.escape = function(html){
  return String(html)
    .replace('&', '&')
    .replace('"', '"')
    .replace('<', '&lt;')
    .replace('>', '&gt;');
}

to wit:

'bbb'.replace('b', 'o'); // results in 'obb' !
'bbb'.replace(/b/g, 'o'); // results in 'ooo'

The proper form should use /expr/g:

exports.escape = function(html){
  return String(html)
    .replace(/&/g, '&')
    .replace(/"/g, '"')
    .replace(/</g, '&lt;')
    .replace(/>/g, '&gt;');
}

Add mochawesome reporter

How to do that? mochawesome is installed, but tap-mocha-reporter doesnt sees it

classic: failure of suite-wrap-up test point is reported very silently

This tap output: https://gist.github.com/isaacs/dd55fec6402bf64585cd
leads to all passing suites, and no errors reported, but then the total is reported as 227/229 with 2 mysterious failures.

Time to switch to ESM?

in order to update this ESM only dep

tap-mocha-reporter/package.json

Line 26 in 5c8846f

"escape-string-regexp": "^2.0.0",

Then it would be grate if we made the switch and started to convert this mocha reporter to ESM-only, thoughts?

Diff colors, configuration, streaming...

If you're not running a terminal with 256 color support (tmux in default config, for example), the default reporter's default diff configuration (actually, by the source, it's hard-coded in even though there are options??) is not only hard on the eyes, but impossible to read. You just get bright-red-on-bright-red and bright-green-on-bright-green lines. It would be nice if this behaved properly without requiring source diving to figure out what's going on, and nicer still if it was configurable.

It's additionally confusing that TAP's reporters appear to be using Mocha's API, but aren't actually respecting the values from the base reporter itself, which specifies entirely different colors.

I'm not a fan of Mocha's reporter API at all, however: given that mocha reporters output to console.log, lots of problems arise (such as log or error messages cropping up in xml reporter output). Since TAP maintains a set of streams, it would be excellent if this separation could be carried forward into the reporter API, allowing for discretely piped input into the reporters themselves and possibly multiple reporters.

I wouldn't be averse to contributing some code on this count, though I could use rather a lot of help refactoring / understanding how to hook into tap's output in order to consume it in such a manner. I suppose it kind of breaks the TAP mentality to have the reporter tied to the output this way, but to be honest, once you've merged everything into a single output stream it can be difficult or impossible to un-merge it all again! (On the other hand, given that the streams are merged by the runner and not the "test anything protocol" itself, perhaps it is within the tap module's purview after all?)

One last thing regarding the diffs - why is null printed as [null]? This is very confusing.

timeouts reported badly

It took me a while to figure out the problem with signal-exit last night was timeout related, because timeout issues are reported in a somewhat cryptic way:

  not ok missing plan
    results:
      ok: false
      count: 0
      pass: 0

  signal-exit
  not ok missing plan
    results:
      ok: false
      count: 1
      pass: 1

I think this is partially because we don't report the last assert when a suite completes:

    // no need to print the trailing assert for subtests
    // we've already emitted a 'suite end' event for this.

If I actually do emit the last test, I found the output less cryptic:

      if (suite) {
        emitTest(this, result)
        runner.emit('suite end', suite)
        return
      }

// vs. the original.

      if (suite) {
        runner.emit('suite end', suite)
        return
      }

This gave the output:

  not ok missing plan
    results:
      ok: false
      count: 0
      pass: 0

  signal-exit
  not ok missing plan
    results:
      ok: false
      count: 1
      pass: 1

  not ok timeout
  not ok ./test/signal-exit-test.js
    timeout: 5000
    file: ./test/signal-exit-test.js
    arguments:
      - ./test/signal-exit-test.js
    signal: SIGKILL
    results:
      ok: false
      count: 1
      pass: 0
      fail: 1
      plan:
        start: 1
        end: 0
        skipAll: true
    command: /Users/benjamincoe/.nvm/versions/io.js/v2.0.2/bin/iojs

Trying to get in touch regarding a security issue

Hey there!

I'd like to report a security issue but cannot find contact instructions on your repository.

If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.

Thank you for your consideration, and I look forward to hearing from you!

(cc @huntr-helper)

Allow multiple reporters

Can you add the feature to add multiple reporters?
For example: spec and xunit.
Meaning, writing spec output and xunit xml file.
There are a few reporters that are doing this, but the tap-mocha-reporter used by tap doesn't allow it.

https://www.npmjs.com/package/spec-xunit-file
https://www.npmjs.com/package/tap-xunit-file