semaphoreci/test-results

test-results's Issues

Add proper README

Combining multiple files fails to upload raw xml

When test results CLI is being used to publish multiple files:

test-results publish file1.xml file2.xml file3.xml

Generated junit.json file is being correctly sent to artifact storage, however due to the fact, that xml files are not really merged, CLI tries to upload 3 files, and fails on 2nd one (due to file being present in artifact storage).

I will think about some solution that we can implement and update the issue.

Add a parser for PHPUnit JUnit files

Publishing PHPUnit JUnit files results in empty JSON files, no matter which parser you use.
This is probably because PHPUnit creates files with nested testsuite tags, like this:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" tests="3" assertions="13" errors="0" failures="0" skipped="0" time="0.001504">
    <testsuite name="" file="" tests="2" assertions="11" errors="0" failures="0" skipped="0" time="0.001226">
      <testcase name="" class=" file="" line="16" assertions="4" time="0.001150"/>
      <testcase name="" class="" classname="" file="" line="39" assertions="7" time="0.000076"/>
    </testsuite>
    <testsuite name="" file="" tests="1" assertions="2" errors="0" failures="0" skipped="0" time="0.000278">
      <testcase name="" class="" classname="" file="" line="15" assertions="2" time="0.000278"/>
    </testsuite>
  </testsuite>
</testsuites>

(class and filesnames are redacted)

Parse durations that use scientific notation

Some JUnit exporters report timestamps in scientific notation, which results in the following warning:

* Duration parsing failed: time: unknown unit "e-" in duration "8.10623168945312e-05s"

Test

Testing Slack integration.

`test-results publish` returns exitcode zero if file is not found

This is a problem since the CI-pipeline (where most people will use this), will show that the task finished correctly, which is not true.
Returning a non-zero exit code will fix this, and also conforms more to the meaning of exit codes.

To reproduce the issue:

$ test-results publish path/to/nonexisting/file.xml
* Input file read failed: stat .junit/unittests.xml: no such file or directory
$ echo $?
0

Update integrations tests for workflow

I had to remove the workflow integration test for now, we need to figure it out how to handle the case where the parser does not implement IsApplicable() and generic can't handle properly the input.

IsApplicable() is not implemented for embedded and phpunit, which breaks the workflow integration test.

CC @skipi