aaswtest/newman-reporter-junitxray

A Newman Junit reporter designed simplify output for use with XRAY for JIRA

newman-reporter-junitxray's Issues

Error getting test execution time

There is an error getting and reporting into Xray the execution time of a test. Next I will try to explain the best I can do.

Executing a test suite with newman and junitxray report, I got the next xml report

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Arlo-mocks" tests="2">
  <testsuite id="d8261b5c-c114-4bcf-b2b1-072e766ee614" name="common-service / suite for xray" tests="1" timestamp="2020-10-27T13:20:18.428" time="0.022" failures="0" errors="0">
    <testcase classname="ArloMocks" name="getConnected" time="0.000"/>
  </testsuite>
</testsuites>

As you can see, time is well reported in the testsuite node, but in the testscase node, the time value is 0.000. Time value in the testscase node is the one that Xray uses to inform the "Duration" period in the test execution details. Show you this in the next picture

Reading the code I found that the value for this field is obtained in index.js line 117

executionTime = _.get(testExecution, "response.responseTime") / 1000 || 0;

The problem is that the object response is always empty (in all my cases), even when the service tested has a response. May be some change from Newman framework.

So the solution I get is to obtain the value from time in testcase node as the same way as doing for the time value in testsuite node. Doing that the next result is obtained

junitxray report

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Arlo-mocks" tests="2">
  <testsuite id="d8261b5c-c114-4bcf-b2b1-072e766ee614" name="common-service / suite for xray" tests="1" timestamp="2020-10-27T13:20:18.428" time="0.022" failures="0" errors="0">
    <testcase classname="ArloMocks" name="getConnected" time="0.022"/>
  </testsuite>
</testsuites>

xray report

Global variable getting override while using loop

global = xml.create("testsuites", { version: "1.0", encoding: "UTF-8" }); global.att("name", collection.name);

Issue:
I need to run multiple collection using newman. On each collection the reports were generated at the same time the global gets overrided with xml. This cause the next collection fail at newman SecureFS class.