Giter Site home page Giter Site logo

benc-uk / k6-reporter Goto Github PK

View Code? Open in Web Editor NEW
374.0 8.0 92.0 328 KB

Output K6 test run results as formatted & easy to read HTML reports

License: MIT License

Go 12.75% Makefile 5.65% Dockerfile 1.84% JavaScript 20.88% EJS 58.88%
html-report k6 load-testing

k6-reporter's Introduction

K6 HTML Report Exporter v2

🔥 Note.

This a complete rewrite/overhaul of the existing report converter, now written in JavaScript to be integrated into the test. This is a more elegant method than the offline conversation process. The previous code has been moved to the 'archive' folder.

The report will show all request groups, checks, HTTP metrics and other statistics

Any HTTP metrics which have failed thresholds will be highlighted in red. Any group checks with more than 0 failures will also be shown in red.

Usage

This extension to K6 is intended to be used by adding into your K6 test code (JavaScript) and utilizes the handleSummary callback hook, added to K6 v0.30.0. When your test completes a HTML file will be written to the filesystem, containing a formatted and easy to consume version of the test summary data

To use, add this module to your test code.

Import the htmlReport function from the bundled module hosted remotely on GitHub

import { htmlReport } from "https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js";

Note. Replace main with a version tag (e.g. 2.2.0) to use a specific version

Then outside the test's default function, wrap it with the handleSummary(data) function which K6 calls at the end of any test, as follows:

export function handleSummary(data) {
  return {
    "summary.html": htmlReport(data),
  };
}

The key used in the returned object is the filename that will be written to, and can be any valid filename or path
Note. This is a change in the v2.1.1 release

The htmlReport function accepts an optional options map as a second parameter, with the following properties

title    string  // Title of the report, defaults to current date

Multiple outputs

If you want more control over the output produced or to output the summary into multiple places (including stdout), just combine the result of htmlReport with other summary generators, as follows:

// This will export to HTML as filename "result.html" AND also stdout using the text summary
import { htmlReport } from "https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js";
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export function handleSummary(data) {
  return {
    "result.html": htmlReport(data),
    stdout: textSummary(data, { indent: " ", enableColors: true }),
  };
}

Screenshots

main report screenshot

another report screenshot

k6-reporter's People

Contributors

benc-uk avatar patricklevy avatar turim-thiago avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

k6-reporter's Issues

How to capture response error and add to html report.

I would like to know if the handleSummary function can be customized to display details of failed requests. Being able to read the details of failed requests in the report would be very helpful.

I have tried several times and discovered that the handleSummary function cannot utilize other functions and parameters. How can I obtain the details of failed requests after running the script in k6? Thank you.

groups and tags

I'm trying to group two requests (though it will give me duration for each request - k6 sum it to group duration :()

But anyway, I'm not sure if this error coming from k6-reporter:

INFO[0028] [k6-reporter v2.2.0] Generating HTML summary report  source=console
ERRO[0028] handleSummary() failed with error "TypeError: Cannot read property 'passes' of undefined", falling back to the default summary  source=console

(and if you have any idea of how to separate two requests to two different metrics of http_req_duration... and it will look good on the html ... I will be more than thankful)

Cannot read property 'values' of undefined"

Hi Ben,

New error popped - never saw it before:

ERRO[0003] handleSummary() failed with error "TypeError: ejs:301
    299|             <h4>Virtual Users</h4>
    300|             <i class="fas fa-user icon"></i>
 >> 301|             <div class="row"><div>Min</div><div><%= data.metrics.vus.values.min %></div></div>
    302|             <div class="row"><div>Max</div><div><%= data.metrics.vus.values.max %></div></div>
    303|           </div>
    304|         </div>

Cannot read property 'values' of undefined", falling back to the default summary  source=console

I don't have a clue why ... because other test suites with setup and teardown functions - the htmlSummary working great.

Question: Is there a way to customize html title?

Dear all

I have got k6 tests with k6 reporter. Can I customize somehow html title .e.g:

  • Not having "K6 Load Test: 2024-03-06 00:31"
  • But having like "Test: Solana: 2024-03-06 00:31 "

Thank you for any information in advance!

Alex

Issue with Custom Metrics

The html report has issue with custom metrics, the way columns are generated are not consistent.
The columns headers are moving and does not show the correct metrics against the it.

Eg:
Here the header of the columns are shifted, like the count should be 4 for all the API's but its moving around for all the 3 rows and showing different values(taking some other metric in place)
report

Actual K6 output:
image

Can you please check on this issue?

Does not support custom summaryTrendStats options

The HTML report doesn't display properly value with the custom summaryTrendStats

export let options = {
    thresholds: {
        "http_req_duration{name:get-/}": ["p(98)<100"],
        BP01T01_Homepage: ["p(98)<100"],
        BP01T01_Homepage_OK: ["rate=1"],
    },
	summaryTrendStats: ["min", "max", "avg", "p(91)", "p(97)", "p(98)", "count"],
};

image
image

Use with Typescript tests

Edit: I do get that error, but it works fine. You can close this.

Hopefully this is an ok method to ask--Does this only work with tests authored in javascript? Our tests are written in Typescript and transpiled to Javascript. When I try to import I get:
Cannot find module 'https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js' or its corresponding type declarations.ts(2307)

Nested groups

We are using nested grouping in our test setup (group functions within group functions).

In the console this is displayed by indentation but the html report does not seem to handle this.
The checks/groups section is now empty and it also displays failed requests (which don't show up in the console).

Is there a way to activate nested grouping in the html reporter?

Feature request: Add the new failed requests stats to the HTML output

Feature request

Part 1

K6 added a new default metric in v0.31.0 called "http_req_failed". It would be great to have that stats also in the HTML summary.
grafik

Part 2 (Question)

Is there an easy way to include the URLs with its status in the HTML output? When I run K6 tests and some requests fail, I probably want to know what the URLs are. Of course, I can always create a CSV file to get that info. But it would be more convenient to have this information directly in the same window.

Can't find the html report inside docker

Running a test on docker and set the volume to /src on docker. After execution we can see the standard out saying generating report. But can't find the file .

Guide me.

Checks & Groups : checks in subgroups are not aggregated

Checks from subgroups should be aggregated at group level.

  • the "Checks stats" shows 4 checks
    image

  • But only 2 checks are reported on the "Checks and Groups" tab

image

  • code to reproduce :
import http from 'k6/http';
import { check, group } from 'k6';
import { htmlReport } from "https://raw.githubusercontent.com/benc-uk/k6-reporter/2.3.1/dist/bundle.js";
import {  textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js';

export const options = {
  stages: [
    { target: 1, duration: '2s', },
  ]
}
  

export default function go() {
  group('Group 1', function () {
    const response = http.get('https://test.k6.io');
    check(response, {
      'Group 1: is status 200': (r) => r.status === 200,
    });
    group('Group 1.1', function () {
     
      check(response, {
        'Group 1.1: is status 200': (r) => r.status === 200,
      });
    })
  });
}

export function handleSummary(data) {
  console.info('Preparing the end-of-test summary...');
  return {
    "summary.html": htmlReport(data),
    'stdout': textSummary(data, { indent: ' ', enableColors: true }), // Show the text summary to stdout...
  };

}
  • run output :
$ k6 run sub-groups.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: sub-groups.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 32s max duration (incl. graceful stop):
           * default: Up to 1 looping VUs for 2s over 1 stages (gracefulRampDown: 30s, gracefulStop: 30s)


running (03.0s), 0/1 VUs, 2 complete and 0 interrupted iterations
default ✓ [======================================] 0/1 VUs  2s
INFO[0003] Preparing the end-of-test summary...          source=console
INFO[0003] [k6-reporter v2.3.0] Generating HTML summary report  source=console
     █ Group 1

       ✓ Group 1: is status 200

       █ Group 1.1

         ✓ Group 1.1: is status 200

     checks.........................: 100.00% ✓ 4       ✗ 0
     data_received..................: 28 kB   9.4 kB/s
     data_sent......................: 616 B   207 B/s
     group_duration.................: avg=744.84ms min=0s    med=593.27ms max=1.79s    p(90)=1.61s    p(95)=1.7s
     http_req_blocked...............: avg=230.07ms min=0s    med=230.07ms max=460.14ms p(90)=414.12ms p(95)=437.13ms
     http_req_connecting............: avg=65.45ms  min=0s    med=65.45ms  max=130.9ms  p(90)=117.81ms p(95)=124.36ms
     http_req_duration..............: avg=1.25s    min=1.18s med=1.25s    max=1.33s    p(90)=1.31s    p(95)=1.32s
       { expected_response:true }...: avg=1.25s    min=1.18s med=1.25s    max=1.33s    p(90)=1.31s    p(95)=1.32s
     http_req_failed................: 0.00%   ✓ 0       ✗ 2
     http_req_receiving.............: avg=9.2ms    min=0s    med=9.2ms    max=18.4ms   p(90)=16.56ms  p(95)=17.48ms
     http_req_sending...............: avg=358.85µs min=0s    med=358.85µs max=717.7µs  p(90)=645.92µs p(95)=681.81µs
     http_req_tls_handshaking.......: avg=134.34ms min=0s    med=134.34ms max=268.69ms p(90)=241.82ms p(95)=255.25ms
     http_req_waiting...............: avg=1.24s    min=1.16s med=1.24s    max=1.33s    p(90)=1.31s    p(95)=1.32s
     http_reqs......................: 2       0.67081/s
     iteration_duration.............: avg=1.48s    min=1.18s med=1.48s    max=1.79s    p(90)=1.73s    p(95)=1.76s
     iterations.....................: 2       0.67081/s
     vus............................: 1       min=1     max=1
     vus_max........................: 1       min=1     max=1
  • version
$ k6 version
k6 v0.36.0 (2022-01-24T09:50:03+0000/ff3f8df, go1.17.6, windows/amd64)

how to make "result.html" dinamic?

i was try

export function getNameOnly() {
  const path = require('path');
  const parsedPath = path.parse(__filename);
  const fileNameWithoutExt = parsedPath.name;
  return fileNameWithoutExt;
}

export function handleSummary(data) {
  const getName = getNameOnly();
  return {
    [`${getName}.html`]: htmlReport(data),
  };
}

but its not working,
ERRO[0002] handleSummary() failed with error "TypeError: Value is not an object: undefined", falling back to the default summary source=console

Show summary report in checks and groups for K6

While executing K6 scenarios, the summary also displays something like this :
█ Scenario: "AccountCreationScenario"

   █ Setup:

     █ Verify token generated correctly

       ✗ Exception raised "the body is null so we can't transform it to HTML - this likely was because of a request error getting the response"
        ↳  0% — ✓ 0 / ✗ 1

   █ Test:

     █ Set the current date to now

       ✗ expected response code is 'OK', got +0
        ↳  0% — ✓ 0 / ✗ 1

     █ Search a customer in CRD to check its existence

Any success or failure like this is not displayed in the HTML report checks tab or anywhere in the HTML report.
This is required in order to identify what has failed from the report itself when it is being delivered to servers.

image

If this block could be filled with the summary of failing steps, the report will be more complete.

Provide precompiled binary as release to support Docker image better

As K6's docker image provides a non-root context plus we do not want to run it as a privileged user either - could you provide the precompiled binary as part of your release too?

Then we could use a simple wget in the vanilla K6 container rather than needing to provide our own variant of the image with go etc installed

Joining results from multiple k6 runs

Hi!

We are using k6 at work and implemented wrapper, which executes k6 on our own cluster multiple instances. Currently I join summaries from k6 --summary-export from all instances, but it will be perfect if your reporter could be upgraded to support multiple k6 runs results joining.

I think I could find time to help to implement it, but I need to discuss with you what will be the best way to implement this functionality. Of course if you agree that it is needed in your project.

Best regards.

npm package

Instead of importing form URL, it would be nice to have npm package that can be imported

Empty report generation error

I am getting an error while installing the Packr2 package. I could not find how to install the package. Missing Packr and Packr2 packages causes reports to be generated empty.

Console Logs:

`
$go get -u github.com/gobuffalo/packr/v2/packr2

go: downloading github.com/gobuffalo/packr/v2 v2.8.1
go: found github.com/gobuffalo/packr/v2/packr2 in github.com/gobuffalo/packr/v2 v2.8.1
go: downloading github.com/spf13/cobra v1.1.3
go: downloading golang.org/x/tools v0.1.0
go: downloading github.com/karrick/godirwalk v1.16.1
go: downloading golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
go: downloading github.com/gobuffalo/logger v1.0.3
go: downloading github.com/sirupsen/logrus v1.8.0
go: downloading github.com/markbates/errx v1.1.0
go: downloading github.com/markbates/oncer v1.0.0
go: downloading github.com/markbates/safe v1.0.1
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading golang.org/x/crypto v0.0.0-20210218145215-b8e89b74b9df
go: downloading github.com/magefile/mage v1.11.0
go: downloading golang.org/x/sys v0.0.0-20210218155724-8ebf48af031b
go: downloading golang.org/x/term v0.0.0-20201210144234-2321bbc49cbf
go: golang.org/x/tools upgrade => v0.1.0
go: golang.org/x/sync upgrade => v0.0.0-20201207232520-09787c993a3a
go: golang.org/x/sys upgrade => v0.0.0-20210218155724-8ebf48af031b
go: github.com/sirupsen/logrus upgrade => v1.8.0
go: github.com/rogpeppe/go-internal upgrade => v1.7.0
go: github.com/spf13/cobra upgrade => v1.1.3
go: github.com/magefile/mage upgrade => v1.11.0
go: golang.org/x/crypto upgrade => v0.0.0-20210218145215-b8e89b74b9df
go: golang.org/x/term upgrade => v0.0.0-20201210144234-2321bbc49cbf
go: github.com/gobuffalo/packd upgrade => v1.0.0
go: github.com/gobuffalo/logger upgrade => v1.0.3
go: github.com/spf13/pflag upgrade => v1.0.5
go: github.com/karrick/godirwalk upgrade => v1.16.1

$./scripts/build.sh

./scripts/build.sh: line 3: packr2: command not found
go: downloading github.com/Masterminds/sprig/v3 v3.2.0
go: downloading github.com/Masterminds/semver/v3 v3.1.1
go: downloading github.com/imdario/mergo v0.3.11
go: downloading github.com/google/uuid v1.1.1
go: downloading github.com/spf13/cast v1.3.1
go: downloading github.com/Masterminds/goutils v1.1.0
go: downloading github.com/mitchellh/copystructure v1.0.0
go: downloading github.com/shopspring/decimal v1.2.0
go: downloading github.com/huandu/xstrings v1.3.1
go: downloading github.com/mitchellh/reflectwalk v1.0.0
go build github.com/benc-uk/k6-reporter: copying /tmp/go-build027705512/b001/exe/a.out: open bin/k6-reporter: permission denied
./scripts/build.sh: line 5: packr2: command not found

$sudo ./scripts/build.sh

./scripts/build.sh: line 3: packr2: command not found
./scripts/build.sh: line 5: packr2: command not found

`

With the command "go run main.go", the k6-reporter application was created, but when I generate the report it is empty.

resim

Issue with "Total Requests" Count when execute the k6 'Checks'

Hi K6 Team,

I used Checks to run K6 scripts and published the results to Grafana dashboards and influx DB 2.0 instances. The run is being performed with Jenkins.

Here is the Code: sample Code from K6 Site. https://k6.io/docs/using-k6/checks/
1

I only hit 1 request here, so we need to see 1 in the Requests in the HTML Total Request widget. but there I can see 2 Requests count on the HTML Report side but in the influx DB side and Grfana side I can see only 1 request. Checks Details and all other details data are correct but have the issue with request count.

2

4

3

Here I attached my Influx DB data and Grafana Dashboard Details.

5
7
6

Please, if anyone on this team can help, please do. that would be really helpful. 🙌

Headers for Custom Metrics are not displayed in the correct order

Hi,

Thanks for this great reporter tool. I'm having an issue when generating the report with Custom Metrics.

For some reason, the column names are not displayed in the correct order.

For example, for "http_req_duration{test_type:auth1}" the value 3046.00 represents the max value, but the column name is p(95).

HTML report:
image

Console:
image

Not sure on how to provide more data to help you debug this. My k6 script looks something like this:

import { auth } from './auth.js';
import { scenariosAuth } from './scenariosAuth.js';
import { htmlReport } from 'https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js';
import { textSummary } from './report.js';

export let options = {
  thresholds: {
    'http_req_duration{test_type:auth1}': ['p(90)<800'], 
    'http_req_duration{test_type:auth2}': ['p(90)<8800'],
    'http_req_duration{test_type:auth3}': ['p(90)<8000'],
  },
  scenarios: {
    auth1: scenariosAuth.auth1,
    auth2: scenariosAuth.auth2,
    auth3: scenariosAuth.auth3,
  },
};

export function auth1() {
  auth();
}

export function auth2() {
  auth();
}

export function auth3() {
  auth();
}

export function handleSummary(data) {
  return {
    'result.html': htmlReport(data),
    stdout: textSummary(data, { indent: ' ', enableColors: true }),
  };
}

handleSummary doesn't work in combination with the standard textSummary output

I would like to get the html report and the standard output of k6 in one run.
The following snippet is my approach to get this running:

export function handleSummary(data) {
  console.log('Preparing the end-of-test summary...');
  return {
    'stdout': htmlReport(data),
    'stdout': textSummary(data, { indent: ' ', enableColors: true}),
  };
}

Notes

I expect that this code generates an html report in the same directory, which doesn't happen.
I also tried to add the filename in the function, but still the same result.
'stdout': htmlReport(data, { filename: "htmlreport.html" }),

Use k6-reporter locally

Hi, I really love to use this project on my local server because it's nice and really easy to read the report. But due to network connectivity, I can't reach github server. Any idea how to build it locally to use without a network connection?

Thank you guys for help !!

K6 report missing red highlighting?

If there is failed threshold (red tab above) should the row in the grid below that failed be highlighted red as well? Currently I can't see what threshold failed given this result...

2022-09-08 14_18_55-K6 Load Test_ PSC_LIMS

Code used...

import { htmlReport } from "https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js";
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export function handleSummary(data) {
  return {
    "summary.html": htmlReport(data, {title: "PSC_LIMS"} ),
    stdout: textSummary(data, {indent: " ", enableColors: true})
  };
}

Date in report file name

I am looking to create a new file each run with the date or date time in

let filename = 'reports/myreport' + datetime + '.html'

return { ${filename} : htmlReport(data, { title: "My First Report", debug: false }) };

I thought this would work.......it doesn't

Support multi group view

Having a group within a group will result in no results shown just the first group name.

group('api', () => group('endpoint', () => check(...)))

visualizing report in Jenkins

Hello,
I'm trying to visualize the report in Jenkins, but I got these errors:

Refused to load the stylesheet 'https://unpkg.com/[email protected]/build/pure-min.css' because it violates the following Content Security Policy directive: "style-src 'self'". Note that 'style-src-elem' was not explicitly set, so 'style-src' is used as a fallback.

k6-test.results.html:7 Refused to load the stylesheet 'https://use.fontawesome.com/releases/v5.15.1/css/all.css' because it violates the following Content Security Policy directive: "style-src 'self'". Note that 'style-src-elem' was not explicitly set, so 'style-src' is used as a fallback.

k6-test.results.html:9 Refused to load the image 'https://raw.githubusercontent.com/benc-uk/k6-reporter/main/assets/icon.png' because it violates the following Content Security Policy directive: "img-src 'self'".

k6-test.results.html:13 Refused to apply inline style because it violates the following Content Security Policy directive: "style-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-yu/Yl9DuKHjUxEBa9OcQyl+0cHmw0UuHs9XVohodlIw='), or a nonce ('nonce-...') is required to enable inline execution.

k6-test.results.html:140 Refused to apply inline style because it violates the following Content Security Policy directive: "style-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-CA/eh4+2R0J7cEQ14gBMtx834RIOjzMUqCM+evtrkp4='), or a nonce ('nonce-...') is required to enable inline execution. Note that hashes do not apply to event handlers, style attributes and javascript: navigations unless the 'unsafe-hashes' keyword is present.

k6-test.results.html:482 Refused to apply inline style because it violates the following Content Security Policy directive: "style-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-IOWYPccv4+GIAWz50PQ4hgBzwty+G8ckj9XrN5jdx6g='), or a nonce ('nonce-...') is required to enable inline execution. Note that hashes do not apply to event handlers, style attributes and javascript: navigations unless the 'unsafe-hashes' keyword is present.

If I disable CSP in Jenkins it is correctly visualized.

Default jenkins CSP is: sandbox; default-src 'none'; img-src 'self'; style-src 'self';

Do you know a workaround less aggressive / more secure?

Thank you for this very useful utility!

Using "htmlReport" Outside of k6 Script via Command Line

Dear k6-reporter Community,

I'm currently using the htmlReport function from the k6-reporter package to generate HTML reports from k6 test results, and I'm facing a specific challenge that I hope someone in the community might be able to help with.

My current workflow involves two separate processes:

  1. Executing a k6 script and generating a JSON result file k6 run --no-summary --no-thresholds -o json=.\result.json .\test.js.
  2. After the first process is complete, generating an HTML report from the JSON result file.

I'm looking for a way to use the htmlReport function outside of a k6 script, specifically via the command line using Node.js. This separation is crucial for my use case as it allows for better process management and resource allocation.

Currently, I understand that the htmlReport function is typically used within the handleSummary callback in a k6 script. However, I need to call this function directly from a Node.js script after the k6 test execution is finished.

Has anyone in the community found a way to achieve this? Or does anyone have suggestions on how to adapt the htmlReport function for this purpose?

Any guidance, code snippets, or alternative approaches would be greatly appreciated. If you've tackled a similar challenge or have ideas on how to proceed, I'd be very grateful to hear about your experiences or solutions.

Thank you all in advance for your time and assistance 🙏

Broken metrics order in HTML report

I've found that custom metrics printing order in report is broken. See screnshots from HTML report and test report below. max is less than min :) And other metrics are messed too.

K6_Load_Test__2021-06-11_07_06

Снимок_экрана_11_06_2021__12_08

Here is minimized code which reproduces this:

import http from 'k6/http';
import {Trend} from 'k6/metrics';

import {htmlReport} from "https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js";
import {textSummary} from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

const url = 'https://test.k6.io/';

let durationTrend = new Trend('requests duration');

export let options = {
    scenarios: {
        example_scenario_public: {
            executor: 'constant-arrival-rate',
            maxVUs: 30,
            preAllocatedVUs: 30,
            rate: 10,
            timeUnit: '1s',
            duration: `10s`,
            gracefulStop: `2s`,
            exec: 'example_scenario',
        },
    },
    summaryTrendStats: [
        'min',
        'avg',
        'med',
        'max',
        'p(90)',
        'p(95)',
        'p(99)', // By default 99-percentile is not printed, so we explicitly say to k6 what we need to be printed and 99-percentile among these
    ],
    thresholds: {
        'requests duration{type:a}': [
            'p(95) < 200',
            'p(99) < 400',
        ],
        'requests duration{type:b}': [
            'p(95) < 250',
            'p(99) < 450',
        ],
    },
};

export function example_scenario() {
    let response = http.get(url, {tags: {type: 'a'}});
    durationTrend.add(response.timings.duration, {type: 'a'});
    response = http.get(url, {tags: {type: 'b'}});
    durationTrend.add(response.timings.duration, {type: 'b'});
}

export function handleSummary(data) {
    return {
        "summary.html": htmlReport(data),
        stdout: textSummary(data, { indent: " ", enableColors: true }),
    };
}

why i can't use "options.scenarios" in handleSummary(data)

i got empty console log {} and this error
handleSummary() failed with error "TypeError: Cannot read property 'values' of undefined", falling back to the default summary source=console

export let options = {
  scenarios: {
    agentDetail: {
      executor: 'constant-vus',
      exec: allCase.agentDetail_Testcase,
      vus: 3,
      duration: '5s',
    }
  },
  thresholds: {
    http_req_failed: ['rate<0.01'],
  },
  summaryTrendStats: ['count', 'avg', 'min', 'med', 'max', 'p(90)', 'p(95)', 'p(99)'],
};
export function handleSummary(data) {
  console.log(options.scenarios)
  for (let key in options.scenarios) {
    let duration = parseInt(options.scenarios[key].duration); 
    let ratePerScenario = data.metrics[`iterations{scenario:${key}}`].values.count / duration; 
    data.metrics[`iterations{scenario:${key}}`].values.rate = ratePerScenario; 
  }
  return {
    "sellingScenario.html": htmlReport(data),
    stdout: textSummary(data, { indent: " ", enableColors: true }),
  };
}

Report do not include Count metrics

I've following script where I use Counter metrics and I see it in standard output , but there is no such results in report

`import http from 'k6/http'
import { check, group } from 'k6'
import { Counter } from 'k6/metrics';
import { randomIntBetween } from 'https://jslib.k6.io/k6-utils/1.1.0/index.js';
import { htmlReport } from "https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js";

export let options = {

thresholds: {
  http_req_failed: ['rate == 0.00'],                                    
  http_req_duration: ["p(95)<10000", "p(90)<10000", "avg<10000"],
  http_reqs: ["count > 0"],
  'RESPONSE_STATUS_200': ['count > 0'],
  'DUPLICATES_COUNTER':  ['count < 10'],
  'RESPONSE_STATUS_500': ['count < 1'],
  'UNKNOWN_ERRORS':      ['count < 1']
},

duration: '2m',
vus: 100,
summaryTimeUnit: 'ms'

}

const duplicatesCounter = new Counter('DUPLICATES_COUNTER')
const responseStatus200 = new Counter('RESPONSE_STATUS_200')
const responseStatus500 = new Counter('RESPONSE_STATUS_500')
const unknownErrors = new Counter('UNKNOWN_ERRORS')

export default function () {
const context = {
headers: {'header1':'test'}}

const productId = 1
const merchant_id = randomIntBetween(60000, 70000) 

const data = {"description": "load-test"}

const res = http.post(`https://my-tests.api`, JSON.stringify(data), context)

if(res.status === 200){
    if(res.json()['status'] !== 'error'){
        responseStatus200.add(1)
    } else if (res.json()['error']['message'].includes("уже был создан")) {
        duplicatesCounter.add(1)
    }
}

if(res.status === 500){
    responseStatus500.add(1)
    console.log(`"*** WARNING 500 ***\n" + "Status code: ${res.status}" + "\n" + ${JSON.stringify(res.body)}`)
}

if(res.status != 200 && res.status !=500 && res.json()['status'] !== 'error') {
    unknownErrors.add(1)
    console.log(`"*** WARNING unknown *** \n" + "Status code: ${res.status}" + "\n" + ${JSON.stringify(res.body)}`)
}

check(res, {'Create payment resposnse status text is "ok"': (res) => res.json()["status"] === 'ok',},) 

}

export function handleSummary(data) {
return {
"summary.html": htmlReport(data),
}
}`

k6

handleSummary() failed with error "TypeError: Not a function: [object Object]"

Hi Bro, I am running with --compatibility-mode=base, got below error
handleSummary() failed with error "TypeError: Not a function: [object Object]", falling back to the default summary source=console

here is my code quiet simple just follow the demo

const htmlReport = require("https://raw.githubusercontent.com/benc-uk/k6-reporter/main/dist/bundle.js");
exports.handleSummary =  function (data) {
    return {
        "summary.html": htmlReport(data),
    };
}

k6 v0.36.0 ((devel), go1.17.6, darwin/arm64)

Custom metrics show wrong results

Hey, the custom metrics show wrong order results

image

This is the correct order but for some reason, the columns headers are in the wrong order:

"times":{"min":7371.099,"avg":8086.7376,"max":8331.795,"p(90)":8330.8086,"p(95)":8331.301800000001}

Error on running k6 report

Hi, I'm trying to run some tests with k6 and generate a report on k6-reporter but I'm not able to do it. I get this error message.
error

I would like the ability to add a custom header, or at least the ability to add some links to the web report.

This web report is a great first step. It's a wonderful high-level overview.

However, if we automate a job and send the web report to people, it will be difficult for the test engineer to drill down deeper.

I work with Mark Tomlinson, and I am shamelessly mentioning his name so maybe we will get higher priority.

We would either like the ability to add our custom header. The header may look like

My Company
ADO ticket

Another thing that would work for us is the ability to add supporting links and a section that has the links.
In that case, I would pass in 2 links, one for my Dynatrace log, and one for my ado ticket

Publish this extension to npmjs

By now in the test file we have to import this extension via a path to github.

Wouldn't it be easier to import it from node_modules (after 'installing' it from npmjs) - at least when k6 is used in javascript / node based environments?

AFAIK for this all what is needed is a package.json with a 'main' property in the dist folder.

Error while using the new JS feature

Hi Ben,
Awesome project, kind of a most when using k6, thanks!

I've got an error when trying to use the handleSummary as described in the README

INFO[0006] [k6-reporter v0.0.1] Generating HTML summary report as: summary.html  source=console
ERRO[0006] handleSummary() failed with error "TypeError: ejs:201
    199|                 <td class="<%= checkFailed(metric, 'avg') %>"><%= metric.values.avg.toFixed(2) %></td>
    200|                 <td class="<%= checkFailed(metric, 'max') %>"><%= metric.values.max.toFixed(2) %></td>
 >> 201|                 <td class="<%= checkFailed(metric, 'med') %>"><%= metric.values.med.toFixed(2) %></td>
    202|                 <td class="<%= checkFailed(metric, 'min') %>"><%= metric.values.min.toFixed(2) %></td>
    203|                 <td class="<%= checkFailed(metric, 'p(90)') %>"><%= metric.values['p(90)'].toFixed(2) %></td>
    204|                 <td class="<%= checkFailed(metric, 'p(95)') %>"><%= metric.values['p(95)'].toFixed(2) %></td>

Cannot read property 'toFixed' of undefined or null", falling back to the default summary  source=console

I'm pretty sure something is missing (bundling, node ...), not sure what.

Enhancement to show failed thresholds in red color

Hi All,

Really appreciate the tool you guys have provided, it is really helping me in generating a great report.

I would like to know if there is a way to distinguish between a failed threshold and passed one. The MaxPOSTResponseTimeExceeded metric shown in below report failed the threshold , is there an option to mark it in a different color , so it is easily distinguishable.

image

Regards

HTML report does not show/support built-in k6 metrics like p(99), p(100)

Hello K6 reporter Team,
I faced a simple issue with k6 and HTML reporter.

Flow:
I use summaryTrendStats,
image

K6 supports p(99) and p(100), and after any run the k6 shows them like that:
image

Only the K6 Html reporter does not show them, namely p(99), p(100)
As you can see, the maximum columName is the "95th Percentile"
image

Would you be so kind as to extend the k6 HTML reporter to support all built-in metrics in k6?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.