errors.Cannot read properties of undefined (reading '0')

[warmuuh]

errors.Cannot read properties of undefined (reading '0')

[hassy]

thanks for the bug report and the diff @warmuuh! looks like a bug, we'll take a look /cc @InesNi

[InesNi]

Hi @warmuuh 👋🏻 ,

Thank you for reporting the issue and for all the details.
I can see you are using the parallel setting in the scenario. Would you be able to try to run the test without it and see if the issue still happens?

parallel is not a setting we officially support (yet) and the tracing in publish-metrics does not work with it atm.

[warmuuh]

it also happens for another flow that we have:

config:
  http:
    extendedMetrics: true
  phases:
    - name: Ramp up
      duration: 30
      arrivalRate: 30
      rampTo: 50
    - name: Sustain
      duration: 30
      arrivalRate: 10
  environments:
   prod:
      target: "..."
   qa3:
      target: "..."
  payload: ...
  plugins:
    ensure: {}
    metrics-by-endpoint:
      useOnlyRequestNames: true
      stripQueryString: true
    publish-metrics:
      - type: open-telemetry
        serviceName: "..."
        traces:
          sampleRate: .1
          useRequestNames: true
          attributes:
            team: ...
            artillery-test: ...
            version: 1.0.0
            env: preprod
      - type: datadog
        apiKey: "..."
        appKey: "..."
        prefix: "artillery.publish_metrics_plugin."
        tags:
          - "team:..."
          - "artillery-test:..."
  ensure:
    thresholds:
      - "http.response_time.p95": 500

before:
  flow:
    - log: "Current environment is set to: {{ $environment }}"

scenarios:
  - name: ...
    flow:
      - post:
          url: "..."
          headers:
            Content-Type: "application/json"
            X-Forwarded-Host: "..."
          body: "..."

[InesNi]

Hi @warmuuh,

When running the test script without parallel do you still get both errors or just the errors.Cannot read properties of undefined (reading 'options')?

[warmuuh]

yes, a whole lot of Cannot read properties of undefined (reading 'options') on debug output. and errors.Cannot read properties of undefined (reading '0') on the actual report

[InesNi]

Hi @warmuuh 👋🏻 ,

I haven't been able to reproduce this on my side, would you mind giving me a few more bits of information if possible:

1. Are some of the vusers ment to be failing (are you intentionally failing them)? If so, could you share more about the failure?
2. Are errors being handled in any way inside any hooks?
3. What format of request body are you sending?
4. When you use your fix how do the exported traces look like on Datadog? Do you get any of the 3 levels of spans within one trace? Trace should have:
   • Scenario level span (named artillery-http-scenario or the name of your scenario) - root span
   • Request level span (named by your request method (e.g. get) - nested inside scenario span
   • Timing phases level spans (dns_lookup, tcp_handshake, etc) - nested inside each request span
     phases
5. Could you console.log a few details when running the test with one vuser:

• res from endHTTPRequestSpan
• requestSpan and err from the otelTraceOnError
• span from endScenarioSpan
  You can share the output either here or via email to [email protected].

Thanks again! 🙇🏻

[warmuuh]

1. no, they are failing (my assumption) because of the error
2. we have a logger for errors but this doesnt do much:

function logError(req, res, context, events, next) {
  if (res.statusCode != 200) {
    console.log(`Error: ${req.url} --> ${res.statusCode}`);
  }

  return next();
}

3. its a get, returning application/json
4. hm, actually, with that fix, i only get the third ones. without the fix, i get the first and third one.

i did a few more tests locally with a minimal example, using artillery 2.0.8 this time:

config:
  phases:
    - name: Sustain
      duration: 1 second
      arrivalRate: 1
  target: https://jsonplaceholder.typicode.com/
  plugins:
    publish-metrics:
      - type: open-telemetry
        serviceName: "artillery"
        traces:
          sampleRate: .1
          useRequestNames: true
          attributes:
            team: teamname
            artillery-test: test
            version: 1.0.0
            env: preprod
      - type: datadog
        apiKey: "{{ $env.DD_API_KEY }}"
        appKey: "{{ $env.DD_APP_KEY }}"
        prefix: "artillery.publish_metrics_plugin."
        tags:
          - "team:teamname"
          - "artillery-test:test"
scenarios:
  - name: Get Todos
    flow:
      - get:
          name: "get-todo"
          url: "/todos/1"

it looks like that the errors.Cannot read properties of undefined (reading '0') entry in the report only shows up if sampling rate is not 1 (in this example, it is .1).

the Cannot read properties of undefined (reading 'options') shows up all the time in the debug logs though

here is the requested output:

res from endHTTPRequestSpan

PassThrough {
  _readableState: ReadableState {
    objectMode: false,
    highWaterMark: 16384,
    buffer: BufferList { head: null, tail: null, length: 0 },
    length: 0,
    pipes: [],
    flowing: true,
    ended: true,
    endEmitted: true,
    reading: false,
    constructed: true,
    sync: false,
    needReadable: false,
    emittedReadable: false,
    readableListening: false,
    resumeScheduled: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    destroyed: false,
    errored: null,
    closed: false,
    closeEmitted: false,
    defaultEncoding: 'utf8',
    awaitDrainWriters: null,
    multiAwaitDrain: false,
    readingMore: false,
    dataEmitted: true,
    decoder: null,
    encoding: null,
    [Symbol(kPaused)]: false
  },
  _destroy: [Function: destroy],
  _events: [Object: null prototype] {
    prefinish: [Function: prefinish],
    unpipe: [Function: onunpipe],
    error: [Function: onerror],
    close: [Function: bound onceWrapper] { listener: [Function: onclose] },
    finish: [Function: bound onceWrapper] { listener: [Function: onfinish] },
    data: [Function (anonymous)],
    end: [ [Function (anonymous)], [Function] ]
  },
  _eventsCount: 7,
  _maxListeners: undefined,
  _writableState: WritableState {
    objectMode: false,
    highWaterMark: 16384,
    finalCalled: true,
    needDrain: false,
    ending: true,
    ended: true,
    finished: false,
    destroyed: false,
    decodeStrings: true,
    defaultEncoding: 'utf8',
    length: 0,
    writing: false,
    corked: 0,
    sync: false,
    bufferProcessing: false,
    onwrite: [Function: bound onwrite],
    writecb: null,
    writelen: 0,
    afterWriteTickInfo: null,
    buffered: [],
    bufferedIndex: 0,
    allBuffers: true,
    allNoop: true,
    pendingcb: 1,
    constructed: true,
    prefinished: true,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    errored: null,
    closed: false,
    closeEmitted: false,
    [Symbol(kOnFinished)]: []
  },
  allowHalfOpen: true,
  socket: [Getter/Setter],
  httpVersionMajor: [Getter/Setter],
  httpVersionMinor: [Getter/Setter],
  httpVersion: [Getter/Setter],
  complete: [Getter/Setter],
  rawHeaders: [Getter/Setter],
  rawTrailers: [Getter/Setter],
  joinDuplicateHeaders: [Getter/Setter],
  aborted: [Getter/Setter],
  upgrade: [Getter/Setter],
  url: [Getter/Setter],
  method: [Getter/Setter],
  statusCode: [Getter/Setter],
  statusMessage: [Getter/Setter],
  client: [Getter/Setter],
  _consuming: [Getter/Setter],
  _dumped: [Getter/Setter],
  req: [Getter/Setter],
  timings: [Getter/Setter],
  headers: [Getter/Setter],
  setTimeout: [Getter/Setter],
  trailers: [Getter/Setter],
  body: '{\n' +
    '  "userId": 1,\n' +
    '  "id": 1,\n' +
    '  "title": "delectus aut autem",\n' +
    '  "completed": false\n' +
    '}',
  [Symbol(kCapture)]: false,
  [Symbol(kCallback)]: null
}

span from endScenarioSpan

 NonRecordingSpan {
  _spanContext: {
    traceId: '8a93106d546234c023f6e3fae0a8c72e',
    spanId: '02ccee78beb15f0c',
    traceFlags: 0,
    traceState: undefined
  }
}

otelTraceOnError in http.js doesnt seem to be called, so no output. but i realized that i didnt show the full stacktrace of the error:

2024-03-21T13:10:21.305Z plugin:publish-metrics:open-telemetry TypeError: Cannot read properties of undefined (reading 'options')
    at OTelHTTPTraceReporter.endHTTPRequestSpan (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/artillery-plugin-publish-metrics/lib/open-telemetry/tracing/http.js:140:23)
    at iteratee (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:626:21)
    at /Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/async/dist/async.js:3113:16
    at replenish (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/async/dist/async.js:1014:17)
    at /Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/async/dist/async.js:1019:9
    at eachLimit$1 (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/async/dist/async.js:3199:24)
    at Object.<anonymous> (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/async/dist/async.js:1049:16)
    at captured (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:607:21)
    at Object.captureOrMatch (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/@artilleryio/int-commons/engine_util.js:400:12)
    at responseProcessor (/Users/username/.nvm/versions/node/v20.3.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:541:22)

[InesNi]

Hi @warmuuh 👋🏻 ,

Thank you so much for all the detailed information! 🙇🏻‍♀️
There seems to be a bug in the sampling. We will let you know when a fix is available.

On a separate note, I just wanted to make you aware that from Artillery v2.0.5 you can send both traces and metrics to Datadog through the Datadog reporter

OTel reporter runs in the background for traces so the configuration will be the similar, only attributes are called tags and match the format of metric tags. e.g. for the latest script you shared you can do:

  plugins:
    publish-metrics:
      - type: datadog
        apiKey: "{{ $env.DD_API_KEY }}"
        appKey: "{{ $env.DD_APP_KEY }}"
        prefix: "artillery.publish_metrics_plugin."
        tags:
          - "team:teamname"
          - "artillery-test:test"
        traces:
          serviceName: "artillery" 
          sampleRate: 0.1
          useRequestNames: true
          tags:
            - "team:teamname"
            - "artillery-test:test"
            - "version:1.0.0"
            - "env:preprod"

[InesNi]

Hi @warmuuh 👋🏻 ,

Thank you once again for reporting the issue. 🙇🏻‍♀️
The fix is now available in Artillery v2.0.9 release.