Questions re ESM support for MikroOrm / knex, Node v20, & OIDC

[er-imageline]

Questions re ESM support for MikroOrm / knex, Node v20, & OIDC

[trentm]

Could someone advise what pg@8 refers to here? Not PostGres v8 surely?

That is referring to the pg npm package (https://www.npmjs.com/package/pg). Version 8.x is the latest version of pg -- though it of course supports versions of PostgreSQL later than postgres v8.

Auto-instrumentation:

* Is ESM auto-instrumentation supported on Node >v20?

Yes, ESM instrumentation is supported versions of Node.js v20 greater than or equal to v20.2.0 -- but only as of the latest v4.4.1 release:
https://www.elastic.co/guide/en/apm/agent/nodejs/current/release-notes-4.x.html#release-notes-4.4.1

Node v20.0.0 and v20.1.0 are not supported because there was a Node.js core bug.

* Can you clarify what  pg@8 means in relation to knex?

A few things here:

1. The supported knex version range mentioned in https://www.elastic.co/guide/en/apm/agent/nodejs/current/esm.html#esm-compat-modules should say >=0.20.0 <4 -- i.e. that it supports knex versions 3.x. I will get that updated.
2. The "knex"-specific instrumentation in the elastic-apm-node APM agent is very limited. It only adds functionality to improve stacktrace information on spans when the feature to enable span stacktraces is enabled. See https://www.elastic.co/guide/en/apm/agent/nodejs/current/supported-technologies.html#compatibility-better-stack-traces In addition enabling that feature is not recommended because collecting stacktraces in Node.js is slow and doing so for every collected APM span is too costly in practice (see https://www.elastic.co/guide/en/apm/agent/nodejs/current/performance-tuning.html#performance-span-stack-traces).
3. On the ECMAScript module support doc page (https://www.elastic.co/guide/en/apm/agent/nodejs/current/esm.html#esm-compat-modules) the knex | >=0.20.0 <3 | Also, only with pg@8. table row is trying to say: "Yes, our knex instrumentation does work with ESM code. However, regularly our knex instrumentation only does its thing when using knex with postgres (pg module) or MySQL (mysql module) and, at this time, ESM support has not yet been added for the mysql module.

* Does that strictly mean 'standalone' knex, or is it possible to auto-instrument a package invoking knex such as MikroOrm?

Typically if the APM agent instruments a particular package A, and some package B uses package A, then that usage would also be instrumented. A good example is the mongoose package that uses the mongodb package under the hood. The APM agent instruments mongodb, so users of mongoose will see tracing data for MongoDB client requests.

Regarding your case:

• I don't know how MikroOrm is using knex specifically, but I suspect that won't be the limiting issue for you.
• The Knex docs (https://knexjs.org/guide/#node-js) say that its MariaDB support uses the mysql package. The APM agent (elastic-apm-node) does instrument mysql, however its current experimental ESM support does not yet support mysql.
• I could be wrong about the mysql package being used, because I see that the MikroOrm docs (https://mikro-orm.io/docs/quick-start) mention installing @mikro-orm/mariadb, which looks like it uses mariadb as its database driver package.

So, IIUC, you are effectively requesting an enhancement that this APM agent support instrumenting mariadb. There is an older issue for this: #1759

If mysql (or perhaps mysql2) is the database driver package being used, then you need ESM support for this APM agent's instrumentation of mysql. Currently the best issue to track for that is #3445

Manual instrumentation:

* Failing that, any suggestions as to how might we manually instrument MikroOrm transactions?

Perhaps configure your usage of MikroOrm to register handlers for Lifecycle Hooks or subscribe to its events -- see https://mikro-orm.io/docs/events -- and then add manual span-creation tracing (e.g. https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-start-span) there?

Bonus: OIDC

* Any plans to support instrumenting OIDC flows?

Nothing specific to OIDC, currently, no.

[er-imageline]

Hi Trent,

Thanks for the quick & detailed reply on this! It's much appreciated.

Yes, ESM instrumentation is supported versions of Node.js v20 greater than or equal to v20.2.0 -- but only as of the latest v4.4.1 release:
https://www.elastic.co/guide/en/apm/agent/nodejs/current/release-notes-4.x.html#release-notes-4.4.1

That's great - I had missed that update.

 it supports knex versions 3.x.

Also glad to hear this!

Regarding knex instrumentation & behaviour, I understand that it is only enabled when spanStackTraceMinDuration is set, & that it's resource-heavy; we intend to enable it only for debugging purposes, not in Production.

Still, I'm surprised it crashes in this way. I'll check if there's a resource spike but I don't think so - it's possible this behaviour is due to some implementation detail in the code - if I have time I'll try to repro this in a shareable way.

So, IIUC, you are effectively requesting an enhancement that this APM agent support instrumenting mariadb. There is an older issue for this: #1759

As it happens, it now turns out we have to migrate to MySQL as Azure is dropping managed MariaDB. So we can un-request MariaDB support, but consider this a +1 for ESM/MySQL support :) - I took a look at the Project page and I guess there's no ETA there?

Perhaps configure your usage of MikroOrm to register handlers for Lifecycle Hooks or subscribe to its events -- see https://mikro-orm.io/docs/events -- and then add manual span-creation tracing (e.g. https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-start-span) there?

Thanks for this tip! I guess the big thing I wanted to verify is that fact that manual instrumentation is still possible with TypeScript/ESM apps, and if I understand correctly it is.

For now I will go modify things to use MySQL and try to add some manual instrumentation.

Typically if the APM agent instruments a particular package A, and some package B uses package A, then that usage would also be instrumented. A good example is the mongoose package that uses the mongodb package under the hood. The APM agent instruments mongodb, so users of mongoose will see tracing data for MongoDB client requests.

This is great practical info! Maybe it's basic/assumed stuff for seasoned JS devs, but I think this paragraph would be helpful in the docs & give users a better intuition of how APM works.

Finally, thanks for the answer re OIDC. I presume we could manually instrument these calls also, right? As they tend to be simple enough, along the lines of try{oidcProvider.dosomething()} and wait for a response.

[trentm]

but consider this a +1 for ESM/MySQL support :) - I took a look at the Project page and I guess there's no ETA there?

Correct. Unfortunately I don't have an ETA.

Finally, thanks for the answer re OIDC. I presume we could manually instrument these calls also, right?

Yes, I believe so. I haven't worked with OIDC libraries myself, so I can't be sure.