krzko/opentelemetry-shell

🚦 OpenTelemetry functions for shells

opentelemetry-shell's Issues

define service.name on users request only

Is your feature request related to a problem?

(I love this issue template)

I'm always frustrated when something overwrites my service.name. actually, the change introduced in #51 (to append the service.name optionally) is probably not that useful. shortly after, I've noticed that "someone else" is now overwriting my globally defined service.name with the bash script name.

that is, in my opinion, also not compliant with the documentation. because the global variable literally just gets thrown away.

Describe the solution you'd like

I don't know if this was a design goal. but, I'd suggest to add explicit properties instead: "if you're a bash script and this property is true, add your name" or "if you are a gitlab pipeline and that (another) property is true, add your name".

Describe alternatives you've considered

making every appendance optional

(like I did in #51)

this way, nobody overwrites my globally defined service.name. on the other hand, the order of the (internal) scripts will then matter, as just the very first one will add its service name. when the code reaches the other ones, it's already done.

opinions @krzko?

Add resource detection for Harness pipelines

Is your feature request related to a problem?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The resource detection functions should detect resource information from the underlying CI/CD system, in this case, Harness pipelines. It should append or override the resource value in telemetry data with this information.

A basis for this work is following the OTEL Collector resource detection processor.

Describe the solution you'd like

Append/add Harness pipeline-specific information, such as;

• Pipeline details
• Workflow details
• Step details

Describe alternatives you've considered

Additional context

• https://docs.harness.io/article/21kgaw4h86-harness-yaml-code-reference
• https://docs.harness.io/article/aza65y4af6-built-in-variables-list
• https://docs.harness.io/article/lml71vhsim-harness-variables

Create trace context

Figure out a solution for traces that span child function calls, suggest a trace context

Send telemetry data using grpcurl

Is your feature request related to a problem?

The library currently supports OTLP (HTTP), but would be nice to support OTLP (gRPC) aswell. Investigate utilising grpcurl for this uplift.

Describe the solution you'd like

Utilise the OTEL protos to then sent unary request via grpcurl

Describe alternatives you've considered

Additional context

unbound variable EPOCHREALTIME

Describe the bug

hi there! great lib, I wanted to use it for monitoring some bash scripts behavior. however, running the README.md traces sample ends up in an error in plain bash Docker runtime:

./spam.sh

 _____ _____ _____ _____ _____ _____ __    _____ _____ _____ _____ _____ __ __
|     |  _  |   __|   | |_   _|   __|  |  |   __|     |   __|_   _| __  |  |  |
|  |  |   __|   __| | | | | | |   __|  |__|   __| | | |   __| | | |    -|_   _|
|_____|__|  |_____|_|___| |_| |_____|_____|_____|_|_|_|_____| |_| |__|__| |_|
 _____ _____ _____ __    __
|   __|  |  |   __|  |  |  |
|__   |     |   __|  |__|  |__
|_____|__|__|_____|_____|_____|

2022-10-04 15:41:58 [INFO] [spam.sh#L43] [source()] Initialising OpenTelemetry Shell v0.0.8
/appl/otello/opentelemetry-shell/library/time.sh: line 34: EPOCHREALTIME: unbound variable
2022-10-04 15:41:58 [INFO] [spam.sh#L10] [sleep_for()] Sleeping for 1 sec...
/appl/otello/opentelemetry-shell/library/time.sh: line 34: EPOCHREALTIME: unbound variable
2022-10-04 15:41:59 [ERROR] [spam.sh#L41] [net_client_post()] The requested URL returned an error: https://[OMITTED]/v1/traces

it looks like $EPOCHREALTIME was not expanded properly.

To Reproduce

Steps to reproduce the behavior:

1. get some linux Docker image
2. clone the repository on main (btw you didn't tag v0.0.8 yet)
3. copy paste the traces sample
4. run it

Expected behavior

since I do not have the $EPOCHREALTIME defined, it should use date instead.

Screenshots

see bash output above.

Environment

• a plain linux in Docker, having installed curl, git, hostname and jq

Additional context

Add resource detection for Jenkins pipelines

Is your feature request related to a problem?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The resource detection functions should detect resource information from the underlying CI/CD system, in this case, Jenkins pipelines. It should append or override the resource value in telemetry data with this information.

A basis for this work is following the OTEL Collector resource detection processor.

Describe the solution you'd like

Append/add Jenkins pipeline-specific information, such as;

• Pipeline details
• Stage details
• Step details

Describe alternatives you've considered

Additional context

Add resource detection for Google Cloud Build pipelines

Is your feature request related to a problem?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The resource detection functions should detect resource information from the underlying CI/CD system, in this case, Google Cloud Build pipelines. It should append or override the resource value in telemetry data with this information.

A basis for this work is following the OTEL Collector resource detection processor.

Describe the solution you'd like

Append/add Github Actions workflow-specific information, such as;

• Pipeline details
• Step details

Describe alternatives you've considered

Additional context

Spec - https://cloud.google.com/build/docs/build-config-file-schema

Add ability pass API Keys as auth headers

Is your feature request related to a problem?

Some vendors require API keys to be passed as headers (curl -H) via POST requests, need to cater for this scenario

ie export LIGHTSTEP_ACCESS_TOKEN='lightstep-access-token: foo'

Describe the solution you'd like

A variable to check that exists, with a an associated format. Pass in to the net_client_post()

Need to remove the opinionated references to /v1/traces and /v1/metrics.

Describe alternatives you've considered

Additional context

environment variables not aligned to spec imo

The default variable OTEL_SERVICE_NAME configures the resource attribute service.name. that's fine, but you're setting a default value when non is present. according to the spec this value is required, so I think it would be good to fail if this is not present.

second, I think that the library should respect the environment variables OTEL_SERVICE_VERSION and OTEL_SERVICE_NAMESPACE. it is already looking at service_version, then adding this into resource attribute service.version. I think, it would make sense to respect OTEL_SERVICE_VERSION (for service.version), as well as OTEL_SERVICE_NAMESPACE (for service.namespace).

let me know what you think of it @krzko, I could of course contribute the change.

Create opentelemetry-shell-action

Is your feature request related to a problem?

To allow for seamless tracing to take place it would be nice to allow traces to take place behind the scenes, without too much user interaction ala the Jenkins OTEL plugin.

Describe the solution you'd like

name: otel-sh-action-example
on: [push]
jobs:
  awesome-job:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: krzko/opentelemetry-shell-action@v1
        with:
          otlp-exporter-otel-endpoint: 'https://otel.awesomedomain.com:443'
      - run: docker ...
      - run: gcloud ...

Describe alternatives you've considered

Additional context

Add resource detection for Gitlab CI/CD

Is your feature request related to a problem?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The resource detection functions should detect resource information from the underlying CI/CD system, in this case, Gitlab CI/CD. It should append or override the resource value in telemetry data with this information.

A basis for this work is following the OTEL Collector resource detection processor.

Describe the solution you'd like

Append/add Github Actions workflow-specific information, such as;

• Pipelines details
• Job details
• Step details

Describe alternatives you've considered

Additional context

• https://docs.gitlab.com/ee/ci/variables/
• CI_SERVER_NAME="GitLab" ?

Distribute via a brew tap

Is your feature request related to a problem?

To help in distribution, brew should be utilised.

Describe the solution you'd like

Make the solution brew install compliant

Describe alternatives you've considered

Additional context

Sanitise $GITHUB_WORKFLOW

Describe the bug

If $GITHUB_WORKFLOW includes special character the metric push will fail

To Reproduce

Steps to reproduce the behavior:

1. Worflow name: Foo Bar (Baz)
2. See error

Expected behavior

To sanitise this and become valid to post to OTEL receiver

Screenshots

Environment

• OS: Linux
• Shell version: -

Additional context

Add support for setting span name

Is your feature request related to a problem?

The span name is currently the second argument after a call to otel_trace_start_parent_span or otel_trace_start_child_span. This results in abnormally-named spans. For example, the code below would yield a span whose name is the SQLite query:

otel_trace_start_parent_span sqlite3 "SELECT 1;"

This wrong.

Describe the solution you'd like

Allow me to manually set the name of the span. Just as custom attributes can be set by defining custom_resource_attributes before calling the function, let me do the same with span_name.

Describe alternatives you've considered

N/A

Additional context

N/A

OTEL_EXPORTER_OTEL_ENDPOINT -> OTEL_EXPORTER_OTLP_ENDPOINT

Describe the bug

OTEL_EXPORTER_OTEL_ENDPOINT dosn't exist

Expected behavior

OTEL_EXPORTER_OTLP_ENDPOINT is used

Additional context

Please change everywhere, including Readme

Add resource detection for AWS CodeBuild pipelines

Is your feature request related to a problem?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The resource detection functions should detect resource information from the underlying CI/CD system, in this case, Github Actions workflows. It should append or override the resource value in telemetry data with this information.

A basis for this work is following the OTEL Collector resource detection processor.

Describe the solution you'd like

Append/add Github Actions workflow-specific information, such as;

• Phase details

Describe alternatives you've considered

Additional context

Specs - https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

[Question] Could this be used to visualise complex `yarn` script nonsense?

Crazy idea: I don't know whether this is in-scope for your project, or whether I should build it myself, but yours feels like the most similar project to what I'm imagining, so I thought I'd ask anyway.

At work, we have a bunch of complicated scripts in our package.json, and they depend on each other in nontrivial ways. Running yarn build will spawn a handful of codegen tasks in a carefully orchestrated manner, with a bunch of parallel tasks specified manually using the concurrently js package. It's like make but strictly worse. I want to rip it out and replace it with something that has an explicit concept of dependencies, and can avoid duplicated work. I've been putting off wrapping my head around the problem because I don't have a good way to visualise it at the moment.

I recently discovered (via https://stackoverflow.com/a/58188828) that if you run yarn config set script-shell $PATH_TO_WRAPPER_SCRIPT then you can change which shell runs your scripts.

I was thinking that I could make a tiny go/rust/typescript/bash program that:

• reads the arguments (normally yarn will call it as sh -c "$BODY_OF_SCRIPT")
• starts a span
  • named "$BODY_OF_SCRIPT"
  • as a child of any existing span that it finds in its environment variables
  • exports the span id into an environment variable so that it can be found by child processes
• spawns sh -c "$BODY_OF_SCRIPT"
• waits on the shell process to complete
• reports the span to the otel collector over http
• exits with the same exit code as sh

If I had this then I would be able to visualise the existing build process, and the whole thing would be a bit less scary.

a) Does the opentelemetry-shell project support linking parent/child relationships between shortlived child shells using environment variables?
b) If I built my idea using opentelemetry-shell, would you be interested in accepting it as an example in this repo?

extra context for if I decide to write it from scratch

My company is currently using datadog, and getting that set up locally was reasonably straightforward. I've not tried piping arbitrary opentelemetry nonsense into it, but it probably wouldn't be that hard. In the worst case I guess I could run zipkin in a docker container or something as a one-off. If I was building it from scratch, I would probably hack it together in a datadog-specific way, even though a standard otel-based implementation would be more useful to more people.

Thinking about it, having a README that says docker run $PORT_FORWARDING_OR_WHATEVER zipkin in one tab and yarn config set script-shell /path/to/shelemetry && yarn build && yarn config delete script-shell && open https://localhost:$ZIPKIN_GUI_PORT/ then it will be easy for people to get started with it with no additional setup or dependencies.

Add resource detection for Github Actions workflows

Is your feature request related to a problem?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The resource detection functions should detect resource information from the underlying CI/CD system, in this case, Github Actions workflows. It should append or override the resource value in telemetry data with this information.

A basis for this work is following the OTEL Collector resource detection processor.

Describe the solution you'd like

Append/add Github Actions workflow-specific information, such as;

• Job details
• Step details

Describe alternatives you've considered

Additional context

Check for $GITHUB_ACTIONS: true

Add shellcheck

Is your feature request related to a problem?

To ensure the releases are of high quality, run shellcheck to lint all *.sh files.

Describe the solution you'd like

Apply check during CI:

• Branch -f warn
• Release -f error`

Describe alternatives you've considered

Additional context

Improve GHA Detctor

Is your feature request related to a problem?

Suppose a detector recognises the CI/CD system its running on. In that case, it should pupulate the service.name attribute and strive to create parent relationships to associated steps, with the command being child dependencies.

Describe the solution you'd like

• Update `service.name1 attribute with - repo name - job name
• Ensure jobs / steps will have the parent/child span relationships
• Update GHA example

Describe alternatives you've considered

Additional context