open-telemetry/opentelemetry-collector-releases

OpenTelemetry Collector Official Releases

OpenTelemetry Collector distributions

⚠️ Important note: Git tags in this repository may change at any time to fix any issues found during a release. They are only meant to trigger Github releases and should not be relied upon.

This repository assembles OpenTelemetry Collector distributions, such as the "core" distribution, or "contrib".

Each distribution contains:

• Binaries for a multitude of platforms and architectures
• Multi-arch container images
• Packages to be used with Linux distributions (apk, RPM, deb), Mac OS (brew) for the above-mentioned architectures

More details about each individual distribution can be seen in its own readme files.

Current list of distributions:

• OpenTelemetry Collector (also known as "otelcol")
• OpenTelemetry Collector Contrib (also known as "otelcol-contrib")
• OpenTelemetry Collector for Kubernetes (also known as "otelcol-k8s")

opentelemetry-collector-releases's Issues

Include all licenses in the resulting containers

From open-telemetry/opentelemetry-collector#2604

Binary name should be otelcol not core

At least that was in core, was thinking to keep backwards compatiblility

Systemd config for otelcol-contrib points to wrong path for v0.47.0

Description

Systemd config for latest release v0.47.0 points to the incorrect path of the binary for otelcol-contrib in the rpm release:
otelcol-contrib_0.47.0_linux_amd64.rpm.
Due to this otelcol-contrib service can't be started without first modifying the path of the binary.

Work around/fix

Edit the ExecStart line in the systemd config /usr/lib/systemd/system/otelcol-contrib.service by removing the local/ entry in the path
From
ExecStart=/usr/local/bin/otelcol-contrib $OTELCOL_OPTIONS
To
ExecStart=/usr/bin/otelcol-contrib $OTELCOL_OPTIONS

Version 0.47.0

$ rpm -qa |grep otelcol-contrib
otelcol-contrib-0.47.0-1.x86_64

$ rpm -ql otelcol-contrib
/etc/otelcol-contrib/config.yaml
/etc/otelcol-contrib/otelcol-contrib.conf
/lib/systemd/system/otelcol-contrib.service
/usr/bin/otelcol-contrib

$ cat /usr/lib/systemd/system/otelcol-contrib.service
[Unit]
Description=OpenTelemetry Collector Contrib
After=network.target

[Service]
EnvironmentFile=/etc/otelcol-contrib/otelcol-contrib.conf
ExecStart=/usr/local/bin/otelcol-contrib $OTELCOL_OPTIONS
KillMode=mixed
Restart=on-failure
Type=simple
User=otelcol-contrib
Group=otelcol-contrib

[Install]
WantedBy=multi-user.target

$ systemctl status otelcol-contrib.service -l
● otelcol-contrib.service - OpenTelemetry Collector Contrib
   Loaded: loaded (/usr/lib/systemd/system/otelcol-contrib.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Thu 2022-03-24 14:52:54 UTC; 8min ago
  Process: 5864 ExecStart=/usr/local/bin/otelcol-contrib $OTELCOL_OPTIONS (code=exited, status=203/EXEC)
 Main PID: 5864 (code=exited, status=203/EXEC)

Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: otelcol-contrib.service: main process exited, code=exited, status=203/EXEC
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: Unit otelcol-contrib.service entered failed state.
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: otelcol-contrib.service failed.
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: otelcol-contrib.service holdoff time over, scheduling restart.
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: Stopped OpenTelemetry Collector Contrib.
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: start request repeated too quickly for otelcol-contrib.service
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: Failed to start OpenTelemetry Collector Contrib.
Mar 24 14:52:54 opentelemetry-collector-test systemd[1]: Unit otelcol-contrib.service entered failed state.

Version 0.46.0

This version worked out of the box, as the binary is in the /usr/local/bin path here

$ rpm -qa |grep otelcol-contrib
otelcol-contrib-0.46.0-1.x86_64

$ rpm -ql otelcol-contrib
/etc/otelcol-contrib/config.yaml
/etc/otelcol-contrib/otelcol-contrib.conf
/lib/systemd/system/otelcol-contrib.service
/usr/local/bin/otelcol-contrib

$ cat /lib/systemd/system/otelcol-contrib.service
[Unit]
Description=OpenTelemetry Collector Contrib
After=network.target

[Service]
EnvironmentFile=/etc/otelcol-contrib/otelcol-contrib.conf
ExecStart=/usr/local/bin/otelcol-contrib $OTELCOL_OPTIONS
KillMode=mixed
Restart=on-failure
Type=simple
User=otelcol-contrib
Group=otelcol-contrib

[Install]
WantedBy=multi-user.target

distribution for arm7

Hi, I want to use the collector on Raspberry PI with docker, and was not able to find a relevant image in dockerhub to use.
Looking at some old issues and PRs it seems to be removed intentionally.
Was wondering if there are plans to support it any time soon

update this repo to pull the builder from collector core

Currently this repo still pulls the builder from the opentelemetry-collector-builder repo which has been deprecated. See https://github.com/open-telemetry/opentelemetry-collector-releases/blob/main/Makefile#L38

collector does not include nginx receiver

Describe the bug

The collector does not include the nginx receiver, which you'll notice if you configure nginx among receivers.

Steps to reproduce

Add an entry for nginx: to receivers in the collector's config .yaml

What did you expect to see?

Successful run of the collector returning metrics for nginx when configuration is right for the environment or reporting on other aspects of configuration.

What did you see instead?

It logs Error: failed to get config: cannot unmarshal the configuration: unknown receivers type "nginx" for "nginx" (valid value s: [statsd awscontainerinsightreceiver ....

What version did you use?

v0.49.0

What config did you use?

receivers:
  nginx:

Environment

Docker images golang:1.17 and golang:latest

Additional context

• I have made changes to code that resolves the issue by adding the nginxreceiver factory to the list of receivers in internal/components/components.go
• There's also a test file where we need to add a reference to the nginx receiver.
• And there are some administrative items to be handled for the build such as adding the nginxreceiver package to the go.mod in the root and in cmd/configschema. Additionally we need replace directives in the go.mod fils to use the relative path for nginxreceiver.

Add elasticsearch receiver

Add rabbitmq receiver

Where are contrib binaries?

I am trying to use the latest binaries and it seems the only binaries available here are for the core otel collector.

Beside, this repository is listed nowhere:

• the collector documentation does not mention this repository once. Always mention the collector and the binaries without ever linking here.
• both opentelemetry-collector and opentelemetry-collector-contrib repository speak about release and release notes without giving any link to this repository

It's quite a nightmare to find the collector and/or install it (beside docker). But I am surprised I am the first one to raise an issue here. Perhaps I have missed something obvious ?

Thanks,

Remove newrelicexporter

Follow up to open-telemetry/opentelemetry-collector-contrib#9894

opentelemetry-collector-releases/distributions/otelcol-contrib/manifest.yaml

Line 63 in c91fef7

- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/newrelicexporter v0.50.0

Add riak receiver

This receiver is not currently enabled in the contrib distro.

Add couchdb receiver

Have a contrib release

As per the latest SIG call, we should start producing releases from contrib using this repo here.

Add mongodb receiver

otelcol rpm package contains incorrect configuration files

It seems that otelcol_0.38.0_linux_amd64.rpm contains files point to the old installation directory.
Package manager unpack the files to /etc/otelcol/ directory but the following files still reference /etc/opentelemetry-collector:

• https://github.com/open-telemetry/opentelemetry-collector-releases/blob/007228a546e7d0812700464e4c09b39c60179a9e/distributions/otelcol/otelcol.service
• https://github.com/open-telemetry/opentelemetry-collector-releases/blob/007228a546e7d0812700464e4c09b39c60179a9e/distributions/otelcol/otelcol.conf

Allow customization of properties

The script that generates go-releaser files makes assumptions that are not always true. For instance, the container image for the core distribution defaults to "otel/otelcol:v0.35.0", when it should in fact use "otel/opentelemetry-collector:0.35.0"

Run all release builds on PRs

Currently, CI for PRs on this repository does not build all the artifacts. This makes it hard to catch issues before actually tagging a release. To prevent this we can do a gorelease dry run on all PRs.

Error: failed to get config: cannot unmarshal the configuration: unknown exporters type "googlecloudpubsub"

Describe the bug
When I am using the GCP pubsub as exporter, I am getting

Error: failed to get config: cannot unmarshal the configuration: unknown exporters type "googlecloudpubsub" for "googlecloudpubsub/test" (valid values: [clickhouse coralogix otlphttp sapm awsemf azuremonitor jaeger awsprometheusremotewrite elastic logging newrelic opencensus signalfx dynatrace carbon jaeger_thrift logzio prometheus splunk_hec awskinesis awsxray file honeycomb loadbalancing otlp parquet prometheusremotewrite awscloudwatchlogs sumologic loki tencentcloud_logservice kafka datadog f5cloud googlecloud influxdb skywalking stackdriver alibabacloud_logservice humio sentry tanzuobservability zipkin elasticsearch])

Steps to reproduce
exporters:
googlecloud/test:
project:
topic: projects//topics/
compression: gzip
watermark:
behavior: earliest
allowed_drift: 1h

Use Go 1.17

Build the releases with Go 1.17

review replaces statements in the manifest

Are these replaces needed anymore? Now that we're using multimod to manage versions in require statements we might be able to omit these. Perhaps something to try after this release is shipped.

Originally posted by @Aneurysm9 in #28 (comment)

Remove windows-arm64 from the build matrix

Windows on ARM64 seems to be a new combination on Go 1.17, but apparently, the collector can't be built on that platform:

https://github.com/open-telemetry/opentelemetry-collector-releases/runs/3522627590?check_suite_focus=true

At least for the first release, we need to remove this from the goreleaser config.

Release v0.35.0

Should the main module on this repo be made internal?

On the v0.47.0 release we had an issue when releasing binaries (see open-telemetry/opentelemetry-collector/issues/5013) that was not caught on CI until after tagging v0.47.0. Having a public Go module on this repository makes it hard to untag and re-tag a release without potentially breaking people's builds.

If we don't intend the main module on this repository to be publicly used (or, at least, if we don't intend it for now), we should place it in an internal folder so that we can safely re-tag if we have the need.

cc @jpkrohling to evaluate this

Release binaries and container images v0.49.0

Publish images to opentelemetry namespaces

Currently, the release is pushing container images to quay.io/jpkroehling, and it should publish to the official namespaces at quay and Docker Hub.

Signing the assembly

We want to sign otelcontribcol.exe file on windows machine as it can go through if it is not signed . Any inputs on how we can accomplish any tweak we can do in go language git hub repo code

Building arm6 container image not working

After #39, running goreleaser --snapshot --rm-dist doesn't seem to work anymore, breaking on this step:

   ⨯ release failed after 53.58s error=failed to build otel/opentelemetry-collector:0.41.0-SNAPSHOT-31eb1eb-armv6: exit status 125: [1/3] STEP 1/2: FROM alpine:3.13 AS certs
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:3.13...
Getting image source signatures
Copying blob sha256:846f3a0f3493d22f22643b6a1c057d2d37e608433cd1033d25dac92032b8b9e3
Copying config sha256:11ed10a9502532148b48c67e39d24a465d67adc7e683e830e6e5f5e597ba61a5
Writing manifest to image destination
Storing signatures
[1/3] STEP 2/2: RUN apk --update add ca-certificates
exec container process `/bin/sh`: Exec format error
[2/3] STEP 1/3: FROM alpine:3.13 AS otelcol
Error: error building at STEP "RUN apk --update add ca-certificates": error while running runtime: exit status 1

@silverlyra, do you remember whether this worked for you?

Failed at step EXEC spawning /usr/local/bin/otelcol-contrib: No such file or directory

RPM packaging for Linux amd64
otelcol-contrib_0.47.0_linux_amd64.rpm cannot start

Use git tags for 0.34.0 release

PR #4 set the versions to the SHA for the tag, but we should be using the tag instead. The SHA was being used before to track a specific unreleased commit and should not be used for the actual release.

Split goreleaser configuration per distribution

Even though GoReleaser is fast on local machines, it's really slow at CI. Two distributions with the current matrix of arch/platforms are sufficient to cause the default timeout to happen (30m).

Ideally, we would split the workflows and goreleaser files per distribution.

Document relationship between available OT Collector Docker images

Hi, there are currently at least three different images for the OpenTelemetry Collector:

• otel/opentelemetry-collector
• otel/opentelemetry-collector-dev
• otel/opentelemetry-collector-contrib

The difference between these three images does not seem to be clearly documented anywhere.

It would be good to see a section of documentation with a comparison between all available OTel Collector images. This would aid users in selecting the correct collector for their use case.

It might also be worth documenting the relationship of the three images above to platform-specific images such as aws-observability/aws-otel-collector.

Publish images to registries other than Docker Hub

Docker Hub has introduced pretty aggressive rate limits. We're trying to migrate our deployments to use either public.ecr.aws or quay.io instead, as trying to pull from Docker Hub fails with 429s frequently.

Any chance the images built here could also be published to other registries?

`latest` tag not updated

The latest tag in dockerhub isn't getting updated during the release process. This is causing user confusion.

Add link to opentelemetry.io under About section

Under the About section of this repo (which you can access from the main repo page -- click the gear icon next to the About title):

• Under Website, add https://opentelemetry.io

We're looking to do this uniformly for all OTel org repos.

For example:

How to enter collector docker container

I pull otel/opentelemetry-collector image from docker hub, then I use docker run -it --rm -d -p 4371:4371 otel/opentelemetry-collector:latest command to run it with a container.
But I cannot enter to the running cntainer, my commands like this:
docker exec -it c21e411918ae2acd5b3fffb06b394fcfa90e8f2c96c6ed3d63e5c8a5f1ff0bc6 sh
docker exec -it c21e411918ae2acd5b3fffb06b394fcfa90e8f2c96c6ed3d63e5c8a5f1ff0bc6 /bin/sh
docker exec -it c21e411918ae2acd5b3fffb06b394fcfa90e8f2c96c6ed3d63e5c8a5f1ff0bc6 bash
docker exec -it c21e411918ae2acd5b3fffb06b394fcfa90e8f2c96c6ed3d63e5c8a5f1ff0bc6 /bin/bash
the error like this:
OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"sh\": executable file not found in $PATH": unknown

Introduce released package signatures

We produce Linux, Windows and Mac OS X executables/packages. These packages are not signed which makes it difficult to verify the authenticity of the packages. Code signing is also an essential part of secure operation of OpAMP.

Ideally we need to sign the code that we release (both in this repo and in contrib).

On Unix systems we should possibly use GPG signing with detached signatures. On Windows (perhaps in addition to GPG signatures) we may need a code signing certificate to sign the EXE and MSI.

The signing certificate's private key needs to be stored privately and used for signing process in the CI. The private key needs to be guarded. The public key needs to be publicly advertised in this repository and/or at https://opentelemetry.io/

For GPG signing the resulting detached signatures must be published for each release alongside the assets that we include in the release (one signature per executable asset).

Questions:

• How do we produce the signing certificate? What is the validity period? How is it rotated?
• Where is the private key of signing certificate stored, how it is guarded? Should it be encrypted and stored in this repository? Who has the decryption keys?
• Is it acceptable that the CI/github actions have the signing certificate's decryption key via the Github's secrets management capability?
• How are the detached signatures published? Is each signature in its own file and included as a release asset?

Remove 'v' from the container image version

When publishing the v0.36.0 release, we noticed that the container image has the 'v' in the version as well, which breaks the pattern from previous versions. To keep compatibility with them, we should remove the 'v' from that.

Add all missing components to the manifest, including the non-ready ones

May I do this in a follow-up PR, if there are no other concerns?

Originally posted by @jpkrohling in #43 (comment)

Enabling debug symbols

We are contemplating whether to enable debug symbols (disabled by -s -w LDFLAGS on build) on our downstream repo aws-otel-collector. We want to be in line with what is being done upstream but it seems that there's an inconsistency. In this opentelemetry-collector-releases repo it appears that they are disabled but in opentelemetry-collector-contrib it appears to be enabled.

Is there any reason for this inconsistence/any guidance on what we should lean towards regarding enabling/disabling debug symbols?

Default config path changed in the contrib container image

Since 0.42.0 the default config.yaml path used in the contrib container image changed from /etc/otelcol/config.yaml to /etc/otelcol-contrib/config.yaml.

As far as I understand it was done here: #43

While I have not found /etc/otelcol/config.yaml in any documentation I imagine that there may be a lot of users relying on this path.

Docker inspect dumps for reference:

• 0420inspect.txt
• 0400inspect.txt

Changes to contrib collector release process, and latest tag

With 0.42.0 of the collector contrib release came about some changes that were not communicated.

The 0.42.0 image has a different binary name, going from otelcolcontrib to otelcol-contrib. Though the entry point should handle this, it is still a change that will cause pain to some deployments.

The latest tag image isn't the same as the 0.42.0 image. latest still has the old binary name, and the image itself is over 50MB, while the 0.42.0 image is around 29MB in size.

Tagging @jpkrohling on this per this CNCF slack thread: https://cloud-native.slack.com/archives/C01N6P7KR6W/p1641846859039500

`opentelemetry-collector-contrib` missing `transformprocessor`

The transformprocessor appears to be missing from the contrib Docker image. Looking in the manifest I think it's just that the line for it isn't there.

Can not start service after installed on Azure VM Ubuntu.

Describe the bug
This issue occurred only Azure VM Linux (Ubuntu).
After installed service couldn't started.

root@telemetry-machine:/home/telemetry# systemctl start otelcol.service
root@telemetry-machine:/home/telemetry# systemctl status otelcol.service
otelcol.service - OpenTelemetry Collector
    Loaded: loaded (/usr/lib/systemd/system/otelcol.service, enabled)
    Active: failed (failed)

Steps to reproduce
Use azure VM Linux and install otelcol following this document.

**What did you see instead? **

(Reading database ... 190438 files and directories currently installed.)
Preparing to unpack otelcol_0.44.0_linux_amd64.deb ...
Unpacking otelcol (0.44.0) over (0.44.0) ...
Setting up otelcol (0.44.0) ...
dpkg: error processing package otelcol (--install):
 installed otelcol package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 otelcol

**What version did you use? **
Version: 0.44.0

**What config did you use? **
Config: just default configuration file.

Environment

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal

Remove Darwin 32-bit release

As reported by @hickeyma:

That makes sense. I have updated the matrix to match the opentelemetry-collector-releases matrix. There is one difference and that is the Darwin 386 build is no longer supported by Go. This is because it dropped support for 32-bit binaries on macOS in Go 1.15.

Originally posted by @hickeyma in open-telemetry/opentelemetry-collector#5142 (comment)

Windows MSI

Is there a windows msi installer release for the collector?

Prepare otelcol-releases v0.49.0

Release fails as base container images are not using the new base name

Seen during the first attempt to release 0.36.0:

   • publishing       
      • docker images    
         • pushing docker image      image=otel/otelcol:v0.36.0-amd64
      • docker manifests 
         • creating docker manifest  images=[otel/opentelemetry-collector:v0.36.0-amd64] manifest=otel/opentelemetry-collector:v0.36.0
   ⨯ release failed after 1266.48s error=docker manifests: failed to publish artifacts: failed to create otel/opentelemetry-collector:v0.36.0: exit status 1: no such manifest: docker.io/otel/opentelemetry-collector:v0.36.0-amd64

Check in generated code - go.mod and main...

At the moment the generated code - main.go, components.go. go.mod and go.sum are not checked in.

Versioning these files gives users a better understanding of what the final distribution does and what is the dependency stack. One can argue that dependencies are versioned in the otel-collector-builder configuration files, however, the otel-collector-builder is still a black-box for user.

Another reason why I/Red Hat would like to see checked in the generated code is related to our internal tooling for building open-source components. Our build tooling expects checked-in all source code (especially go.mod).

Add ARM binaries and images back

The ARM binaries and images were first added via #39 and removed via #41. This ticket is to track it being added back once the failures are sorted out.

An option to keep the debug information in the collector binary

It's hard to impossible to debug a collector binary using gdb due to stripped DWARF and symbols tables.
https://github.com/open-telemetry/opentelemetry-collector/blob/main/cmd/builder/internal/builder/main.go#L89

It would be wonderful to have an option in the builder configuration to keep the debug information. Possibly it could be generalized to a pass of any arguments to the go build command call with the default value ["-ldflags=-s -w", "-trimpath"] to keep the compatibility.