prometheus/common

Go libraries shared across Prometheus components and libraries.

Prometheus

Visit prometheus.io for the full documentation, examples and guides.

Prometheus, a Cloud Native Computing Foundation project, is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts when specified conditions are observed.

The features that distinguish Prometheus from other metrics and monitoring systems are:

• A multi-dimensional data model (time series defined by metric name and set of key/value dimensions)
• PromQL, a powerful and flexible query language to leverage this dimensionality
• No dependency on distributed storage; single server nodes are autonomous
• An HTTP pull model for time series collection
• Pushing time series is supported via an intermediary gateway for batch jobs
• Targets are discovered via service discovery or static configuration
• Multiple modes of graphing and dashboarding support
• Support for hierarchical and horizontal federation

Architecture overview

Install

There are various ways of installing Prometheus.

Precompiled binaries

Precompiled binaries for released versions are available in the download section on prometheus.io. Using the latest production release binary is the recommended way of installing Prometheus. See the Installing chapter in the documentation for all the details.

Docker images

Docker images are available on Quay.io or Docker Hub.

You can launch a Prometheus container for trying it out with

docker run --name prometheus -d -p 127.0.0.1:9090:9090 prom/prometheus

Prometheus will now be reachable at http://localhost:9090/.

Building from source

To build Prometheus from source code, You need:

• Go version 1.17 or greater.
• NodeJS version 16 or greater.
• npm version 7 or greater.

Start by cloning the repository:

git clone https://github.com/prometheus/prometheus.git
cd prometheus

You can use the go tool to build and install the prometheus and promtool binaries into your GOPATH:

GO111MODULE=on go install github.com/prometheus/prometheus/cmd/...
prometheus --config.file=your_config.yml

However, when using go install to build Prometheus, Prometheus will expect to be able to read its web assets from local filesystem directories under web/ui/static and web/ui/templates. In order for these assets to be found, you will have to run Prometheus from the root of the cloned repository. Note also that these directories do not include the React UI unless it has been built explicitly using make assets or make build.

An example of the above configuration file can be found here.

You can also build using make build, which will compile in the web assets so that Prometheus can be run from anywhere:

make build
./prometheus --config.file=your_config.yml

The Makefile provides several targets:

• build: build the prometheus and promtool binaries (includes building and compiling in web assets)
• test: run the tests
• test-short: run the short tests
• format: format the source code
• vet: check the source code for common errors
• assets: build the React UI

Service discovery plugins

Prometheus is bundled with many service discovery plugins. When building Prometheus from source, you can edit the plugins.yml file to disable some service discoveries. The file is a yaml-formated list of go import path that will be built into the Prometheus binary.

After you have changed the file, you need to run make build again.

If you are using another method to compile Prometheus, make plugins will generate the plugins file accordingly.

If you add out-of-tree plugins, which we do not endorse at the moment, additional steps might be needed to adjust the go.mod and go.sum files. As always, be extra careful when loading third party code.

Building the Docker image

The make docker target is designed for use in our CI system. You can build a docker image locally with the following commands:

make promu
promu crossbuild -p linux/amd64
make npm_licenses
make common-docker-amd64

Using Prometheus as a Go Library

Remote Write

We are publishing our Remote Write protobuf independently at buf.build.

You can use that as a library:

go get buf.build/gen/go/prometheus/prometheus/protocolbuffers/go@latest

This is experimental.

Prometheus code base

In order to comply with go mod rules, Prometheus release number do not exactly match Go module releases. For the Prometheus v2.y.z releases, we are publishing equivalent v0.y.z tags.

Therefore, a user that would want to use Prometheus v2.35.0 as a library could do:

go get github.com/prometheus/[email protected]

This solution makes it clear that we might break our internal Go APIs between minor user-facing releases, as breaking changes are allowed in major version zero.

React UI Development

For more information on building, running, and developing on the React-based UI, see the React app's README.md.

More information

• Godoc documentation is available via pkg.go.dev. Due to peculiarities of Go Modules, v2.x.y will be displayed as v0.x.y.
• See the Community page for how to reach the Prometheus developers and users on various communication channels.

Contributing

Refer to CONTRIBUTING.md

License

Apache License 2.0, see LICENSE.

common's Issues

Bump minimum version to 1.6 and get rid of `GO15VENDOREXPERIMENT`?

Subject says it all.

Update MAINTAINERS.md

Current MAINTAINERS.md lists @fabxc as the maintainer, but with his CoreOS mail address (which presumably wouldn't work anymore as he doesn't work for CoreOS anymore).

There has been very little commit activity an this repo recently, but there are a bunch of open PRs, new and old, and a lot of meaningful issues, again new and old. Most of recent activities on issues and PRs is (unsurprisingly) by @brian-brazil . The current maintainer doesn't seem to have shown any recent activity here.

In short: This repo appears abandoned. However, it is used by many others across the project, thus having this properly maintained is crucial. Many of the changes here, and in particular some of the long-standing open issues, touch quite basic decisions that have impact on the project as a whole and need consistency across the project (including the meta issue of what should go into common at all – the different opinions about this have been colliding here and there over the last years, up to the current day… #97 is just the tip of the iceberg). I would love to see somebody who is not only actively maintaining this but also has the ability to mediate all those basic decisions into a productive consensus.

Log: Sylog logs should not contain cee tag unless they are json formatted

Logs sent to syslog currently tag logs (regardless of format) with the @cee: tag.

$ prometheus -log.format logger:syslog?appname=pm\&local=7

Produces these incorrect logs in syslog:

Dec 18 09:26:48 Argon pm[62206]: @cee:ERRO[0000] Could not lock data/DIRTY, Prometheus already running?  source=persistence.go:181
Dec 18 09:26:48 Argon pm[62206]: @cee:ERRO[0000] Error opening memory series storage: resource temporarily unavailable  source=main.go:184

$ prometheus -log.format logger:syslog?appname=pm\&local=7\&json=true

Produces these correct logs in syslog:

Dec 18 09:25:04 Argon pm[62168]: @cee:{"level":"error","msg":"Could not lock data/DIRTY, Prometheus already running?","source":"persistence.go:181","time":"2016-12-18T09:25:04-05:00"}
Dec 18 09:25:04 Argon pm[62168]: @cee:{"level":"error","msg":"Error opening memory series storage: resource temporarily unavailable","source":"main.go:184","time":"2016-12-18T09:25:04-05:00"}

Syslog log entries should not be cee-enhanced unless they are json formatted.

expfmt: Handle duplicate buckets and quantiles during decoding

See prometheus/pushgateway#51 for a detailed example.

While originally, this was thought to be a responsibility of the text parser, it could also happen in the same way in the protobuf format. It has to be detected during decoding in any case.

Logrus case change

the log package imports "github.com/Sirupsen/logrus" but it should be "github.com/sirupsen/logrus" check it here. This is painful because I use Logrus in my project and I don't want to have the same library in different folders with different names.

Circular dependency with client_golang

Hey,

I just got reported (I had not noticed), that there is a circular dependency between common and client_golang, which is pretty problematic for transitions, backports, etc.

In the file version/info.go, the client library is imported to create metrics with version information. Is there any sane way to break this cycle? Otherwise, I guess I should just merge the packages; they can't be used independently anyway.

How to pass a httprouter.Handle to a prometheus http.HandleFunc

Cannot pass prometheus midware into httprouter endpoint definitions.

I'm trying to add a prometheus midware into our endpoint implementation. But our endpoint are using a third party mux package called httprouter. Then when I tried to add this midware into existing code base, I cannot find a good way to integrate both together.

router := httprouter.New()
router.GET("/hello", r.Hello)

func (r configuration) Hello(w http.ResponseWriter, req *http.Request, ps httprouter.Params)

func InstrumentHandlerFunc(name string, handler http.HandlerFunc) http.HandlerFunc {
    counter := prometheus.NewCounterVec(
		do something...
	)

	duration := prometheus.NewHistogramVec(
		do something...
	)
	return promhttp.InstrumentHandlerDuration(duration,
		promhttp.InstrumentHandlerCounter(counter, handler))
}

Below is what I want to do

func InstrumentHandlerFunc(name string, handler httprouter.Handle) httprouter.Handel {

}

router.Get("/hello", InstrumentHandlerFunc("/hello", r.Hello))

Consider using inlined fnv hasher for signature.

As done in prometheus/client_golang#181, we can probably avoid some allocation churn and lock contention in signature.go.

TextParser: parsing timestamps for summaries and histograms

Hey!

It looks like: https://github.com/prometheus/common/blob/master/expfmt/text_parse.go#L448
there is no validation that for histograms or summaries timestamps are the same across buckets. quantiles, count and sum. Basically last specified timestamp wins.

Implement CI

Implement a CI like travis or something would be great because mistakes that cause a compile error could be easily avoided (or at least detected).

Missing License

Hi,

A new dependency, a new Debian package! :-)

There is a small problem with this one: even if most files have a copyright and license statement, not all of them have, and you don't have an explicit one at the root.of the project either. In particular, these source files have no attribution:

route/route_test.go
route/route.go

And same thing with the test data.

Thanks!

Syslog and eventlog "formatters" are wrong

Both are hooking into logrus' formatting mechanism. This is meant to create a byte slice, which logrus then writes to a sink. Instead both formatters write to their own sync as a side effect. The formatted log lines proceed being printed into the original sink, i.e. stdout.

The side effect is obviously bad as there's stuff logrus does internally afterwards.
But it's also just inconsistent from the configuration side. We pretend the flag flips the sink, when in fact it just adds one that as written to in a non-intended way.

The configuration aspect is possibly too far along to fix.

proxy_url: can contain secrets

Not sure if this is a bug or a feature request.

What did you do?

We use a proxy in our config of the form proxy_url: "http://proxy_user:proxy_password@...", because this isn't considered a Secret, these show up in our Status pages in alertmanager. and config pages on Prometheus web ui. Forcing us to block these pages

What did you expect to see?

I'd hoped we could mask these out, or mark the proxy as Secret

What did you see instead? Under which circumstances?

I saw the password.

Environment

• System information:

/ # uname -srm
Linux 4.4.98-1.el7.elrepo.x86_64 x86_64

• Alertmanager version:

/ # alertmanager --version
alertmanager, version 0.15.0-rc.0 (branch: HEAD, revision: 68122e7005958df611baec01b316355a6f408bae)
  build user:       root@46262c45c129
  build date:       20180228-11:57:18
  go version:       go1.10

I'd be happy to help with this, although I'm not 100% sure of the best approach - maybe SecretURL - or maybe the responsibility should be in the respective UI's to mask the possibly secret part of any url out.

expfmt.Decoder rejects an endpoint Prometheus TSD successfully scrapes

Commit used:
prometheus/common: HEAD
prometheus/prometheus: HEAD

The common module of prometheus is used by elastic/beats https://github.com/elastic/beats/blob/master/metricbeat/helper/prometheus.go#L27 to do scraping of prometheus end points. The following endpoint which comes from envoyproxy https://github.com/envoyproxy/envoy/blob/master/source/server/http/admin.cc :
https://gist.githubusercontent.com/vjsamuel/5898d02f8039d8d1bae4d8642182d1c3/raw/deff0c16f746bb2ab7a959c4d5ed9856d17c425d/gistfile1.txt

is successfully scraped by the prometheus TSD and fails to be decoded by the Decoder with error:

text format parsing error in line 9: second TYPE line for metric name "envoy_listener_http_downstream_rq_xx", or TYPE reported after samples

Can there be consistency in the common module with the scraper module in prometheus TSD?

expfmt: Consider filter the illegal metrics rather than drop all the metrics

common/expfmt/decode.go

Lines 104 to 108 in 2f17f4a

	if !model.LabelValue(l.GetValue()).IsValid() {
	return fmt.Errorf("invalid label value %q", l.GetValue())
	}
	if !model.LabelName(l.GetName()).IsValid() {
	return fmt.Errorf("invalid label name %q", l.GetName())

Rather than return error cause losing the whole metrics, it would be reasonable to just filter them and log it.

Using addlog as constraint for direct dep github.com/prometheus/common

When using dep tool for managing dependencies, I ran into the following issue.

$ dep init
...
  Using addlog as constraint for direct dep github.com/prometheus/common
  Locking in addlog (89d8028) for direct dep github.com/prometheus/common

For whatever reason, dep prefers addlog branch to master branch.

The relevant entry in Gopkg.toml file is:

[[constraint]]
  branch = "addlog"
  name = "github.com/prometheus/common"

The Gopkg.lock looks like this:

[[projects]]
  branch = "addlog"
  digest = "1:44ee685ecc3c1950dcbc605e1f519cc8c3f462c9dd4e11576892191ffbf4a3b9"
  name = "github.com/prometheus/common"
  packages = [
    "expfmt",
    "internal/bitbucket.org/ww/goautoneg",
    "log",
    "model",
    "version",
  ]
  pruneopts = "UT"
  revision = "89d80287644767070914e30199b4d959e491bd3d"

The above results in broken builds.

remove checkOverflow workaround from the config package

checkOverflow and XXX map[string]interface{} yaml:",inline"are no longer needed as the yaml package introduced theUnmarshalStrict()` to detect unknown or duplicated fields.

After removing checkOverflow need to find all packages that use common/config , update the yaml package and start using the new UnmarshalStrict().

Info and color codes logged to stdout

From @jsha on September 4, 2016 22:19

Steps to reproduce:

node_exporter --log.format 'logger:syslog?appname=node_exporter&local=7' &
less /var/log/syslog

Expected result: node_exporter log lines don't contain spelled-out severities or ANSI color codes.

Actual result: node_exporter log lines do contain severities and color codes.

Sep  4 22:16:31 btudb node_exporter[15767]: @cee:#033[34mINFO#033[0m[0000] Starting node_exporter (version=0.12.0, branch=master, revision=df8dcd2)  #033[34msource#033[0m=node_exporter.go:135
Sep  4 22:16:31 btudb node_exporter[15767]: @cee:#033[34mINFO#033[0m[0000] Build context (go=go1.6.2, user=root@ff68505a5469, date=20160505-22:14:18)  #033[34msource#033[0m=node_exporter.go:136
Sep  4 22:16:31 btudb node_exporter[15767]: @cee:#033[34mINFO#033[0m[0000] No directory specified, see --collector.textfile.directory  #033[34msource#033[0m=textfile.go:57

Copied from original issue: prometheus/node_exporter#298

expfmt should parse HELP text

See: prometheus/prometheus#4602

promtool is using expfmt https://github.com/prometheus/prometheus/blob/master/util/promlint/promlint.go#L68 which doesn't check the HELP text.

Feature: HTTP Config Additional Headers

As slightly touched on in prometheus/prometheus#4314 having the ability to provide custom headers for webhooks would solve any bespoke integration with 3rd party API's that do not conform to strict authorization formats.

Subpackage version imports repo client_golang which imports this repo…

This does not cause an import cycle while compiling, so no one’s builds have broken, and go does a decent job at caching the go.mod downloads, so no one has really noticed the stress yet. But just importing the most recent version of client_golang (v1.4.1) ends up cascading through three different versions of client_golang and two versions of common.

https://github.com/prometheus/client_golang/blob/v1.4.1/go.mod#L11 depends on common v0.9.1
https://github.com/prometheus/common/blob/v0.9.1/go.mod#L13 depends on client_golang v1.0.0
https://github.com/prometheus/client_golang/blob/v1.0.0/go.mod#L10 depends on common v0.4.1
https://github.com/prometheus/common/blob/v0.4.1/go.mod#L17 depends on client_golang v0.9.1
https://github.com/prometheus/client_golang/blob/v0.9.1 has no go.mod so this is the end of the line.

As interdependent versions progress, this cyclic dependency is likely to only grow longer and longer, potentially exponentially exploding the results of go mod graph in turn. It might be a good idea to consider how to deal with it now, rather than later.

expfmt: Don't panic on decoding corrupt data

https://github.com/prometheus/common/blob/master/expfmt/decode.go#L182 panics upon an unknown metric family type. This can be triggered by corrupt input data. We should return an error instead to not crash e.g. the whole Prometheus server during ingesting metrics with corrupted data.

TextParser: Allow streaming

Recently, we have made the definition of the line-based text format more restrictive. Lines for one metnic family must be continuous, so we can now implement streaming. Change the interface of the parser to a channel spitting out metric families (and provide the old interface for compatibility, noting that it is less efficient).

Allow rule labels with slash

Spec for PrometheusRule notes that labels may match selectors of replication controllers and services. According to this labels like app.kubernetes.io/name are recommended for kubernetes. But apparantly / is not allowed here and I get errors like this:

level=error ts=2019-03-21T14:32:57.642600466Z caller=manager.go:754 component="rule manager" msg="loading groups failed" err="group \"analytics-csgo-stats-api-dev.rules\", rule 0, \"5xxErrors\": invalid label name: app.kubernetes.io/name"

Syslog and Eventlog sinks for promlog

The new go-kit based logger only prints to stderr. Should we add the syslog and eventlog sinks that are present in the previous logger?

@brian-brazil @grobie @fabxc

Internally vendor all packages that are not exposed in any types exported by prometheus/common

I can't build my project which I believe is because of this...

godep version godep v71 (linux/amd64/go1.6) doesn't seems to import the github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg directory to vendor so I got a:

$ go install ./...
....
  imports github.com/prometheus/common/expfmt
  imports github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg: use of internal package not allowed

Linking to godep issue tools/godep#490

TextParser: Tolerate trailing whitespace

Apparently, trailing whitespace is not tolerated, see prometheus/prometheus#1469 .

Silence model in common is out of sync with Alertmanager

The struct definition for a Silence in common/model is out of sync with the definition in alertmanager/types, for example common/model defines the ID field as a uin64 whereas it is now a string in alertmanager/types

Perhaps as Silences are only used in Alertmanager and not Prometheus the definition should be removed from common/model?

NewClientFromConfig has no mechanism to set timeouts

NewClientFromConfig calls NewRoundTripperFromConfig which returns an http.RoundTripper. That interface has no mechanism to set dial options (like dial timeout, etc.). Under the hood the DialContext is set to one from go-conntrack -- which supports passing a dialer. I'd like a mechanism to pass these dial options in. The main reason being that as it stands today if the scrape target etc. is blocking the packets or slow the scrape hands for quite a while (holding FDs, etc.).

I see a couple options:

• we add some options to the HTTPClientConfig for this
• we add a new method NewClientFromConfigWithDialer which takes a dialer and does the "right thing"
• something else I haven't thought of yet

Block any further extractions to common/config

I want this issue to serve as a canonical blocker to extract any further structs into common/config.
I consider it largely harmful and do not want this rabbit hole to grow until we've given our overall strategy on shared functionality a revisit.

That's a longer discussion we shouldn't have here, so I'll elaborate on the full details elsewhere later.

Support JSON formatting of logs

common/log supports a json formatter for the log output, which common/promlog does not. Would a PR to add a flag for switching between json/plain text be acceptable? This would make it much easier to integrate exporters etc into environments which mandate structured logging (vs having an extra binary to rewrite the logs).
go-kit already has the necessary NewJSONLogger function, which could be conditionally used instead of NewLogfmtLogger into here:

common/promlog/log.go

Lines 58 to 63 in c7de230

	func New(al AllowedLevel) log.Logger {
	l := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
	l = level.NewFilter(l, al.o)
	l = log.With(l, "ts", log.DefaultTimestampUTC, "caller", log.DefaultCaller)
	return l
	}

Can`t build service for kubernetes

Can`t build service, after i added k8s "github.com/micro/kubernetes/go/micro" import

go: github.com/micro/[email protected]: unknown revision v0.6.0
go: github.com/micro/[email protected]: unknown revision v0.14.0
go: finding sigs.k8s.io/structured-merge-diff v0.0.0-20190130003954-e5e029740eb8
go: finding github.com/micro/go-grpc v0.11.0
go: error loading module requirements

Alert struct need a SampleValue type var

When we send alert we want to know the current value of a metric, so I think this value should also post to alert manager.

Please version prometheus libs too

It would be mightily fine if common prometheus packages were reved like prometheus and its exporters, instead of having golden prometheus versions depend on magic vendor snapshots

That would help integrate prometheus in linux distributions

Extend model.Duration with `Miliseconds()` method as well as `model.{Milisecond,Second, ...}` vars.

Hi 👋

I would propose extending model.Duration. It's super painful to work with and prone to errors. Examples of lack of Seconds or model.Milisecond:

• int64(db.opts.RetentionDuration)/int64(time.Millisecond) to convert to miliseconds in Prometheus
• 15 * 24 * model.Duration(time.Hour)
• model.Duration(time.Duration(DefaultBlockDuration) * time.Millisecond), 😱

I was thinking about adding Miliseconds method (we use miliseconds in Prometheus a lot!) as well as adding all model.Milisecond model.Days model.Hours model.Seconds model.Microsecond model.Nanosecond vars.

What do you think? Would we accept just contribution? @beorn7 @brian-brazil @SuperQ

Build error concerning mf

go vet ./... prints:
`# github.com/prometheus/client_golang/prometheus

../../../../github.com/prometheus/client_golang/prometheus/http.go:88:24: cannot use mf (type *"github.com/prometheus/client_model/go".MetricFamily) as type *"github.com/prometheus/common/vendor/github.com/prometheus/client_model/go".MetricFamily in argument to enc.Encode

../../../../github.com/prometheus/client_golang/prometheus/registry.go:558:41: cannot use mf (type *"github.com/prometheus/client_model/go".MetricFamily) as type *"github.com/prometheus/common/vendor/github.com/prometheus/client_model/go".MetricFamily in argument to expfmt.MetricFamilyToText`

Consistently treat a label with an empty value as non-existant

Currently, the signature AKA fingerprint of a metric changes if a label with an empty value is added.

Unable to print logs to a file

Hey, I am unable to print logs to a file.

f, err := os.OpenFile("logs.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666) if err != nil { log.Fatal(err) } defer f.Close() log.NewLogger(f) log.Info("Sucesssss")

This is going to console and I think my issue is related with #82 . What is happening and how can this be fixed?

Introduce a version indentifier

I know that the export format is not supposed to change any more, but never say never...

prometheus_metric_version or even something more generic like metric_exposure_version would allow for upgrades, and for queries to see if anything still runs old versions prior to upgrading.

The example for -log.format is unclear/incorrect

The current example for the
-log.format parameter used e.g. node_exporter is unclear.

It's

`Set the log target and format. Example: "logger:syslog?appname=bob&local=7" or "logger:stdout?json=true"`,

Source

should be

`Set the log target and format. Example: "'logger:syslog?appname=bob&local=7'" or "'logger:stdout?json=true'"`,

Without additional single quotes it's not possible to use that example - tested under CentOS 6.x.

expfmt: Implement OpenMetrics generation

To support OpenMetrics in v1 of prometheus/client_golang, we need to add OpenMetrics generation to the expfmt package, effectively a proto-to-OpenMetrics converter.

For this to work, we need to add the needed additional fields to metrics.proto. This should be backwards compatible for still existing protobuf consumers, while it provides an "inofficial" protobuf representation for OpenMetrics as a byproduct. This has the potential of creating confusion. However, the prometheus/client_model repo is anyway mostly deprecated, so we might as well abuse it for this "hack".

Update: Basic support is done. Here are the remaining TODOs (some of them will require more or less breaking changes, so we might consider bundling them and perhaps even starting to use major version numbers for this repo):

• Feature support:
  • # UNIT.
  • _created.
  • info type.
  • stateset type.
  • gaugehistogram type.
• Figure out how to deal with the _total suffix. (Currently, we require it to be in the metric name and truncate it for TYPE/HELP, which is arguably the opposite of what would be expected but which simplifies the transition from current Prom text format usage.)
• Reject exemplars that are too long.
• Reject counters with NaN
• More sophisticated timestamp rendering for both samples and exemplars (to not run into precision and overflow issues with float64 and time.UnixNano).
• Get rid of NegotiateIncludingOpenMetrics (but we have to make sure there remains some way of letting users opt out of OpenMetrics to support legacy setups – which could be on the Prometheus server side by making the Accept header configurable).
• Add the Close method directly to the Encoder interface.

Optimize `MetricFamilyToText`

This code has been written with the assumption it would never be a in a hot code path. With Prometheus 2.x dropping protobuf support, scrapes by Prometheus are now always using the text format, and thus every monitored Go program is currently running this code quite a lot. (@mxinden has just told me how the kube-state-metrics CPU profiling is dominated by labelPairsToText.)

client_golang will undergo more invasive changes in the future, which will provide opportunities to solve the problem in another way, but MetricFamilyToText has a lot of low hanging fruit to make the current usage of client_golang more efficient.

Allow additional characters in label names

This is in response to the closed issue #180. I would like to understand the consequence of not following the Prometheus label naming? Currently the official Prometheus exporters for every language contain characters outside this list such as slashes. Even the page about valid label names for exporters contains an example using slashes directly underneath https://prometheus.io/docs/concepts/data_model/#notation, which I think confuses users (myself included):

For example the official https://github.com/prometheus/node_exporter uses ".", "-" - both of which I myself use because they are good for representing canonical locations of pkgs and urls. My main concern is down the road having all my metrics break, because I've made a huge commitment to Prometheus and community provided exporters.

It would be catastrophic to the ecosystem if suddenly all the official exporters, middlewares, tooling and such all stopped working. Knowing this, if a guarantee is made that no such breakage will occur, it makes sense to me they are just adopted as allowed to cement this promise.

Can't handle federation text output

Prometheus can't scrape the text version of it's own federation output, e.g.

# TYPE cpu untyped
cpu{job="promclient_triton_5s",instance="container-metrics:9092",gs_instanceid="aabd3924",gs_svcuser="svcuser2",type="idle",gs_dc="us-east-1",gs_env="dev",id="prom-1"} 0.0886400014181114 1473307588759
# TYPE cpu untyped
cpu{instance="container-metrics:9092",gs_instanceid="aabd3924",gs_svcuser="svcuser2",type="wait",gs_dc="us-east-1",gs_env="dev",job="promclient_triton_5s",id="prom-1"} 0.6554035022739826 1473307588759
# TYPE cpu untyped
cpu{instance="container-metrics:9093",job="promclient_triton_5s",gs_dc="us-east-1",gs_env="dev",gs_instanceid="231deaa1",gs_svcuser="svcuser2",type="sys",id="prom-1"} 0.3062938875174634 1473307586226
# TYPE cpu untyped
cpu{job="promclient_triton_5s",gs_instanceid="fb310edz",gs_svcuser="svcuser1",type="user",gs_dc="us-east-1",instance="container-metrics:9091",gs_env="dev",id="prom-1"} 0.5589919972685718 1473307585070

From https://groups.google.com/forum/#!topic/prometheus-developers/ZVb2NjjUJ5A

Logger does not implement promhttp.Logger

promhttp.Logger requires Println(v ...interface{}). This means I can't use common/log.Base() in my exporter without adapter.

expfmt: Consider allowing empty MetricFamilies

Currently, a MetricFamily with no Metrics is invalid (i.e. it will never be created by encoders, and it will yield an error upon decoding).

As it turns out, client libraries that are ignorant of the protobuf exchange format are making use of empty metric families as represented in the text format, i.e.

# HELP blah some help
# TYPE blah gauge
# HELP blub another metric
# ...

This has never caused any trouble because the only entry point of the text format into the Prometheus universe is the text-format parser, which kicks out those metric families here.

@brian-brazil thinks empty metric families is something we should allow in general. That would require to change all code encoding and decoding protobuf, including the text-format parser. (Ideally, all that code is only in expfmt but there might be legacy.) Also, consumers of decoded protobuf have to be checked to deal gracefully with empty MetricFamilies, as they were so far guaranteed to not appear.

Need to fix last broken commit

Last commit was broken 8be020a

model.ParseDuration bug

"1m0s" can be parsed normally by time.ParseDuration (https://play.golang.org/p/QbEuk34UKSm) but not by model.ParseDuration:

not a valid duration string: "1m0s"

Introduce a termination marker for scrapes

To positively confirm the completion of a scrape (rather than only negatively detect an incomplete scrape by a connection error), a termination marker is needed.

Proposal:

If an exposition is meant to be terminated by a termination marker, it's content-type contains an additional parameter termination=marked. (It will still be possible to not use termination markers so that trivial one-line expositions are still possible by default.) Termination can be requested via the usual content-type negotiation, and should only be applied if requested.

With termination=marked, the following termination marker has to end the payload:

• A line # END\n for the text format.
• A uvarint-encoded value of MaxUint32 in case of protobuf. (Has a length of 5 bytes - protobufs of that size cannot be encoded anymore but the value is any waymeant to be larger than a protobuf can reasonably be. Some protobuf implementation limit the size of a proto message at 64MiB anyway.)

An exposition with termination=marked and no termination marker is considered incomplete, which should be treated the same as an ingestion interrupted by a connection error.

Dependency on bitbucket.org/ww/goautoneg is the only mercurial dependency

Hi! In various places, I don't have mercurial available (for example, CoreOS). I've also taken to using git submodules for vendoring. Out of the 7 external packages required to import prometheus/client_golang, the only non-git dependency is bitbucket.org/ww/goautoneg. See https://godoc.org/github.com/prometheus/client_golang/prometheus?import-graph&hide=2.

ww/goautoneg has not had a commit since 2012. It's a very small amount of code: https://bitbucket.org/ww/goautoneg/src/75cd24fc2f2c2a2088577d12123ddee5f54e0675/autoneg.go?at=default&fileviewer=file-view-default

Any chance the source could be imported wholesale, and live where it is needed? (perhaps to /internal/autoneg, for example).

---

As a side note in the project I'm using this, there are 26 external repositories. All of which live in git repositories and happily live in the git submodule ecosystem. This is the first package in more than a year of development on this project where I've hit a non-git dependency.