On kubernetes 1.7.3 the data is null

[giantswarm]

On kubernetes 1.7.3 the data is null

[justlooks]

i update my cluster to v1.7.5 ,still get some problem on get metrics from prometheus

i add the following content to prometheus.yaml as document says

      - job_name: 'kubernetes-cadvisor'
        scheme: https
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
          - role: node
        relabel_configs:
          - action: labelmap
            regex: __meta_kubernetes_node_label_(.+)
          - target_label: __address__
            replacement: kubernetes.default.svc:443
          - source_labels: [__meta_kubernetes_node_name]
            regex: (.+)
            target_label: __metrics_path__
            replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor

but prometheus get error info "server returned HTTP status 403 Forbidden", the "Kubernetes Pods resource" panel still no data

[chenww]

same. here is the log:
Error from server (BadRequest): a container name must be specified for pod node-directory-size-metrics-bjzdv, choose one of: [read-du caddy]

configuration is wrong?

[txg1550759]

k8s version 1.7 ，metrics not data for dashboard ! this is bug ?

[txg1550759]

More than 1.7 k8s of grafana do not have data, hope to be able to fix it, thanks！

[canghai908]

I have slove this, I add this in configmap on prometheus-core

-  job_name: 'kubernetes-cadvisor'
    scheme: http
    tls_config:
      ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
    kubernetes_sd_configs:
      - role: node
    relabel_configs:
      - source_labels: [__address__]
        regex: '(.*):10250'
        replacement: '${1}:4194'
        target_label: __address__

[arrkaye]

Is that the word 'address' or did you put the address of something there?

[MarcelMue]

@arrkaye I just edited his comment to use a code block, I think __address__ is a keyword here. I will look into it later today to hopefully merge a fix.

[frittenlab]

justlooks fix worked for me. I have data in my dashboard now. Running kubernetes 1.7.5

[like-inspur]

But I can't get metric data on kubernetes 1.8.2，so why? @simon-k8s @canghai908

[frittenlab]

@like-inspur did you add the content that @justlooks suggested to the prometheus.yaml?

[like-inspur]

@simon-k8s OK ,it works, thanks!

[pipo02mix]

It should be ok with the current master. Feel free to reopen if it still happens to you