topine/ibm-spectrum-exporter

Prometheus Exporter for IBM Spectrum Control

ibm-spectrum-exporter's Issues

Error parsing pool parameters values when locale is not in EN.

I'm getting a theese erors in my logs. It seem to have issues with collecting and/or publishing the metrics. Log entries like the one attached are generetated every time I access the published metrics. I am using your docker container with standard configs.

If I access the metrics with wget, as suggested in IBM's manual A typcial valeue will read:

"Total Volume Capacity": "821\u00a0272,08"

Parsing the metrics file with jq, the same value will be:

"Total Volume Capacity": "821 272,08",

My guess is that your code is not decoding '\u00a0' correctly.

ref: https://www.ibm.com/support/knowledgecenter/SS5R93_5.3.6/com.ibm.spectrum.sc.doc/mgr_rest_api_retrieving_cli.html

2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "98\u00a034340": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "241\u00a067021": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "340\u00a001361": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "382\u00a043104": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "116\u00a064281": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "223\u00a037080": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "340\u00a001361": invalid syntax
github.com/topine/ibm-spectrum-exporter/collector.(*poolCollector).Update
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/pool.go:73
github.com/topine/ibm-spectrum-exporter/collector.(*IbmSpectrumCollector).Collect.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/collector/collector.go:135
2020-06-29T06:01:05.252Z        ERROR   collector/pool.go:73    Error converting values.strconv.ParseFloat: parsing "438\u00a084104": invalid syntax

Error message from spectrum breaks storage_metrics

Adding certain metrics IDs to the metrics_config.yaml seems to break the collection of metrics. (see attachd logs)

The metric ID in this specific case is 1029. I added it to my metrics_conf.yaml like this:

metrics:
  storage_systems:
    - ibm_spectrum_metric_id: 1029
      prometheus_name: storage_invalid_link_transmission_rate
      prometheus_help: The average number of times per second that an invalid transmission word was detected by the port while the link did not experience any signal or synchronization loss.

I have no problems accesseing the metric by itself:

wget --no-check-certificate --load-cookies cookies.txt https://spectrumcontrol:9569/srm/REST/api/v1/StorageSystems/9669/Performance/1029

returns:

[
   {
      "metricDetails": {
         "1029": {
            "description": "The average number of times per second that an invalid transmission word was detected by the port while the link did not experience any signal or synchronization loss.",
            "name": "Invalid Link Transmission Rate",
            "units": "cnt\/s"
         }
      }
   },
   {
      "current": [
         {
            "x": 1593676221744,
            "y": null
         },
         {
.. blanked out for readability
         {
            "x": 1593762621744,
            "y": null
         }
      ],
      "deviceId": 9669,
      "deviceName": "blanked-out-before-sharing-publically",
      "endTime": 1593762621744,
      "label": "Invalid Link Transmission Rate",
      "maxValue": 0.0,
      "metricId": 1029,
      "minValue": 0.0,
      "precision": 4,
      "resourceID": 9669,
      "startTime": 1593676221744,
      "units": "cnt\/s"
   }
]

metric collection still works for switches and pools. But all storage metrics are gone., the logs gives the following information:

2020-07-03T08:06:50.092Z        INFO    ibm-spectrum-exporter/main.go:122       Starting to collect the metrics.
2020-07-03T08:06:50.165Z        INFO    spectrumservice/client.go:427   Number of Switches retrieved: 24
2020-07-03T08:06:50.266Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3071 : 1 .
2020-07-03T08:06:50.312Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3068 : 1 .
2020-07-03T08:06:50.340Z        INFO    spectrumservice/client.go:449   Metrics received for switch 125761 : 0 .
2020-07-03T08:06:50.381Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3055 : 1 .
2020-07-03T08:06:50.430Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3032 : 1 .
2020-07-03T08:06:50.476Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3065 : 1 .
2020-07-03T08:06:50.524Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3010 : 1 .
2020-07-03T08:06:50.572Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3021 : 1 .
2020-07-03T08:06:50.617Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3043 : 1 .
2020-07-03T08:06:50.650Z        INFO    spectrumservice/client.go:513   Number of Pools retrieved: 2
2020-07-03T08:06:50.663Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3074 : 1 .
2020-07-03T08:06:50.717Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3051 : 1 .
2020-07-03T08:06:50.768Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3040 : 1 .
2020-07-03T08:06:50.821Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3061 : 1 .
2020-07-03T08:06:50.869Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3018 : 1 .
2020-07-03T08:06:50.915Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3029 : 1 .
2020-07-03T08:06:50.948Z        INFO    spectrumservice/client.go:449   Metrics received for switch 125757 : 0 .
2020-07-03T08:06:50.978Z        INFO    spectrumservice/client.go:449   Metrics received for switch 125754 : 0 .
2020-07-03T08:06:51.026Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3005 : 1 .
2020-07-03T08:06:51.070Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3025 : 1 .
2020-07-03T08:06:51.117Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3058 : 1 .
2020-07-03T08:06:51.162Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3036 : 1 .
2020-07-03T08:06:51.209Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3014 : 1 .
2020-07-03T08:06:51.255Z        INFO    spectrumservice/client.go:449   Metrics received for switch 3047 : 1 .
2020-07-03T08:06:51.282Z        INFO    spectrumservice/client.go:233   Number of Storage volumes retrieved: 1
2020-07-03T08:06:51.282Z        INFO    spectrumservice/client.go:237   Selecting storage system with regex : .*
2020-07-03T08:06:51.288Z        INFO    spectrumservice/client.go:449   Metrics received for switch 125764 : 0 .
2020-07-03T08:06:51.396Z        INFO    spectrumservice/client.go:269   Metrics received for storageID 9669 : 10 .
2020-07-03T08:06:53.451Z        INFO    spectrumservice/client.go:308   Volumes retrieved for Storage System 9669 : 515
2020-07-03T08:06:54.018Z        ERROR   spectrumservice/client.go:319   Error during volume metrics call.[{ " r e s u l t " : { " m s g I d " : " B P C U I 0 0 9 9 E " , " t e x t " : " T h e   s t o r a g e   r e s o u r c e   i s   n o t   a v a i l a b l e . " , " t i m e " : " J u l   3 ,   2 0 2 0 ,   1 0 : 0 6 : 5 4 " , " t y p e " : " E " } }]lient
github.com/topine/ibm-spectrum-exporter/spectrumservice.(*Client).collectVolumeMetrics
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/spectrumservice/client.go:319
github.com/topine/ibm-spectrum-exporter/spectrumservice.(*Client).CollectStorageMetrics
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/spectrumservice/client.go:197
github.com/topine/ibm-spectrum-exporter/spectrumservice.(*Client).CollectAndCacheMetrics.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/spectrumservice/client.go:107
2020-07-03T08:06:54.018Z        ERROR   spectrumservice/client.go:199   Error collecting volumes metrics for storage blanked-out-before-sharing-publically. [{ " r e s u l t " : { " m s g I d " : " B P C U I 0 0 9 9 E " , " t e x t " : " T h e   s t o r a g e   r e s o u r c e   i s   n o t   a v a i l a b l e . " , " t i m e " : " J u l   3 ,   2 0 2 0 ,   1 0 : 0 6 : 5 4 " , " t y p e " : " E " } }]lient
github.com/topine/ibm-spectrum-exporter/spectrumservice.(*Client).CollectStorageMetrics
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/spectrumservice/client.go:199
github.com/topine/ibm-spectrum-exporter/spectrumservice.(*Client).CollectAndCacheMetrics.func1
        /Users/apimenteldasilvatopi/github/ibm-spectrum-exporter/spectrumservice/client.go:107
2020-07-03T08:06:54.018Z        INFO    ibm-spectrum-exporter/main.go:128       Finished collecting metrics
2020-07-03T08:06:54.018Z        INFO    ibm-spectrum-exporter/main.go:89        Scheduler started with success with interval @every 5m

2020-07-03T08:06:54.018Z        INFO    collector/collector.go:103      Starting IBM Spectrum collect.
2020-07-03T08:06:54.019Z        INFO    ibm-spectrum-exporter/main.go:117       Exporter started with Success.

The error from spectrum is according to the IBM manual:

If the URL is not used correctly, you might receive the following error message:
{"result":{"type":"E","msgId":"BPCUI0099E","time":"Apr 4, 2016 16:25:07","text":
"The storage resource is not available."}}

ref: https://www.ibm.com/support/knowledgecenter/SS5R93_5.3.6/com.ibm.spectrum.sc.doc/mgr_rest_api_retrieving_cli.html