Data outside time range in Grafana when doing `TS.MRANGE`

[redisgrafana]

Data outside time range in Grafana when doing `TS.MRANGE`

[mikhail-vl]

@dpipemazo Thank you for confirming, I am checking with the RedisTimeSeries team what will be the expected behavior in this case.

I agree that extra intervals can be clipped in the plug-in, but I expect RedisTimeSeries to return the correct values.

[dpipemazo]

Sure, thanks for helping to bring it to their attention.

[mikhail-vl]

@dpipemazo Thank you for submitting another issue.

Have you tried the same command from redis-cli and see the kworker returning 0 for the outside interval? How about remove aggregation?

I will take a look and get back to you.

[dpipemazo]

@mikhailredis Thanks for taking a look. I should have confirmed in redis-cli, apologies for not doing that originally. I reproduced a similar situation to the graphs above through running:

ts.mrange 1598405335639 1598410333697 AGGREGATION avg 1000 WITHLABELS FILTER element=monitor type=process subtype1=cpu_user

The full resultant text file is attached and is telling -- the source of the issue is indeed coming from Redis Time Series as we can see some example entries from the query:

1484) 1) "monitor:process:99:kworker/7:1:cpu_user"
      2) 1) 1) "element"
            2) "monitor"
         2) 1) "type"
            2) "process"
         3) 1) "container"
            2) "x86_64"
         4) 1) "device"
            2) "default"
         5) 1) "language"
            2) "Python"
         6) 1) "version"
            2) "1.7.2"
         7) 1) "level"
            2) "INFO"
         8) 1) "subtype0"
            2) "99:kworker/7:1"
         9) 1) "subtype1"
            2) "cpu_user"
      3) 1) 1) (integer) 1598456570000
            2) 0

and

1482) 1) "monitor:process:9866:kworker/3:0:cpu_user"
      2) 1) 1) "element"
            2) "monitor"
         2) 1) "type"
            2) "process"
         3) 1) "container"
            2) "x86_64"
         4) 1) "device"
            2) "default"
         5) 1) "language"
            2) "Python"
         6) 1) "version"
            2) "1.7.2"
         7) 1) "level"
            2) "INFO"
         8) 1) "subtype0"
            2) "9866:kworker/3:0"
         9) 1) "subtype1"
            2) "cpu_user"
      3) 1) 1) (integer) 1598404695000
            2) 0

Where we can see for data existing on either side of the time series we are getting a 0 entry back with a timestamp outside of the original range.

I can (and will) file an issue on the Redis Time Series repo about this however I'm not sure it's as straightforward as asking them to do it. This is a debatably reasonable response on their end and I can see use cases where we'd want to know there was something that existed outside of the time range query. It's likely a flag we ask for from them and likely would take a while to get in if approved?

I do think given how this "Data outside of time range" cannot be turned off in grafana and materially impacts the usability of the graphs it would be nice to clip any time values returned from redis to (min, max) on the time series queried. That would solve this pretty easily I think without too much effort.

Full output of the mrange query attached below.

mrange.txt

[dpipemazo]

I did go ahead and file the issue on the Redis Time Series repo as well: RedisTimeSeries/RedisTimeSeries#506

[mikhail-vl]

@dpipemazo Thank you, let's wait for the RedisTimeSeries team response.

[mikhail-vl]

@dpipemazo RedisTimeSeries team accepted the issue and will work on it.

Do you mind to close this issue and follow RedisTimeSeries/RedisTimeSeries#506?