TimeSeries Query returning timestamps with rounding error [JIRA: CLIENTS-889]

[bobhall]

TimeSeries Query returning timestamps with rounding error [JIRA: CLIENTS-889]

[lukebakken]

Hello,

Please provide this information:

• Your table definition
• The data you have inserted
• Your query
• The version of Ruby you're using
• The version of Riak TS you're using
• The version of the riak-client gem you're using

Most likely, this is due to rounding in the Float type in Ruby, but I would like to be able to reproduce it.

Thank you.

[bobhall]

Table definition:

CREATE TABLE test_bob (                                                                                                                                                           
  bob_id sint64 not null,                                                                                                                                                         
  creation_time timestamp not null,                                                                                                                                               
  PRIMARY KEY ((bob_id, quantum(creation_time, 7, d)),                                                                                                                            
  bob_id,                                                                                                                                                                         
  creaton_time)                                                                                                                                                                   
)

Data I inserted:

riak-shell(7)>select * from test_bob where bob_id=11 and creation_time > 1459382400000 and creation_time < 1459468800000;
+------+-------------+
|bob_id|creation_time|
+------+-------------+
|  11  |1459443822778|
|  11  |1459444070103|
+------+-------------+

Query:

select * from test_bob where bob_id=11 and creation_time > 1459382400000 and creation_time < 1459468800000

Ruby version:
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]

Riak TS Version:
1.3

riak-client version:
2.3.2

Then I print the result of my query (using strftime) and I get:

11, 2016-03-31T10:03:42.778000Z
11, 2016-03-31T10:07:50.102999Z

... And I expect:
11, 2016-03-31T10:03:42.778000Z
11, 2016-03-31T10:07:50.103000Z

[lukebakken]

One last question, just so I'm using the same code, what strftime format are you using?

[bobhall]

"%Y-%m-%dT%H:%M:%S.%6NZ"

[lukebakken]

I'm glad I asked!

[lukebakken]

Looks like I'm going to have to make the code a bit more intelligent:

irb(main):010:0> Time.at(1459444070103 / 1000.0).strftime("%Y-%m-%dT%H:%M:%S.%6NZ")
=> "2016-03-31T10:07:50.102999Z"

Basically, this is what is done here

[lukebakken]

@bobhall I did some more research in irb. Note that using to_f on the Time object returned by Time.at does return the correct value for milliseconds, but that using the nsec and usec methods reveals the difference due to the use of a float to store the value internally.

I recently changed the Python client to not auto-convert timestamp values to datetime objects since using integer values preserves accuracy. I could add a similar feature to the Ruby client to use integers rather than converting to Time objects. What do you think?

irb(main):019:0> time = Time.at(1459444070103.to_f / 1000)
=> 2016-03-31 10:07:50 -0700
irb(main):020:0> time.to_f
=> 1459444070.103
irb(main):021:0> time.nsec
=> 102999925
irb(main):022:0> time.usec
=> 102999
irb(main):024:0> time.strftime("%Y-%m-%dT%H:%M:%S.%6NZ")
=> "2016-03-31T10:07:50.102999Z"

[lukebakken]

@bobhall the following code should resolve your issue, I'll get it in the next release:

irb(main):010:0> t = 1459444070103
=> 1459444070103
irb(main):011:0> secs = t / 1000
=> 1459444070
irb(main):012:0> millis = t % 1000
=> 103
irb(main):013:0> Time.at(secs, millis * 1000)
=> 2016-03-31 10:07:50 -0700
irb(main):014:0> time = Time.at(secs, millis * 1000)
=> 2016-03-31 10:07:50 -0700
irb(main):015:0> time.strftime("%Y-%m-%dT%H:%M:%S.%6NZ")
=> "2016-03-31T10:07:50.103000Z"

I am still interested to know if you would find it useful to have optional conversion from Riak TS timestamp to Time objects.

[bobhall]

@lukebakken , thanks for the workaround.

It would be great to have an option to have RiakTS timestamps convert to Time objects or integers. It might be cool to have that be an option on the init method of the client itself (so you don't have to pass convert_to_time=true over and over, for example). The same option would be cool to have in the Python client as well.

[lukebakken]

Yep, that's how the Python client works.

I'll implement something similar in Ruby, and, to be consistent, will make the default to be no auto conversion. That is the default in the Python client since it is used as middleware between Riak TS and other data consumers, so the conversion added a performance cost.

[bobhall]

Ah, so it does!