s-u / fasttime Goto Github PK
View Code? Open in Web Editor NEWFast utility for date/time parsing and conversion in R
Fast utility for date/time parsing and conversion in R
fasttime does not correctly return dates.
'''library(data.table)
library(fasttime)
dt<-data.table(Time=c("29/12/2022 12:13:51"
,"29/12/2022 12:13:52"
,"29/12/2022 12:13:53"
,"29/12/2022 12:13:54"
,"29/12/2022 12:13:56"
,"30/12/2022 6:29:11"
,"30/12/2022 6:29:12"
,"30/12/2022 6:29:13"
,"30/12/2022 6:29:14"
,"30/12/2022 6:29:15"))
fastPOSIXct(dt$Time,"GMT",required.components = 6L)
[1] "2035-06-14 12:13:51 GMT" "2035-06-14 12:13:52 GMT" "2035-06-14 12:13:53 GMT"
[4] "2035-06-14 12:13:54 GMT" "2035-06-14 12:13:56 GMT" "2036-06-13 06:29:11 GMT"
[7] "2036-06-13 06:29:12 GMT" "2036-06-13 06:29:13 GMT" "2036-06-13 06:29:14 GMT"
[10] "2036-06-13 06:29:15 GMT"
fastPOSIXct(dt$Time)
[1] "2035-06-14 15:13:51 IDT" "2035-06-14 15:13:52 IDT" "2035-06-14 15:13:53 IDT"
[4] "2035-06-14 15:13:54 IDT" "2035-06-14 15:13:56 IDT" "2036-06-13 09:29:11 IDT"
[7] "2036-06-13 09:29:12 IDT" "2036-06-13 09:29:13 IDT" "2036-06-13 09:29:14 IDT"
[10] "2036-06-13 09:29:15 IDT"
'''
Just a suggestion, I think a fastDate
function would be worthwhile. This:
fastDate <- function(x) as.Date( fasttime::fastPOSIXct( x, "UTC", 3L ) )
Is around 60x faster than both as.Date
and lubridate::as_date
, and produces identical results. Just a simple wrapper, but I'm finding myself using it a lot, so thought others might find it useful too. I'd be happy to make a pull request if you like the idea.
# test data
dates <- as.Date( seq.int( 1, to = 1E4, by = 1 ), origin = "1970-01-01" )
x <- as.character( dates )
res <- microbenchmark::microbenchmark(
fast = { fast <- fastDate( x ) },
lubr = { lubr <- lubridate::as_date( x ) },
base = { base <- as.Date( x ) },
times = 10
)
setDT(res)[ , .( median = signif( median( time ) / 1E6, 4 ) ), by = expr ]
# expr median
# 1: lubr 61.020
# 2: base 60.520
# 3: fast 1.054
identical( dates, fast ) # TRUE
identical( dates, lubr ) # TRUE
identical( dates, base ) # TRUE
BTW, fantastic package, I'm blown away every time I run a benchmark.
Hello, @s-u. Any reason the most recent version here and on r-forge has not been pushed to CRAN?
I have to process a very large (several millions rows) timeseries with a 0.1 sec interval.
After importing a file using fread() timestamps are correctly stored as strings:
"2018-08-29 12:45:00.6"
"2018-08-29 12:45:00.7"
"2018-08-29 12:45:00.8"
"2018-08-29 12:45:00.9"
"2018-08-29 12:45:01"
"2018-08-29 12:45:01.1"
"2018-08-29 12:45:01.2"
Then I tried to convert them to posixct using:
dati1$TIMESTAMP = fastPOSIXct(dati1$TIMESTAMP, tz = "UTC", required.components = 6L)
But looks like something goes wrong, the function still returns decimals of seconds but they look somehow rounded like this:
2018-08-29 12:45:00.5
2018-08-29 12:45:00.7
2018-08-29 12:45:00.7
2018-08-29 12:45:00.9
2018-08-29 12:45:01.0
2018-08-29 12:45:01.0
2018-08-29 12:45:01.2
2018-08-29 12:45:01.2
Is that a limit of fasttime or a bug?
Thanks
Hi Simon,
I'd like to suggest a small enchancement: whenever four digits have been "collected", call it a year and move on to month, regardless if a separator is present or not. Same between months and days but with two digits. Maybe a new parameter require.separators=true
makes sense, for rare cases where y,m,d are represented by more than 4,2,2 digits respectively.
Not sure this is the place for a feature request (nor if it is done) but it would be very nice to get a fastDate function that would cast Character to Date. Looking at tparse.c it might just be cuting some part off ?
fastDate('2011-01-01') or fastDate('20110304') would give [1] Date '2011-01-01'
Starting from 2039, the parsed result become incorrect
> fastPOSIXct('2039-12-31', 'GMT')
[1] "1903-11-24 17:31:44 GMT"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.