choptastic/qdate

Erlang date, time, and timezone management: formatting, conversion, and date arithmetic

qdate - Erlang Date and Timezone Library

Purpose

Erlang Date and Time management is rather primitive, but improving.

dh_date, of which ec_date in erlware_commons is a fork, is a huge step towards formatting and parsing dates in a way that compares nicely with PHP's date and strtotime functions.

Unfortunately, ec_date doesn't deal with timezones, but conveniently, the project erlang_localtime does.

It is the express purpose of this qdate package to bring together the benefits of ec_date and erlang_localtime, as well as extending the capabilities of both to provide for other needed tools found in a single module.

qdate provides date and time formatting and parsing from and into:

• Formatting Strings
• Erlang Date Format
• Erlang Now Format
• Unixtime integers
• Timezones

And all this while dealing with timezone parsing, formatting, conversion and overall management.

Acceptable Date Formats

• Erlang Date Format: {{Y,M,D},{H,M,S}}
• Erlang Now Format: {MegaSecs, Secs, MicroSecs}
• Date String: "2013-12-31 08:15pm" (including custom formats as defined with qdate:register_parser/2 - see below)
• Integer Unix Timestamp: 1388448000
• A Two-tuple, where the first element is one of the above, and the second is a timezone. (i.e. {{{2008,12,21},{23,59,45}}, "EST"} or {"2008-12-21 11:59:45pm", "EST"}). Note: While, you can specify a timezone along with unix timestamps or the Erlang now format, it won't do anything, as both of those timestamps are absolute, and imply GMT.

All while doing so by allowing you to either set a timezone by some arbitrary key or by using the current process's Pid is the key.

Further, while ec_date doesn't support PHP's timezone characters (e, I, O, P, T, Z, r, and c), qdate will handle them for us.

Exported Functions:

Conversion Functions

• to_string(FormatString, ToTimezone, Date) - "FormatString" is a string that follows PHP's date function formatting rules. The date will be converted to the specified ToTimezone.
• to_string(FormatString, Date) - same as to_string/3, but the Timezone is intelligently determined (see below)
• to_string(FormatString) - same as to_string/2. but uses the current time as Date
• to_date(ToTimezone, Date) - converts any date/time format to Erlang date format. Will first convert the date to the timezone ToTimezone.
• to_date(Date) - same as to_date/2, but the timezone is determined (see below).
• to_now(Date) - converts any date/time format to Erlang now format.
• to_unixtime(Date) - converts any date/time format to a unixtime integer

A ToTimezone value of the atom auto will automatically determine the timezone. For example, to_date(Date, auto) is exactly the same as to_date(Date)

A Note About Argument Order: In all cases, ToTimezone is optional and if omitted, will be determined as described below in "Understanding Timezone Determining and Conversion". If ToTimezone is specified, it will always be immediately left of the Disambiguate argument (if it's specified), which is always immediately left of Date argument. Date will always be the last argument to any of the conversion and formatting functions.

Understanding Timezone Determining and Conversions

There is a lot of timezone inferring going on here.

If a Date string contains timezone information (i.e. "2008-12-21 6:00pm PST"), then qdate will parse that properly, determine the specified PST timezone, and do conversions based on that timezone. Further, you can specify a timezone manually, by specifying it as as a two-tuple for Date (see "Acceptable Date formats" above).

If no timezone is specified or determinable in a Date variable, then qdate will infer the timezone in the following order.

• If specified by qdate:set_timezone(Timezone) for that process. Note, as specified below (in the "Timezone Functions" section), set_timezone/1 is a shortcut to set_timezone(self(), Timezone), meaning that set_timezone/1 only applies to that specific process. If none is specified.
• If no timezone is specified for the process, qdate looks at the qdate application variable default_timezone. default_timezone can be either a hard-specified timezone, or a {Module, Function} tuple. The tuple format should return either a timezone or the atom undefined.
• If no timezone is specified by either of the above, qdate assumes "GMT" for all dates.
• A timezone value of auto will act as if no timezone is specified.

Disambiguating Ambiguous Timezone Conversions

Sometimes, when you're converting a datetime from one timezone to another, there are potentially two different results if the conversion happens to land on in a timezone that's in the middle of a Daylight Saving conversion. For example, converting "11-Nov-2013 1:00:am" in "America/New York" to "GMT" could be both "5am" and "6am" in GMT, since "1am EST". This is a side effect of the "intelligence" of qdate - qdate would notice that 1am in New York is EST, and should be converted to "1am EST", and then do the conversion from "1am EST" to "GMT". This can lead to confusion.

Further, since qdate attempts to be "smart" about mistakenly entered timezones (ie, if you entered "2013-01-01 EDT", qdate knows that "EDT" (Eastern Daylight Time) doesn't apply to January first, so it assumes you meant "EST".

THE SOLUTION to this tangled mess that we call Daylight Saving Time is to provide an option to disambiguate if you so desire. By default disambiguation is disabled, and qdate will just guess as to it's best choice. But if you so desire, you can make sure qdate does both conversions, and returns both.

You can do this by passing a Disambiguation argument to to_string, to_date, to_unixtime, and to_now. Disambiguation can be an atom of the values:

• prefer_standard (Default Behavior): If an ambiguous result occurs, qdate will return the date in standard time rather than daylight time.
• prefer_daylight: If an ambiguous result occurs, qdate will return the preferred daylight time.
• both: If an ambiguous result occurs, qdate will return the tuple: {ambiguous, DateStandard, DateDaylight}, where DateStandard is the date in Standard Time, and DateDaylight is the date in Daylight Saving Time.

So the expanded conversions functions are:

• to_date(ToTimezone, Disambiguate, Date)
• to_string(FormatString, ToTimezone, Disambiguate, Date)
• to_unixtime(Disambiguate, Date)
• to_now(Disambiguate, Date)

Examples:

1> qdate:set_timezone("GMT").
ok

%% Here, converting GMT 2013-11-03 6AM to America/New York yields an ambiguous
%% result
2> qdate:to_date("America/New York", both, {{2013,11,3},{6,0,0}}).
{ambiguous,{{2013,11,3},{1,0,0}},{{2013,11,3},{2,0,0}}}

%% Let's just use daylight time
3> qdate:to_date("America/New York", prefer_daylight, {{2013,11,3},{6,0,0}}).
{{2013,11,3},{2,0,0}}

%% Let's just use standard time (the default behavior)
4> qdate:to_date("America/New York", prefer_standard, {{2013,11,3},{6,0,0}}).
{{2013,11,3},{1,0,0}}

5> qdate:set_timezone("America/New York").
ok

%% Switching from 1AM Eastern Time to GMT yields a potentially ambiguous result
6> qdate:to_date("GMT", both, {{2013,11,3},{1,0,0}}).
{ambiguous,{{2013,11,3},{6,0,0}},{{2013,11,3},{5,0,0}}}

%% Use daylight time for conversion
7> qdate:to_date("GMT", prefer_daylight, {{2013,11,3},{1,0,0}}).
{{2013,11,3},{5,0,0}}

%% Here we demonstrated that even if we ask for "both", if there is no
%% ambiguity, the plain date is returned
8> qdate:to_date("GMT", both, {{2013,11,3},{5,0,0}}).
{{2013,11,3},{10,0,0}}

Conversion Functions provided for API compatibility with ec_date

• parse/1 - Same as to_date(Date)
• nparse/1 - Same as to_now(Date)
• format/1 - Same as to_string/1
• format/2 - Same as to_string/2

Date and Time Comparison

qdate provides a few convenience functions for performing date comparisons.

• compare(A, B) -> -1|0|1 - Like C's strcmp, returns:
  • 0: A and B are exactly the same.
  • -1: A is less than (before) B.
  • 1: A is greater than (after) B.
• compare(A, Operator, B) -> true|false - Operator is an infix comparison operator, and the function will return a boolean. Will return true if:
  • '=', or '==' - A is the same time as B
  • '/=', or '=/=' or '!=' - A is not the same time as B
  • '<' - A is before B
  • '>' - A is after B
  • '=<' or '<=' - A is before or equal to B
  • '>=' or '=>' - A is after or equal to B
• between(A, Date, B) -> true|false - The provided Date is (inclusively) between A and B. That is, A =< Date =< B.
• between(A, B) -> true|false - shortcut for between(A, now(), B)
• between(A, Op1, Date, Op2, B) -> true|false - the fully verbose option of comparing between. Op1 and Op2 are custom operators. For example, if you wanted to do an exclusive between, you can do: between(A, '<', Date, '<', B)

Note 1: Operator must be an atom.

Note 2: These functions will properly compare times with different timezones (for example: compare("12am CST",'==',"1am EST") will properly return true)

Sorting

qdate also provides a convenience functions for sorting lists of dates/times:

• sort(List) - Sort the list in ascending order of earliest to latest.
• sort(Op, List) - Sort the list where Op is one of the following:
  • '<' or '=<' or '<=' - Sort ascending
  • '>' or '>=' or '=>' - Sort descending
• sort(Op, List, Opts) - Sort the list according to the Op, with options provided in Opts. Opts is a proplist of the following options:
  • {non_dates, NonDates} - Tells it how to handle non-dates. NonDates can be any of the following:
    • back (default) - put any non-dates at the end (the back) of the list
    • front - put any non-dates at the beginning of the list
    • crash - if there are any non-dates, crash.

Example:

	 1> Dates = ["non date string", <<"garbage">>,
		1466200861, "2011-01-01", "7pm",
		{{1999,6,21},{5,30,0}}, non_date_atom, {some_tuple,123}].
	 2> qdate:sort('>=', Dates, [{non_dates, front}]).
     [<<"garbage">>,"non date string",
	  {some_tuple,123},
	  non_date_atom,1466200861,"2011-01-01",
	  {{1999,6,21},{5,30,0}},
	  "7pm"]

Note 1: This sorting is optimized to be much faster than using a home-grown sort using the compare functions, as this normalizes the items in the list before comparing (so it's only really comparing integers, which is quite fast).

Note 2: This is one of the few qdate functions that don't have the "Date" as the last argument. This follows the pattern in Erlang/OTP to put options as the last argument (for example, re:run/3)

Note 3: You'll notice that qdate's sorting retains the original terms (in the example above, we compared a datetime tuple, unix timestamp, and two strings (along with a number of non-dates, which were just prepended to the front of the list).

Timezone Functions

• set_timezone(Key, TZ) - Set the timezone to TZ for the key Key
• set_timezone(TZ) - Sets the timezone, and uses the Pid from self() as the Key. Also links the process for removal from the record when the Pid dies.
• get_timezone(Key) - Gets the timezone assigned to Key
• get_timezone() - Gets the timezone using self() as the Key
• clear_timezone(Key) - Removes the timezone record associated with Key.
• clear_timezone() - Removes the timezone record using self() as Key. This function is not necessary for cleanup, most of the time, since if Key is a Pid, the qdate server will automatically clean up when the Pid dies.

Note: If no timezone is set, then anything relying on the timezone will default to GMT.

Registering Custom Parsers and Formatters

You can register custom parsers and formatters with the qdate server. This allows you to specify application-wide aliases for certain common formatting strings in your application, or to register custom parsing engines which will be attempted before engaging the ec_date parser.

Registering and Deregistering Parsers

• register_parser(Key, ParseFun) - Registers a parsing function with the qdate server. ParseFun is expected to have the arity of 1, and is expected to return a DateTime format ({{Year,Month,Day},{Hour,Min,Sec}}) or, if your ParseFun is capable of parsing out a Timezone, the return the tuple {DateTime, Timezone}. Keep in mind, if your string already ends with a Timezone, the parser will almost certainly extract the timezone before it gets to your custom ParseFun. If your custom parser is not able to parse the string, then it should return undefined.
• deregister_parser(Key) - If you previously registered a parser with the qdate server, you can deregister it by its Key.
• get_parsers() - Get the list of all registered parsers and their keys.

Registering and Deregistering Formatters

• register_format(Key, FormatString) - Register a formatting string with the qdate server, which can then be used in place of the typical formatting string.
• deregister_format(Key) - Deregister the formatting string from the qdate server.
• get_formats() - Get the list of all registered formats and their keys.

About backwards compatibility with ec_date and deterministic parsing

ec_date and dh_date both have a quirk that bothers me with respect to the parsing of dates that causes some date parsing to be non-deterministic. That is, if parsing an incomplete date or time (ie, a text string that is missing a time or a date), ec_date will automatically insert the current values of those as read by the system clock.

For example, if the following lines are run a few seconds apart:

1> ec_date:parse("2012-02-04").
{{2012,2,4},{0,1,10}}
2> ec_date:parse("2012-02-04").
{{2012,2,4},{0,1,12}}
3> ec_date:parse("2012-02-04").
{{2012,2,4},{0,1,13}}

As you can see, even though the inputs are the same each time, the resulting parsed dates have the current time inferred. The same behavior can be observed if parsing a time without a date:

4> ec_date:parse("7pm").
{{2013,4,30},{19,0,0}}

As you can see, even though the time did not specify a date, the resulting parsed datetime has the date inferred from the current date. Admittedly, inferring the date bothers me less than inferring the time, but in the name of consistency, there should be options for enabling or disabling both.

The Solution For Non-deterministic parsing

To solve this issue for users that are bothered by this, while preserving backwards compatibility for folks who prefer this, we're going to introduce a qdate application environment variable called deterministic_parsing.

The value of deterministic_parsing can be a tuple of the following format:

{DatePref, TimePref}

Where DatePref and TimePref are either of the following atoms:

• now - Automatically fill in the missing date or time components with the current time (the is the behavior described above)
• zero - Fill in the missing date or time components with zeroed out values. This means that if a date is missing, it'll be set to the unix epoch ({1970,1,1}) and if a time is missing, it'll be set to midnight: {0,0,0}.

So, the acceptable combinations can be the following:

• {zero, zero} - Any missing components will be replaced with zero-values. (This is the qdate default behavior)
• {now, zero} - If a date is missing, insert the current date, but if a time is missing, set it to midnight.
• {zero, now} - If a date is missing, set it to the unix epoch, and if a time is missing, set it to the current time of day.
• {now, now} - If either date or time are missing, set it to the current date or current time.

Note: If this application value is not set, the default behavior for qdate is to avoid non-determinism and use {zero, zero}.

To set this value, you can either set the value manually in code with:

application:set_env(qdate, deterministic_parsing, {now, zero}).

or (and this is the preferred method) use a config file and load it with

erl -config path/to/file.config

Sample config file specifying this application variable:

[{qdate, [
    {deterministic_parsing, {now, zero}}
]}].

Demonstration

Basic Conversion and Formatting

%% Let's start by making a standard Erlang DateTime tuple
1> Date = {{2013,12,21},{12,24,21}}.
{{2013,12,21},{12,24,21}}

%% Let's do a simple formatting of the date
2> DateString = qdate:to_string("Y-m-d h:ia", Date).
"2013-12-21 12:24pm"

%% We can also specify the format string as a binary
3> DateBinary = qdate:to_string(<<"Y-m-d h:ia">>,Date).
<<"2013-12-21 12:24pm">>

%% And we can parse the original string to get back a DateTime object
4> qdate:to_date(DateString).
{{2013,12,21},{12,24,0}}


%% We can do the same with a binary
5> qdate:to_date(DateBinary).
{{2013,12,21},{12,24,0}}

%% We can also parse that date and get a Unix timestamp
6> DateUnix = qdate:to_unixtime(DateString).
1387628640

%% And we can take that Unix timestamp and format it to a string
7> qdate:to_string("n/j/Y g:ia", DateUnix).
"12/21/2013 12:24pm"

%% We can take a date string and get an Erlang now() tuple
8> DateNow = qdate:to_now(DateString).
{1387,628640,0}

%% And we can convert it back

9> DateString2 = qdate:to_string("n/j/Y g:ia", DateNow).
"12/21/2013 12:24pm"

Note: That by this point, we've used, as the Date parameter, all natively supported date formats: Erlang datetime(), Erlang now(), Unix timestamp, and formatted text strings either as a list or as a binary.

For the most part, this will be the bread and butter usage of qdate. Easily converting from one format to another without having to worry about what format your data is currently in. qdate will figure it out for you.

But now, we're going to start getting interesting!

Registering Custom Parsers

%% Let's format our date into something shorter. This may, for example, be a
%% date format you may deal with when receiving a data-set from a client.
10> CompactDate = qdate:to_string("Ymd", DateNow).
"20131221"

%% Let's try to parse it
11> qdate:to_date(CompactDate).
** exception throw: {ec_date,{bad_date,"20131221"}}
     in function  ec_date:do_parse/3 (src/ec_date.erl, line 92)
     in call from qdate:to_date/2 (src/qdate.erl, line 169)

%% Well obviously, this isn't a standard format by any means, so it crashes.
%% You can parse it yourself before passing it to `qdate` or if you deal with
%% this format frequently enough, you can register it as a custom parser and
%% qdate will intelligently parse it if it can.

%% So let's make a simple parser for it that uses regular expressions:
12> ParseCompressedDate =
12>  fun(RawDate) when length(RawDate)==8 ->
12>       try re:run(RawDate,"^(\\d{4})(\\d{2})(\\d{2})$",[{capture,all_but_first,list}]) of
12>         nomatch -> undefined;
12>         {match, [Y,M,D]} ->
12>           ParsedDate = {list_to_integer(Y), list_to_integer(M), list_to_integer(D)},
12>           case calendar:valid_date(ParsedDate) of
12>              true -> {ParsedDate, {0,0,0}};
12>              false -> undefined
12>           end
12>       catch _:_ -> undefined
12>       end;
12>     (_) -> undefined
12>  end.
#Fun<erl_eval.6.82930912>

%% And now we'll register the parser with the `qdate` server, giving it a "Key"
%% of the atom 'compressed_date'
13> qdate:register_parser(compressed_date,ParseCompressedDate).
compressed_date

%% Now, let's try parsing that again
14> qdate:to_date(CompactDate).
{{2013,12,21},{0,0,0}}

%% Huzzah! It worked. From here on out, `qdate`, will properly parse that kind
%% of data if that format is passed, otherwise, it will merely skip over that
%% parser and engage the standard parser in `ec_date`

Note: Currently, qdate expects custom parsers to not crash. If a custom parser crashes, an exception will be thrown. This is done in order to help you debug your parsers. If a parser receives an unexpected input and crashes, the exception will be generated and you will be able to track down what input caused the crash.

Another Note: Custom parsers are expected to return either:

• A datetime() tuple. (ie {{2012,12,21},{14,45,23}}).
• An integer, which represents the Unix timestamp.
• The atom undefined if this parser is not a match for the supplied value

Included Parser: Relative Times

qdate ships with an optional relative time parser. To speed up performance (since this parser uses regular expressions), this parser is disabled by default. But if you wish to use it, make sure you call qdate:register_parser(parse_relative, fun qdate:parse_relative/1).

Doing this allows you to parse relative time strings of the following formats:

• "1 hour ago"
• "-15 minutes"
• "in 45 days"
• "+2 years"

And doing so allows you to construct slightly more readable comparison calls for sometimes common comparisons. For example, the following two calls are identical:

qdate:between("-15 minutes", Date, "+15 minutes").

qdate:between(qdate:add_minutes(-15), Date, qdate:add_minutes(15)).

Registering Custom Formats

%% Let's format a date to a rather long string
15> qdate:to_string("l, F jS, Y g:i A T",DateString).
"Saturday, December 12st, 2013 12:24 PM GMT"

%% Boy, that sure was a long string, I hope you can remember all those
%% characters in that order!

%% But, you don't have to: if that's a common format you use in your
%% application, you can register your format with the `qdate` server, and then
%% easily refer to that format by its key.

%% So let's take that format and register it
16> qdate:register_format(longdate, "l, F jS, Y g:i A T").
ok

%% Now, let's try to format our string 
17> LongDateString = qdate:to_string(longdate, DateString).
"Saturday, December 21st, 2013 12:24 PM GMT"

%% It was certainly easier to remember the atom 'longdate' than trying to
%% remember the seemingly random "l, F jS, Y g:i A T".

Ain't it nice, making things easier for you?

Timezone Demonstrations

The observant reader would have noticed something else. We used timezones in the last couple of calls. Indeed, not only can qdate deal with formatting timezones, but it can also parse them, convert them, and set them for simplified conversions.

Let's see how we do this

%% Let's take that last long date string (that was in GMT) and move it to
%% Pacific time
18> LongDatePDT = qdate:to_string(longdate, "PDT", LongDateString).
"Saturday, December 21st, 2013 4:24 AM PST"

%% See something interesting there? Yeah, we told it it was PDT, but it output
%% PST.  That's because PST is not in daylight saving time in December, and 
%% `qdate` was able to intelligently infer that, and fix it for us.

%% Note, that when in doubt, `qdate` will *not* convert. For example, not all
%% places in Eastern Standard Time do daylight saving time, and as such, EST
%% will not necessarily convert to EDT.

%% However, if you provide the timezone as something like "America/New York",
%% it *will* figure that out, and do the correct conversion for you. 

%% Let's see how it handles unix times with strings that contain timezones.
%% If you recall, LongDateString = "Saturday, December 21st, 2013 12:24 PM GMT"
%% and LongDatePDT = "Saturday, December 21st, 2013 4:24 AM PST"
19> qdate:to_unixtime(LongDateString).
1387628640

%% Now let's try it with the Pacific Time one
20> qdate:to_unixtime(LongDatePDT).
1387628640

%% How exciting! `qdate` properly returned the same unix timestamp, since unix
%% timestamps are timezone neutral. That is because unix timestamps are the
%% number of seconds since midnight on 1970-01-01 GMT. As such, unix timestamps
%% should not change, just because you're in a different timezone.

%% Let's set the timezone for the current process to EST to test that previous
%% assertion
21> qdate:set_timezone("EST").
ok

%% Now let's try converting those dates to unixtimes again
22> qdate:to_unixtime(LongDateString).
1387628640
23> qdate:to_unixtime(LongDatePDT).
1387628640

%% Great! They didn't change, as we expected. The unix timestamps have remained
%% Timezone neutral.

%% Let's clear the current process's timezone (which basically means setting it
%% to the application variable `default_timezone`, or, in this case, just
%% resetting it to "GMT"
24> qdate:clear_timezone().
ok

%% Now, let's imagine you run a website. The main site may have its own
%% timezone, and the users each also have their own timezones.  So we'll
%% register timezones for each the main site, and each user. That way, if we
%% need to ensure that a date is presented in an appropriate timezone.


%% Let's register some timezones by "Timezone Keys".  
25> qdate:set_timezone(my_site, "America/Chicago").
ok
26> qdate:set_timezone({user,1},"Australia/Melbourne").
ok

%% So we'll get the date object of the previously set unix timestamp `DateUnix`
27> qdate:to_date(DateUnix).
{{2013,12,21},{12,24,0}}

%% And let's format it, also showing the timezone offset that was used
28> qdate:to_string("Y-m-d H:i P", DateUnix).
"2013-12-21 12:24 +00:00"

%% Since we cleared the timezone for the current process, it just used "GMT"

%% Let's get the date again, but this time, use to the Timezone key `my_site`
29> qdate:to_date(my_site, DateUnix).
{{2013,12,21},{6,24,0}}

%% And let's format it to show again the timezone offset
30> qdate:to_string("Y-m-d H:i P", my_site, DateUnix).
"2013-12-21 06:24 -06:00"

%% Finally, let's get the date using the User's timezone key
31> qdate:to_date({user,1}, DateUnix).
{{2013,12,21},{23,24,0}}

%% And again, formatted to show the timezone offset
32> UserDateWithHourOffset = qdate:to_string("Y-m-d H:i P", {user,1}, DateUnix).
"2013-12-21 23:24 +11:00"

%% And finally, let's just test some more parsing and converting. Here, despite
%% the fact that the timezone is presented as "+11:00", `qdate` is able to
%% do the proper conversion, and give us back the same unix timestamp that was
%% used.
33> qdate:to_unixtime(UserDateWithHourOffset).
1387628640

One last bit of magic that may confuse you without an explanation

Magic is usually bad, you know what's worse? Timezones and Daylight Saving Time. So we use a little magic to try and simplify them for us. Below is the extent of the confusion with related to inferring timezones and formatting dates

%% First, let's set the timezone to something arbitrary
34> qdate:set_timezone("EST").
ok

%% Let's convert this date to basically the same time format, just without the
%% timezone identifier.
35> qdate:to_string("Y-m-d H:i","2012-12-21 00:00 PST").
"2012-12-21 03:00"

%% WHAT?! We entered a date and time, and out came a different time?!
%% I CALL SHENANIGANS!

%% Let's add that timezone indicator back in with the conversion to see what
%% happened:

36> qdate:to_string("Y-m-d H:i T","2012-12-21 00:00 PST").
"2012-12-21 03:00 EST"

%% OOOOOOOHHH! I see!
%% Because we set our current timezone to EST, it took the original provided
%% date in PST, and converted it to EST (since EST is the timezone we've chosen
%% for the current process). So it's taking whatever date, and if it can
%% determine a timezone, it'll extract that timezone, and convert the time from
%% that timezone to our intended timezone.

Beginning or Ending of time periods (hours, days, years, weeks, etc)

qdate can determine beginnings and endings of time periods, like "beginning of the month"

This is abstracted to beginning_X functions, which return a date/time format with the dates and times truncated to the specified level.

• beginning_minute(Date)
• beginning_hour(Date)
• beginning_day(Date)
• beginning_month(Date)
• beginning_year(Date)

There are also 0-arity versions of the above, in which Date is assumed to be "right now". For example, calling qdate:beginning_month() would return midnight on the first day of the current month.

Beginning of Week

qdate can also do a special "beginning" case, particularly the "beginning of the week" calculation. This has three forms, specifically:

• beginning_week() - Returns first day of the current week.

• beginning_week(Date) - Assumes the beginning of the week is Monday (chosen because Erlang's calendar:day_of_the_week uses 1=Monday and 7=Sunday).

• beginning_week(DayOfWeek, Date) - Calculates the beginning of the week based on the provided DayOfWeek. Valid values for DayOfWeek are the integers 1-7 or the atom versions of the days of the week. Specifically:

  • Monday: 1 | monday | mon
  • Tuesday: 2 | tuesday | tue
  • Wednesday: 3 | wednesday | wed
  • Thursday: 4 | thursday | thu
  • Friday: 5 | friday | fri
  • Saturday: 6 | saturday | sat
  • Sunday: 7 | sunday | sun

These all return 12am on the day that is the first day of the week of the provided date.

(My apologies to non-English speakers. I'm a lazy American who only speaks English, hence the Anglocentric day names).

End of time period

There are also the related end_X functions available, using the same conventions, except return the last second of that time period.

So end_month("2016-01-05") will return the unix timestamp representing "2016-01-31 11:59:59pm"

Date Arithmetic

The current implementation of qdate's date arithmetic returns Unixtimes.

There are 8 main functions for date arithmetic:

• add_seconds(Seconds, Date)
• add_minutes(Minutes, Date)
• add_hours(Hours, Date)
• add_days(Days, Date)
• add_weeks(Weeks, Date)
• add_months(Months, Date)
• add_years(Years, Date)
• add_date(DateToAdd, Date) - DateToAdd is a shortcut way of adding numerous options. For example. qdate:add_date({{1, 2, -3}, {-500, 20, 0}}) will add 1 year, add 2 months, subtract 3 days, subtract 500 hours, add 20 minutes, and not make any changes to seconds.

For the date arithmetic functions, Date, like all qdate functions, can be any format.

Date Arithmetic from "now"

There are 7 other arithmetic functions that take a single argument, and these do arithmetic from "now." For example, add_years(4) is a shortcut for add_years(4, os:timestamp()).

• add_seconds(Seconds)
• add_minutes(Minutes)
• add_hours(Hours)
• add_days(Days)
• add_weeks(Weeks)
• add_months(Months)
• add_years(Years)

Date and Time Ranges

qdate provides a number of range functions that give applicable dates/times within a start and end time. For example, "All days from 2015-01-01 to today", "every 3rd month from 2000-01-01 to 2009-12-31", or "every 15 minutes from midnight to 11:59pm on 2015-04-15".

The functions are as follows:

• range_seconds(Interval, Start, End)
• range_minutes(Interval, Start, End)
• range_hours(Interval, Start, End)
• range_days(Interval, Start, End)
• range_weeks(Interval, Start, End)
• range_months(Interval, Start, End)
• range_years(Interval, Start, End)

Where Interval is the number of seconds/days/years/etc.

So for example:

%% Get every 15th minute from "2015-04-15 12:00am to 2015-04-15 11:59am"
> qdate:range_minutes(15, "2015-04-15 12:00am", "2015-04-15 11:59am").
[1429056000,1429056900,1429057800,1429058700,1429059600,
 1429060500,1429061400,1429062300,1429063200,1429064100,
 1429065000,1429065900,1429066800,1429067700,1429068600,
 1429069500,1429070400,1429071300,1429072200,1429073100,
 1429074000,1429074900,1429075800,1429076700,1429077600,
 1429078500,1429079400,1429080300,1429081200|...]

%% Get every day of April, 2014
> qdate:range_days(1, "2014-04-01", "2014-04-30").
[1396310400,1396396800,1396483200,1396569600,1396656000,
 1396742400,1396828800,1396915200,1397001600,1397088000,
 1397174400,1397260800,1397347200,1397433600,1397520000,
 1397606400,1397692800,1397779200,1397865600,1397952000,
 1398038400,1398124800,1398211200,1398297600,1398384000,
 1398470400,1398556800,1398643200,1398729600|...]

Note, that the return value (just like qdate's arithmetic functions) is a list of integers. These integers are unix timestamps and can be easily formatted with qdate:

> Mins = qdate:range_minutes(15, "2015-04-15 12:00am", "2015-04-15 11:59am"),
> [qdate:to_string("Y-m-d h:ia", M) || M <- Mins].
["2015-04-15 00:00am","2015-04-15 00:15am",
 "2015-04-15 00:30am","2015-04-15 00:45am",
 "2015-04-15 01:00am","2015-04-15 01:15am",
 "2015-04-15 01:30am","2015-04-15 01:45am",
 "2015-04-15 02:00am","2015-04-15 02:15am",
 "2015-04-15 02:30am","2015-04-15 02:45am",
 "2015-04-15 03:00am","2015-04-15 03:15am",
 "2015-04-15 03:30am","2015-04-15 03:45am",
 "2015-04-15 04:00am","2015-04-15 04:15am",
 "2015-04-15 04:30am","2015-04-15 04:45am",
 "2015-04-15 05:00am","2015-04-15 05:15am",
 "2015-04-15 05:30am","2015-04-15 05:45am",
 "2015-04-15 06:00am","2015-04-15 06:15am",
 "2015-04-15 06:30am","2015-04-15 06:45am",
 [...]|...]

Also note that the range functions are inclusive.

Age Comparison

There are two main comparisons right now, age in years, and age in days.

• age(Date) - Number of years since Date
• age(FromDate, ToDate) - Number of years between FromDate to ToDate
• age_days(Date) - Number of full days since Date (for example from 2pm yesterday to 1:59pm today is still 0.
• age_days(FromDate, ToDate) - Number of full days between FromDate and ToDate.

Configuration Sample

There is a sample configuration file can be found in the root of the qdate directory. Or you can just look at it here.

Thanks

A few shoutouts to Dale Harvey and the Erlware team for dh_date/ec_date, and to Dmitry Melnikov for the erlang_localtime package. Without the hard work of all involved in those projects, qdate would not exist.

Thanks to Additional Contributors

• Mark Allen
• Christopher Phillips
• Nicholas Lundgaard
• Alejandro Ramallo
• Heinz Gies

Changelog

See CHANGELOG.markdown

TODO

• Make qdate backend-agnostic (allow specifying either ec_date or dh_date as the backend)
• Add -spec and -type info for dialyzer
• Research the viability of ezic for a timezone backend replacement for erlang_localtime.

Conclusion

I hope you find qdate helpful in all your endeavors and it helps make your wildest dreams come true!

If you have any bugs, feature requests, or whatnot, feel free to post a Github issue, ping me on Twitter, or email me below.

I'm open to pull requests. Feel free to get your hands dirty!

Author: Jesse Gumm

Email: [email protected]

Twitter: @jessegumm

Released under the MIT License (see LICENSE file)

qdate's Issues

Wishlist: Replace erlang_localtime

erlang_localtime does not seem to read the tzdata files, which are a more authoritative source than the ad-hoc version in erlang_localtime. Also, erlang_localtime seems to have lots of GPL code in it, which might be nice to avoid for some users.

beginning_minute seems wrong.

beginning_minute(Date) ->
{{Y,M,D},{H,M,_}} = to_date(Date),
{{Y,M,D},{H,M,0}}.

Based on the code first line will match only in case Minute is equal with Month. Otherwise will fail...

M is used for both

Silviu

Add Business Day arithmetic

Per the recommendation from @RomanShestakov, add business day arithmetic.

This will require either accepting a list of business days, or more likely, have a predicate function passed to it which would evaluate whether or not a date is a business day.

This would work with the Modifier in #1 by adding three new date units: business_day and business_week, business_month

• business_day would check each day, only decrementing the counter if the day is actually a business day.
• business_week will increment like a normal week, and then continue incrementing each day until it finds the first business day. Example: Assume today is monday, and next monday is a holiday: 1 business week would be next tuesday, while 2 business weeks would just be two mondays from now.
• business_month will work exactly like business_week, except it increments the month.

In short: business_week and business_month would do valid-business-day checks only at the end, while business_day does one for each day.

Crashes when timezone is set to "UTC" vs "GMT"

If I set the timezone to "UTC", certain date format strings cause crashes:

1> qdate:set_timezone("UTC").
2> qdate:to_string("d/M/Y:H:i:s T").
"21/Feb/2018:16:36:23 UTC"
3> qdate:to_string("d/M/Y:H:i:s Z").
** exception error: no match of right hand side value 0
     in function  qdate:to_string_worker/4 (src/qdate.erl, line 217)
     in call from qdate:to_string_worker/4 (src/qdate.erl, line 231)
4> qdate:to_string("d/M/Y:H:i:s O").
** exception error: no function clause matching qdate:format_shift(0,[]) (src/qdate.erl, line 245)
     in function  qdate:to_string_worker/4 (src/qdate.erl, line 212)
     in call from qdate:to_string_worker/4 (src/qdate.erl, line 231)

Setting the timezone to "GMT" works the way I'd expect it to:

1> qdate:set_timezone("GMT").       
ok
2> qdate:to_string("d/M/Y:H:i:s Z").
"21/Feb/2018:16:13:04 +0"
3> qdate:to_string("d/M/Y:H:i:s O").
"21/Feb/2018:16:13:06 +0000"

This seems to be caused by the first clause of localtime:tz_shift which is hardcoded to return 0 for UTC instead of the {Sign, Hours, Minutes} format that it normally returns.

This seems like a bug, so I checked the upstream version of erlang_localtime, and it was indeed fixed a couple years ago in this pull request: dmitryme/erlang_localtime#32

Any reason not to apply the same fix to qdate_localtime? I can open a pull request if you'd like.

Stop custom parser log messages

I've defined a custom date parser which works fine but generates a very large number of console log messages of the form

Trying Parser: iso8601

Is there any way to turn this logging off ?

Thx!

Add date arithmetic

Add Single-line date-arithmetic and formatting. Perhaps just as a 3rd or 4th argument to the various conversion functions.

For example: qdate:to_string(Modifier, Format, TZ, Date) This seems messy, since qdate:to_string(Modifer, Format, Date) would be ambiguous with qdate:to_string(Format, TZ, Date).

Alternatively, it could be a simple qdate:math(Modifer, Date), which would return perhaps whatever the original format was (integer() -> integer(), datetime() -> datetime()), with the exception being string() -> datetime().

Modifier would be a proplist that gets evaluated in order:

[
   {day, 5},
   {hour, -1},
]

Add 5 days, then subtract an hour.

This could, given Daylight Saving Time, be different if the hour was first decremented, then he days were added.

Generally, Hours, Minutes, and Seconds will be added based on their conversion to seconds and added to the unixtime, whereas Dates, Months, and years will be added to the erlang date 3-tuple.

Add date/time comparison

Also, a function called qdate:compare(Date1,Date2), which will do automatic conversion and comparison, returning perhaps -1, 0, or 1.

Or qdate:compare(Date1,Operator,Date2), which will return boolean, with Operator being the atoms <, =, or >.

For example: qdate:compare("2012-03-05",'<',565424567233)

Best way to get the raw timezone

Hi there,

What would be the best approach to add a function that returns the timezone for a given date (string or otherwise)? I need:

"2001-11-12T10:05:01Z" --> "GMT"
"2001-11-12T10:05:01EST" --> "EST"

Ultimately it is about parsing whatever date and timezone into its components and timezone so that it can be maintained. I don't want the implicit conversion of the timezone when using to_date/1. The plan was to use this bit of code to do that:

to_date(ToTZ, Disambiguate, RawDate)  ->
    {ExtractedDate, ExtractedTZ} = extract_timezone(RawDate),
    {RawDate3, FromTZ} = case try_registered_parsers(RawDate) of
        undefined ->
            {ExtractedDate, ExtractedTZ};
        {ParsedDate,undefined} ->
            {ParsedDate,ExtractedTZ};
        {ParsedDate,ParsedTZ} ->
            {ParsedDate,ParsedTZ}
    end,    

Thanks,

Rudolph

Module name conflict between erlang_localtime (erlware_commons) and qdate_localtime

Hi,

I noticed you have a new qdate_localtime package, taken from erlang_localtime. Not sure if this is a bug or a feature but erlware_commons is used in a lot of places; if you now use qdate with it (within a release) it will generate a module naming conflict - I had to use a qdate commit which predates the creation of qdate_localtime.

Maybe use erlang_localtime as a dependency, or do some module renaming ?

Just my 2c

Thx!

add_months(-12) being executed on 2018-01-05 throws function_clause

look the weird behavior:

3> qdate:add_months(-11).
1486320803
4> qdate:add_months(-14).
1478372009
5> qdate:add_months(-13).
1480964011
6> qdate:add_months(-12).
** exception error: no function clause matching calendar:last_day_of_the_month1(2016,13) (calendar.erl, line 244)
     in function  qdate:add_months/2 (/***/_build/default/lib/qdate/src/qdate.erl, line 625)
7> qdate:add_months(-11).
1486320816
8> qdate:add_months(-10).
1488740150
9> os:timestamp().
{1515,178569,325880}

Dep conflict with amqp-client / rabbitmq-common

Hi,

This isn't really your problem but just FYI I have a long standing project which has been happily using qdate and amqp_client, suddenly giving me the following

==> Errors generating release
Duplicated modules:
ec_semver_parser specified in erlware_commons and rabbit_common
ec_semver specified in erlware_commons and rabbit_common

Looks like sometime around 12th Oct 2016 rabbitmq-common introduced some changes including a naming conflict with erlware_commons, as used by qdate. I have posted an issue asking them maybe to change things

rabbitmq/rabbitmq-common#139

But am just letting you know in case you want to change anything.

I guess I should really be notifying all projects using erlware_commons! But qdate seems like an obvious place to start given it's widely used (by me at least)

Best wishes -

Compilation fails due to deprecated erlware commons

The compilation fails on erl 20+ due to some depreciations in erlware_commons which I fixed in my fork: https://github.com/Spoowy/erlware_commons

Cheers,
Erwin

Server is not required

The gen_server with which everything is getting registered is not a necessity. Instead, registering Timezones to self() should just use the process dictionary, and custom formatters, parsers, and timezones can just be loaded into and read-from the application env_vars.

This eliminates a potential for bottlenecks around a gen_server.

Further, this means users who wish to use a more purely functional method (passing TZ around, rather than relying on some kind of state), will no longer need to start a process they won't be using anyway.

UTC unixtime error

I was using this library to convert a time to unixtime.

qdate:set_timezone("UTC")
qdate:to_unixtime("2018-3-25T2:32:00")

now returns an error instead of 1521945120 .

I believe this is caused by a change in erlang_localtime after bd0083ffe6107537a555133c9b94656ca5c854f3 .

iex(1)> :qdate.set_timezone("UTC")
:ok
iex(2)> :qdate.to_unixtime("2018-3-25T2:32:00")
** (FunctionClauseError) no function clause matching in :calendar.datetime_to_gregorian_seconds/1
(stdlib) calendar.erl:137: :calendar.datetime_to_gregorian_seconds(:time_not_exists)
src/qdate.erl:322: :qdate.to_unixtime/2

Add compare_date/2,3 for comparing only the date or time portion of a provided date/time

Perhaps the introduction of a function compare_date/2,3 and/or compare_time/2/3.

To demonstrate the thought:

A = "2013-09-01 10am CST",
B = "2013-09-01 12pm CST",
C = "2013-09-02 12am CST",
D = "2013-09-02 12am EST",

qdate:compare_date(A,B).
%% returns 0

qdate:compare_date(A,C).
%% returns -1

qdate:compare_date(C,D).
%% returns ??. 

That last case illustrates the major issue with date comparison.

If C is converted to EST, then it's 2013-09-02 1am, where it would be the same day, whereas if D is converted to CST, then it's 2013-09-01 11pm, in which case it would be different days.

Perhaps the solution is to accept a "Reference Timezone" which both dates would be converted to, or if none is provided to use the timezone stored with set_timezone (which would fall back to GMT).

I'm not entirely sure this is a feasible proposition, and might add too much "magic".

I'd say we should just extract the date portion without even considering the timezone, but what happens when comparing against a unix timestamp or now tuple? Then we would still need to do something with implicit timezones anyway.

hex depedency issue

The build is breaking for dependency

===> Verifying dependencies...
===> Fetching erlware_commons ({pkg,<<"erlware_commons">>,<<"1.0.0">>})
===> Downloaded package, caching at /Users/maru/.cache/rebar3/hex/default/packages/erlware_commons-1.0.0.tar
===> Fetching qdate_localtime ({pkg,<<"qdate_localtime">>,<<"1.1.0">>})
===> Downloaded package, caching at /Users/maru/.cache/rebar3/hex/default/packages/qdate_localtime-1.1.0.tar
===> Fetching cf ({pkg,<<"cf">>,<<"0.2.2">>,
                              <<"7F2913FFF90ABCABD0F489896CFEB0B0674F6C8DF6C10B17A83175448029896C">>})
===> Failed to fetch and copy dep: {pkg,<<"cf">>,<<"0.2.2">>,
                                   <<"7F2913FFF90ABCABD0F489896CFEB0B0674F6C8DF6C10B17A83175448029896C">>}

Obsolete time zone database

Time zone database needs maintaining. Upstream erlang_localtime library gets some attention and already received several updates since the fork.

I think we would be better off if we merge the fork and use the upstream erlang_localtime directly.
Dmitry Melnikov (maintainer of upstream erlang_localtime) has agreed to merge choptastic/erlang_localtime@6e14b6e so I do not see any barriers in doing so. Please see dmitryme/erlang_localtime#25 and comment. Thanks.

hex package

it would be really great if there were a hex package provided for qdate :)

OTP-19 support

Hi

qdate fail to compile with OTP-19 because of unsupported spec format in erlware-commons 0.15.0.
This as been fixed in release 0.20.0, so qdate rebar.config should be updated to use this version

dependency on erlware commons 0.20.0 instead of 1.0.0

qdate depends on a very old version erlware commons which does not compile with r19 any chance of bumping this for the hex package?

Get ISO8601 parsing working properly.

not building any more

git clone https://github.com/choptastic/qdate.git
cd qdate/
make

===> Verifying dependencies...
===> Fetching erlang_localtime ({git,                                                                                                
                                    "git://github.com/choptastic/erlang_localtime.git",                                          
                                    {branch,master}})                                                                            
===> Fetching erlware_commons ({git,                                                                                                 
                                   "git://github.com/erlware/erlware_commons.git",                                               
                                   {branch,master}})                                                                             
===> Package not buildable with rebar3: cf-0.2.1

cf is a dep of erlware_commons.
./rebar3 update

===> Updating package index...
===> Only existing version of ezmq is 0.2.0 which does not match constraint ~> 0.3.1. Using anyway, but it is not guarenteed to work.

Update not helps :(
Although dep erlware_commons builds:
cd _build/default/lib/erlware_commons/
./rebar3 compile

===> Verifying dependencies...
===> Fetching cf ({pkg,<<"cf">>,<<"0.2.1">>})
===> Version cached at /home/tihon/.cache/rebar3/hex/default/packages/cf-0.2.1.tar is up to date, reusing it
===> Compiling cf
===> Compiling erlware_commons

qdate:set_timezone("UTC+8")'s error

qdate:set_timezone("UTC+8"),
qdate:to_unixtime(calendar:local_time())

will raise error:

([email protected])32> qdate:to_unixtime(calendar:local_time()).
** exception error: no function clause matching calendar:date_to_gregorian_days(error) (calendar.erl, line 124)
     in function  calendar:datetime_to_gregorian_seconds/1 (calendar.erl, line 137)
     in call from qdate:to_unixtime/2 (src/qdate.erl, line 294)

qdate:to_date("UTC+8", util:unixtime()). will be "{error,unknown_tz}"

"GMT" works

Differences for ranges

Qdate could benefit from more date arithmetic in the sense of finding out the difference between two dates.

difference_days(Date1, Date2) would return the number of days between those two dates.

difference_minutes(Date1, Date2) would do the same (though this is easy because of the easy conversion to unix timestamps - same with seconds (obviously), hours.

calculating differences of days with respect is more complicated if a range crosses a daylight savings time.

My gut is telling me that the answer to both is to strip off any time component and use just dates, then convert to gregorian days with calendar:date_to_gregorian_days(Date). From there, the math is simple. Maybe the time component could then be re-added to the original dates and the time component compared to retrieve the fractional part of a day with compared times.

Something to chew on.