mathieuprog/tz

Time zone support for Elixir

tz's Issues

Question: avoid compile on unit test but keep update in app

Hi,

I followed your Readme and configured my app to auto-update data. In the end, I had to remove it since it would always compile when I run mix test: it adds a lot of significant time (and to me was not worth it)

Is there a way to enable auto-update for data within my app (in production for instance) but not when running unit tests ?

Thanks

`(CaseClauseError) no case clause matching: nil` for some lookups

iex(1)> DateTime.now!("America/Argentina/Salta", Tz.TimeZoneDatabase)
** (CaseClauseError) no case clause matching: nil
    (tz 0.20.0) lib/time_zone_database.ex:43: Tz.TimeZoneDatabase.do_find_period_for_secs/3
    (tz 0.20.0) lib/time_zone_database.ex:33: Tz.TimeZoneDatabase.find_period_for_secs/3
    (elixir 1.12.0) lib/calendar/datetime.ex:623: DateTime.shift_zone_for_iso_days_utc/5
    (elixir 1.12.0) lib/calendar/datetime.ex:736: DateTime.now!/2

This is with Elixir 1.12.0 and Erlang 24.0 on macOS.

required files missing in CI

I'm getting this error only in a particular CI env and not sure why, are those files that tz is meant to download or expects to already be there, and if so where do they usually come from?

    ** (EXIT) an exception was raised:

        ** (RuntimeError) files ["antarctica", "asia", "australasia", "backward", "etcetera", "europe", "northamerica", "southamerica", "iso3166.tab", "zone1970.tab"] are missing in /var/www/bonfire/_build/prod/rel/bonfire/lib/tz-0.26.5/priv/tzdata2023d

            (tz 0.26.5) lib/tz/compiler.ex:196: Tz.Compiler.validate_tzdata_dir_has_required_files/1

            (tz 0.26.5) lib/tz/compiler.ex:15: Tz.Compiler.compile/0

            (tz 0.26.5) lib/tz/updater.ex:23: Tz.Updater.maybe_recompile/0

            (tz 0.26.5) lib/tz/update_periodically.ex:31: Tz.UpdatePeriodically.init/1

            (stdlib 4.0.1) gen_server.erl:848: :gen_server.init_it/2

            (stdlib 4.0.1) gen_server.erl:811: :gen_server.init_it/6
 
            (stdlib 4.0.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
 
[os_mon] memory supervisor port (memsup): Erlang has closed

Allow Tz.WatchPeriodically to accept and call a function when triggered.

Since using Tz.UpdatePeriodically on memory-restricted platforms may not be possible, it would be useful if Tz.WatchPeriodically could call a function whenever it logs that there is an update. This would allow users of the library to do something proactive, like trigger an email or SMS.

Is it possible to make the custom data dir a runtime value?

Currently it is set at compile time.

Would that be ok for the value to be fetched at runtime, so it can be set as an environment variable in production environments or docker containers?

Thank you

Suggestion: Add a CHANGELOG

Figured I'd toss this out there. I follow this project's releases and at the moment the releases don't contain any info about what changed in each release. It'd be great to either have a CHANGELOG.md file (like Keep A Changelog suggests) or to add the changes to the Github releases. It would make it easier to follow the project updates.

The other idea would be to point in each release notes to the hex.pm version diffs.

Just an idea 💡

Release crashes when configuring data_dir at runtime

I just migrated a project from tzdata to your tz module, but I'm running into an issue when trying to run a release that sets the :data_dir config variable at runtime, in the config/runtime.exs file.

ERROR! the application :tz has a different value set for key :data_dir during runtime compared to compile time. Since this application environment entry was marked as compile time, this difference can lead to different behaviour than expected:

  * Compile time value was not set
  * Runtime value was set to: "local/tz/"

To fix this error, you might:

  * Make the runtime value match the compile time one

  * Recompile your project. If the misconfigured application is a dependency, you may need to run "mix deps.compile tz --force"

  * Alternatively, you can disable this check. If you are using releases, you can set :validate_compile_env to false in your release configuration. If you are using Mix to start your system, you can pass the --no-validate-compile-env flag

{"init terminating in do_boot",{<<"aborting boot">>,[{'Elixir.Config.Provider',boot,2,[]}]}}
init terminating in do_boot ({,[{Elixir.Config.Provider,boot,2,[]}]})

Crash dump is being written to: erl_crash.dump...done

First, I found Issue #11 where you added runtime fetching of the :data_dir config, so thank you for that. And technically, this works. However, due to some checking that's done by releases when they start up, a release that uses this module will crash if it also sets the :data_dir config env during runtime.

I duplicated this issue with a bare-bones elixir project at https://github.com/nshafer/tzrel. I wrote up the description of the problem and how to duplicate it in the README. So it should be trivial for you to clone that repo, create a release, and duplicate the problem.

But in short, while the code is actually running just fine because Tz.IanaDataDir.dir/0 is doing a Application.get_env(:tz, :data_dir) at runtime, the problem instead is being caused in Tz.CompileRunner. In there, in the module level code, a call to Application.compile_env(:tz, :data_dir) is being made. So the elixir release is spotting this and throwing the error because the value of :data_dir that Tz.CompileRunner received during the compile is different than what it's going to get during runtime. I confirmed this is the problem by commenting out lines 29-39 in that file, and the release ran fine. I think it's a basic sanity check that the release code is doing, and it can be disabled, but that seems a bit like a sledgehammer approach, and I would prefer to keep this sanity check enabled for my releases.

Looking at the code in Tz.CompileRunner, it looks like it's intended to throw a compile-time error if the user of the module doesn't configure :iana_version and :data_dir correctly. I don't know the exact intent or impact of changing this code, or the story behind it, so I didn't feel comfortable submitting a PR that just removed it. At least not without discussing it first. And I'm not sure there's any other way to do this compile-time check without calling Application.compile_env(). But, since the data_dir can be changed at runtime, this code won't catch runtime misconfigurations anyways. So maybe it needs to be moved into runtime code somewhere. But I'm not sure where.

Anyways, thanks for your time and the module.

Zone abbreviation lookup inconsistent (EST and EDT)

I'm hoping the PeriodsBuilder will be able to lookup a timezone based on the abbreviation, however it seems to be inconsistent. For example

Tz.TimeZoneDatabase.time_zone_periods_from_wall_datetime(~N[2020-04-01 00:00:00], "EDT")
{:error, :time_zone_not_found}
Tz.TimeZoneDatabase.time_zone_periods_from_wall_datetime(~N[2020-12-01 00:00:00], "EST")
{:ok, %{std_offset: 0, utc_offset: -18000, zone_abbr: "EST"}}

When I look at the rules it's based on, I see this:

# Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
Rule	US	2007	max	-	Mar	Sun>=8	2:00	1:00	D
Rule	US	2007	max	-	Nov	Sun>=1	2:00	0	S

So I'm thinking that the EDT abbreviation ought to work between Mar8 thru Nov1, and EST abbreviation ought to work between Nov1 thru Mar8.

WatchPeriodically keeps complaining about a new version

Hello,

I just installed this package, and added the Tz.WatchPeriodically as a child in my supervisor.

When going into the console with iex -S mix I see these messages:

[debug] Tz is checking for IANA time zone database updates
[warning] Tz found a more recent time zone database available for download at https://data.iana.org/time-zones/releases/tzdata2022c.tar.gz

So I did a manual update with:

Tz.Updater.maybe_recompile()

All good. But when I close the console and re-open it, I'm still seeing the warning above.
The output of Tz.IanaDataDir.tzdata_version() is "2022c".

Do you know what could be going on?

Thanks!

Time converting and back is not symmetrical

I have some property tests that I wrote for a project where we are using tz. I ran into a specific date time that has an issue. Here is a test for it.

  # in config
  config :elixir, :time_zone_database, Tz.TimeZoneDatabase

  # TEST

  test "bad property" do
    date_time_utc = ~U[2029-12-31 10:15:00Z]
    time_zone = "Pacific/Chatham"

    zoned_date_time =
      date_time_utc
      |> DateTime.shift_zone!(time_zone)

    naive = DateTime.to_naive(zoned_date_time)

    assert zoned_date_time == DateTime.from_naive!(naive, time_zone)
  end

It fails until ~U[2030-01-01 00:00:00Z] is the input at which point it passes.

Any year before 2029 passes. Any year after 2029 fails.

I tried to determine why and ran out of time. I'm happy to help figure this out. I don't really know where to start.

Out of memory when recompiling time zone periods on a memory-limited system

I'm running an app on fly.io, with a 1GB container size. Using Tzdata I can boot without issue. However, switching to Tz I'm seeing this on startup:

14:18:01.356 [info] Tz is downloading the latest IANA time zone database (version 2021c)...
14:18:02.785 [info] Tz download done
14:18:02.814 [info] Tz is recompiling time zone periods...
[    7.749887] Out of memory: Killed process 511 (beam.smp) total-vm:2665752kB, anon-rss:902056kB, file-rss:0kB, shmem-rss:63628kB, UID:0 pgtables:2380kB oom_score_adj:0
Main child exited with signal (with signal 'SIGKILL', core dumped? false)
Reaped child process with pid: 595, exit code: 0
Starting clean up.
Process appears to have been OOM killed!

Config is:

config :elixir, :time_zone_database, Tz.TimeZoneDatabase
config :tz, reject_time_zone_periods_before_year: 2019

I'm not able to replicate this locally, unfortunately. I'm using docker run --memory 512m --rm -i -p 8080:8080 {container} but I'm on macOS and don't hit a memory error.

Error with Iana version 2023a

When downloading version 2023a of database and compiling, an error is raised:

** (EXIT) an exception was raised:
        ** (RuntimeError) unexpected case: 3 ongoing rules for rule "Palestine"
            (tz 0.24.0) lib/tz/compiler.ex:25: anonymous fn/1 in Tz.Compiler.compile/0
            (elixir 1.14.1) lib/enum.ex:980: anonymous fn/3 in Enum.each/2
            (stdlib 4.1.1) maps.erl:411: :maps.fold_1/3
            (elixir 1.14.1) lib/enum.ex:2480: Enum.each/2
            (tz 0.24.0) lib/tz/compiler.ex:22: anonymous fn/3 in Tz.Compiler.compile/0
            (elixir 1.14.1) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
            (tz 0.24.0) lib/tz/compiler.ex:18: Tz.Compiler.compile/0
            (tz 0.24.0) lib/tz/updater.ex:21: Tz.Updater.maybe_recompile/0
            (tz 0.24.0) lib/tz/update_periodically.ex:26: Tz.UpdatePeriodically.init/1
            (stdlib 4.1.1) gen_server.erl:851: :gen_server.init_it/2
            (stdlib 4.1.1) gen_server.erl:814: :gen_server.init_it/6
            (stdlib 4.1.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Add option to force the use of a specific iana version?

Logger.warn/1 is deprecated on Elixir 1.15 in favour of Logger.warning/2

With the publishing of Elixir 1.15.0-rc.0 today, the following deprecation is now in effect:

==> tz
Compiling 16 files (.ex)
warning: Logger.warn/1 is deprecated. Use Logger.warning/2 instead
  lib/tz/watch_periodically.ex:19: Tz.WatchPeriodically.watch/1

Given Logger.warn/1 is a macro perhaps there's a way to conditionally compile in either Logger.warning/2 if its defined and Logger.warn/1 if not? I'm fine to work on a PR if that's helpful.

Wrong zone_abbr in some time zones

Hello,

I have found some wrong zone_abbr:

iex(1)> Tz.TimeZoneDatabase.time_zone_periods_from_wall_datetime(~N[1905-09-15 05:36:31], "Europe/London")
{:ok,
 %{
   from: %{
     utc_gregorian_seconds: 58314556875,
     wall_gregorian_seconds: 58314556875
   },
   std_offset: 0,
   to: %{
     utc_gregorian_seconds: 60475255200,
     wall_gregorian_seconds: 60475255200
   },
   utc_offset: 0,
   zone_abbr: ""
 }}
iex(2)> Tz.TimeZoneDatabase.time_zone_periods_from_wall_datetime(~N[1923-05-22 11:54:39], "Asia/Tokyo")
{:ok,
 %{
   from: %{
     utc_gregorian_seconds: 59579506800,
     wall_gregorian_seconds: 59579539200
   },
   std_offset: 0,
   to: %{
     utc_gregorian_seconds: 61483417200,
     wall_gregorian_seconds: 61483449600
   },
   utc_offset: 32400,
   zone_abbr: "JT"
 }}
iex(3)> Tz.TimeZoneDatabase.time_zone_periods_from_wall_datetime(~N[1948-06-05 21:50:01], "America/Nassau")
{:ok,
 %{
   from: %{
     utc_gregorian_seconds: 60342124170,
     wall_gregorian_seconds: 60342106170
   },
   std_offset: 0,
   to: %{
     utc_gregorian_seconds: 61987878000,
     wall_gregorian_seconds: 61987860000
   },
   utc_offset: -18000,
   zone_abbr: "ET"
 }}

The expected zone_abbr are:

input: ~N[1905-09-15 05:36:31], "Europe/London"
expected zone_abbr: "GMT"

input: ~N[1923-05-22 11:54:39], "Asia/Tokyo"
expected zone_abbr: "JST"

input: ~N[1948-06-05 21:50:01], "America/Nassau"
expected zone_abbr: "EST"

I assume this happens for zone-state that refers to a rule without a transition (no rule is applied) in the zone-state.

print a warning but allow the app to start if there's an HTTP error

Not sure if this is expected behaviour, but would be great to not block the app from starting:

** (Mix) Could not start application my_app: MyApp.Application.start(:normal, []) returned an error: shutdown: failed to start child: Tz.UpdatePeriodically
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, %Mint.HTTP1{host: "data.iana.org", port: 443, request: nil, streaming_request: nil, socket: {:sslsocket, {:gen_tcp, #Port<0.110>, :tls_connection, :undefined}, [#PID<0.1444.0>, #PID<0.1443.0>]}, transport: Mint.Core.Transport.SSL, mode: :active, scheme_as_string: "https", requests: {[], []}, state: :closed, buffer: "", proxy_headers: [], private: %{}, log: false}, %Mint.TransportError{reason: :closed}, []}
            (tz 0.26.2) lib/tz/http/mint/http_client.ex:30: Tz.HTTP.Mint.HTTPClient.recv_response/2
            (tz 0.26.2) lib/tz/http/mint/http_client.ex:18: Tz.HTTP.Mint.HTTPClient.request/2
            (tz 0.26.2) lib/tz/updater.ex:56: Tz.Updater.fetch_latest_iana_tz_version/0
            (tz 0.26.2) lib/tz/updater.ex:32: Tz.Updater.maybe_update_tz_database_to_latest_version/0
            (tz 0.26.2) lib/tz/updater.ex:14: Tz.Updater.maybe_recompile/0
            (tz 0.26.2) lib/tz/update_periodically.ex:31: Tz.UpdatePeriodically.init/1
            (stdlib 5.1.1) gen_server.erl:962: :gen_server.init_it/2
            (stdlib 5.1.1) gen_server.erl:917: :gen_server.init_it/6

(RuntimeError) unexpected case: 3 ongoing rules for rule "Palestine" (2023c)

10:18:11.182 [info]  Tz is downloading the latest IANA time zone database (version 2023c)...

10:18:12.298 [info]  Tz download done

10:18:12.348 [info]  Tz is recompiling time zone periods...
** (RuntimeError) unexpected case: 3 ongoing rules for rule "Palestine"
    (tz 0.20.1) lib/compiler.ex:27: anonymous fn/1 in Tz.Compiler.compile/0
    (elixir 1.13.4) lib/enum.ex:942: anonymous fn/3 in Enum.each/2
    (stdlib 4.3) maps.erl:411: :maps.fold_1/3
    (elixir 1.13.4) lib/enum.ex:2408: Enum.each/2
    (tz 0.20.1) lib/compiler.ex:24: anonymous fn/3 in Tz.Compiler.compile/0
    (elixir 1.13.4) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
    (tz 0.20.1) lib/compiler.ex:20: Tz.Compiler.compile/0
    (tz 0.20.1) lib/updater.ex:20: Tz.Updater.maybe_recompile/0