simongray/el

Danish electricity price calendar.

el's Issues

Android/Google Calendar difficulties

webcal:// links seemingly do not work on Android. I am unsure whether they would work with clients other than Google's. I tried installing Outlook for Android (no account setup) and it didn't capture any intents from the browser. Google calendar apparently doesn't even support subscribing from the Android app, not even by typing in a URL.

In Chrome on macOS you can—I verified this—use a Google calendar URI to open directly in Google calendar, i.e. the link

https://calendar.google.com/calendar/u/0/r?cid=webcal://el.simongray.dk/calendar?language%3Dda&currency=DKK&max-price=1.00&region=DK2

will open the Google calendar web client and subscribe the Google user to the calendar.

Having a separate place to go for Google calendar requires having a separate submit button. This can be accomplished by setting a formaction on the extra submit button.

Overall approach

The simplest approach AFAIK is to create a basic web service that serves an ics feed at some URL. A user can then subscribe to this URL at e.g. https://simongray.dk/el-priser—or some version of that URL containing query-params to get a custom calendar feed. From how I read the—fairly sparse and dated—documentation it is up to the calendar software to keep these feeds updated via regular pulling.

So when a user accesses the website I basically need to do a network request to the API providing electricity prices (probably caching the result for n minutes) to generate ics data, serving this data via (probably) a Pedestal service with some basic rate limiting. That way the service can be kept completely open.

Links

• API for el-spotpriser: https://www.energidataservice.dk/tso-electricity/elspotprices
• ical4j feed tutorial: https://blog.lunatech.com/posts/2006-04-19-dynamic-calendars-icalendar-feeds-using-ical4j
  • Uses ical4j: https://github.com/ical4j/ical4j
  • Clojure libs/wrappers:
    • https://github.com/malcolmsparks/clj-ical
    • https://github.com/jeffmad/clj-icalendar
    • https://github.com/liquidz/clojure-ical
• ics/icalender spec documentation: https://icalendar.org/RFC-Specifications/all/

Alternatives

The other much more complex option is to make some service that integrates with Google Calendar, involving authentication etc. — I would definitely prefer not to go this route.

Danish translation of calendar events

Not too hard, just need to format the numbers differently and write the correct translations. The only real novel work is having to use the templating features of Tongue.

Unsupported language fail

When neither Danish nor English is specified, the language negotiation step will fail, resulting in a blank page. I'm speculating that this is why the Norwegian on Clojurians Slack couldn't get it working.

Internal server error: exception

Since around 6th of June 2023 I get the following error when trying to fetch the calendar feed:
https://el.simongray.dk/calendar?language=da&currency=DKK&max-price=1.00&region=DK1

Internal server error: exception

Also seem like I can't get any new feed when using https://el.simongray.dk since no valid data is received when clicking "Subscribe".

Rate limiting

Since this isn't built in to Caddy, several third party modules exist:

• https://caddyserver.com/docs/modules/http.handlers.rate_limit
  • https://github.com/RussellLuo/caddy-ext
  • https://github.com/mholt/caddy-ratelimit
• https://github.com/xuqingfeng/caddy-rate-limit
• https://golangexample.com/a-caddy-v2-extension-to-apply-rate-limiting-for-http-requests/
  • https://github.com/owlwang/caddy-ratelimit

Not sure if this implies compiling the server from scratch...?

Bring down memory usage

The web service consumes around a quarter of the available memory according to systemctl status el (~250MB out of 1GB), Caddy only consumes ~20MB. Altogether, something like 56-58% of the droplet's total memory is being consumed in idle state. It should be possible bring the JVM memory consumption down significantly.

Some good advice available on Clojurians Slack: https://clojurians.slack.com/archives/C03S1KBA2/p1652169123478019

Relative price ceiling

As suggested by a redditor: https://www.reddit.com/r/Denmark/comments/ukuixj/sparsommelig_s%C3%B8ndag_frugal_sunday_0805_2022/i7w1mui/

• The price ceiling should be able to be a relative figure above the daily minima.
  • e.g. if the daily minimum is 1.50 DKK and the price ceiling is 0.30 DKK, events are scheduled whenever the price is below 1.80 DKK for that day.
• I should probably signal this via a boolean query param such as relative=true on the API side.
• In the landing page, the form could be expanded with a set of radio buttons and some explanatory text to let the user know this is possible.
• There's no need to change anything in the prices micro-library. The existing prices-below function can be reused.