earowang/sugrrants

SUpporting GRaphics with R for ANalysing Time Series

sugrrants

The goal of sugrrants is to provide supporting graphs with R for analysing time series data. It aims to fit into the tidyverse and grammar of graphics framework for handling temporal data.

Installation

You could install the stable version on CRAN:

install.packages("sugrrants")

You could also install the development version from Github using:

# install.packages("remotes")
remotes::install_github("earowang/sugrrants")

Usage

Calendar-based graphics

The fully-fledged faceting calendar facet_calendar() unlocks day-to-day stories.

library(dplyr)
library(sugrrants)
hourly_peds %>%
  filter(Date < as.Date("2016-05-01")) %>% 
  ggplot(aes(x = Time, y = Hourly_Counts, colour = Sensor_Name)) +
  geom_line() +
  facet_calendar(~ Date) + # a variable contains dates
  theme_bw() +
  theme(legend.position = "bottom")

On the other hand, the frame_calendar() provides tools for re-structuring the data into a compact calendar layout, without using the faceting method. It is fast and light-weight, although it does not preserve the values.

p <- hourly_peds %>%
  filter(Sensor_ID == 9, Year == 2016) %>%
  mutate(Weekend = if_else(Day %in% c("Saturday", "Sunday"), "Weekend", "Weekday")) %>%
  frame_calendar(x = Time, y = Hourly_Counts, date = Date) %>% 
  ggplot(aes(x = .Time, y = .Hourly_Counts, group = Date, colour = Weekend)) +
  geom_line() +
  theme(legend.position = "bottom")
prettify(p)

Google Summer of Code 2017

This package is part of the project—Tidy data structures and visual methods to support exploration of big temporal-context data, which has been participated in Google Summer of Code 2017 (gsoc), for R project for statistical computing.

A new function frame_calendar() [here and here] in the sugrrants package has been developed and documented for calendar-based graphics. I have also written a vignette [source and reader view], which introduces and demonstrates the usage of the frame_calendar() function. Many unit tests have been carried out to ensure the expected performance of this function. The function implements non-standard evaluation and highlights the tidy evaluation in action. The initial release (v0.1.0) of the package has been published on CRAN during the gsoc summer time.

I have initialised a new R package tsibble for tidy temporal data, as part of the project. The tsibble() function constructs a new tbl_ts class for temporal data, and the as_tsibble() helps to convert a few ts objects into the tbl_ts class. Some key verbs (generics) from the dplyr package, such as mutate(), summarise(), filter(), have been defined and developed for the tbl_ts data class. The tsibble package was highly experimental over the period of the gsoc [commits], and these functions are very likely to be changed or improved in the future.

A new package rwalkr has been created and released on CRAN during the gsoc summer. This package provides API to Melbourne pedestrian sensor data and arrange the data in tidy temporal data form. Two functions including walk_melb() and shine_melb(), have been written and documented as the v0.1.0 and v0.2.0 releases on CRAN. The majority of the code for the function run_melb() has been done, but the interface needs improving after the gsoc.

Miscellaneous

The acronym of sugrrants is SUpporting GRaphs with R for ANalysing Time Series, pronounced as “sugar ants” that are a species of ant endemic to Australia.

---

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

sugrrants's Issues

facet_calendar: x axis not scaling

The x-axis is not freely scaling when scale = "free_x" is set.

library(tsibble)
#> 
#> Attaching package: 'tsibble'
#> The following object is masked from 'package:stats':
#> 
#>     filter
library(lubridate)
#> 
#> Attaching package: 'lubridate'
#> The following objects are masked from 'package:tsibble':
#> 
#>     interval, new_interval
#> The following object is masked from 'package:base':
#> 
#>     date
library(ggplot2)
elec_mth <- tsibbledata::elecdemand %>% 
  filter_index(~ "2014-01-30") %>% 
  index_by(datehour = floor_date(index, "hour")) %>% 
  summarise(avg_supply = mean(Demand)) %>% 
  mutate(
    date = as_date(datehour)
  )

# Scaling of x-axis seems ignored
elec_mth %>% 
  ggplot(aes(x = datehour, y = avg_supply)) +
  geom_line() + 
  sugrrants::facet_calendar(~ date, scales = "free_x")

# Works fine with facet_wrap
elec_mth %>% 
  ggplot(aes(x = datehour, y = avg_supply)) +
  geom_line() + 
  facet_wrap(~ date, scales = "free_x")

^{Created on 2019-01-01 by the reprex package (v0.2.1)}

Issue with Monthly Calendar Display of Months that span 6 weeks

Monthly Calendar Display of Months that span 6 weeks do not display properly. For example, July of 2017

A variant function of `frame_calendar()` to handle multiple columns

frame_calendar() accepts one or multiple variables in the y argument. That looks like a bad API and inefficient internals.

The current coding also disrupts the environment. If a variable named "n" is passed to y, it triggers an error in dots2str(), due to the conflict from dplyr::n().

Use frame_calendar2() instead to handle multiple variables? #5

Plot labels for incomplete time series

Hi Earo, great work with sugrrants - been using it lots. As discussed noticed some odd behaviour when using facet_wrap with an incomplete time series. Reproducible example to play around with below.

rm(list=ls())

require(sugrrants)
require(lubridate)
require(dplyr)
require(tidyr)

n_days <- 365
n_groups <- 4
start_date <- dmy("1/1/2010")
bla <- data.frame(Group = rep(LETTERS[1:n_groups], each = 24*n_days),
                  Date = rep(start_date + days(0:(n_days-1)), each = 24),
                  Time = rep(1:24, n_groups*n_days),
                  Value = rnorm(24*n_days*n_groups)) %>% 
  filter(!(Group == "B" & month(Date) %in% 5:12))

p <- bla %>% 
  group_by(Group) %>% 
  frame_calendar(x = Time, y = Value, date = Date,
                 calendar = "monthly", ncol = 2) %>% 
  ggplot(aes(x = .Time, y = .Value, group = Date)) +
  geom_line() +
  facet_wrap(~Group)
prettify(p)

You'll see the plot for Group B has it's time-series stretched to fill all the space, but the plot labels are the same as for all the other groups.

Need to improve `frame_calendar()` in conjunction with `group_by()`

library(tidyverse)
#> ── Attaching packages ───────────────────────────── tidyverse 1.2.1 ──
#> ✔ ggplot2 2.2.1     ✔ purrr   0.2.4
#> ✔ tibble  1.3.4     ✔ dplyr   0.7.4
#> ✔ tidyr   0.7.2     ✔ stringr 1.2.0
#> ✔ readr   1.1.1     ✔ forcats 0.2.0
#> ── Conflicts ──────────────────────────────── tidyverse_conflicts() ──
#> ✖ dplyr::filter() masks stats::filter()
#> ✖ dplyr::lag()    masks stats::lag()
library(sugrrants)
sml_flagstaff <- pedestrian %>% 
  filter(Sensor_Name == "Flagstaff Station", Month == "January")
sx <- pedestrian %>% 
 filter(Sensor_Name == "Southern Cross Station")  
ped <- bind_rows(sml_flagstaff, sx) %>% 
  filter(Year == 2016)
p <- ped %>%
  group_by(Sensor_Name) %>% 
  frame_calendar(Time, Hourly_Counts, Date) %>% 
  ggplot(aes(.Time, .Hourly_Counts, group = Date)) +
  geom_line() +
  facet_wrap(~ Sensor_Name)
prettify(p)

Issue with hms inputs to frame_calendar

library(sugrrants)
#> Loading required package: ggplot2
{tsibbledata::elecdemand %>% 
        mutate(hour = hms::as.hms(index),
               date = date(index)) %>%
        frame_calendar(x = hour, y = Demand, date = date) %>%
        ggplot(aes(x=.hour, y=.Demand, group = date)) + 
        geom_line()} %>%
    prettify()
#> Error in mutate(., hour = hms::as.hms(index), date = date(index)): could not find function "mutate"

Created on 2018-07-05 by the reprex package (v0.2.0).

Y axis liimts when using multiple variables

The ylim seems for each panel is only dependant on the range of the first variable in vars().

I.e., the following code results in an ugly graph ...

frame_calendar(x = time, y = vars(vearly, early, ontime, late, vlate), date = date)

... while this code (I've just put vlate as var 1) looks nice ...

frame_calendar(x = time, y = vars(vlate, vearly, early, ontime, late), date = date)

tibble is getting stricter about recycling

This used to work with previous versions of tibble but not anymore:

library(tibble)
df <- tibble(x = 1:21)
df$y <- 1:7
#> Error: Tibble columns must have consistent sizes, only values of size one are recycled:
#> * Size 21: Existing data
#> * Size 7: Column `y`

^{Created on 2020-03-02 by the reprex package (v0.3.0.9000)}

And this is IIUC the cause of the failure we see for this package when tested against the dev version of dplyr (to become dplyr 1.0.0):

[master] 197.2 MiB ❯ revdepcheck::revdep_details(revdep = "sugrrants")
══ Reverse dependency check ═════════════════════════════════════════════════════════════════════════════════════════════════ sugrrants 0.2.5 ══

Status: BROKEN

── Newly failing

x checking examples ... ERROR
x checking tests ...

── Before ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
0 errors ✓ | 0 warnings ✓ | 0 notes ✓

── After ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> checking examples ... ERROR
  Running examples in ‘sugrrants-Ex.R’ failed
  The error most likely occurred in:
  
  > ### Name: frame_calendar
  > ### Title: Rearrange a temporal data frame to a calendar-based data format
  > ###   using linear algebra
  > ### Aliases: frame_calendar prettify
  > 
  > ### ** Examples
  > 
  > library(dplyr, warn.conflicts = FALSE)
  > # compute the calendar layout for the data frame
  > calendar_df <- hourly_peds %>%
  +   filter(Sensor_ID == 13, Year == 2016) %>%
  +   frame_calendar(x = Time, y = Hourly_Counts, date = Date, nrow = 4)
  Error: Tibble columns must have consistent sizes, only values of size one are recycled:

> checking tests ...
  See below...

── Test failures ───────────────────────────────────────────────────────────────────────────────────────────────────────────────── testthat ────

> library(testthat)
> library(sugrrants)
Loading required package: ggplot2
> 
> test_check("sugrrants")
── 1. Error: Multiple y's and NA's (@test-calendar.R#11)  ──────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 21: Existing data
* Size 7: Column `day`
Backtrace:
  9. sugrrants::frame_calendar(., x = x, y = vars(ymin, ymax), date = Date)
 12. sugrrants:::gen_reference.monthly(...)
 14. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 15. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 16. tibble:::tbl_subassign_col(x, j, value)
 17. tibble:::vectbl_recycle_rows(...)

── 2. Error: Variable scoping (@test-calendar.R#30)  ───────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
  1. testthat::expect_named(...)
  5. sugrrants:::frame_calendar.default(ped, x = Time, y = n, date = Date)
  7. sugrrants:::gen_reference.monthly(...)
  9. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 10. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 11. tibble:::tbl_subassign_col(x, j, value)
 12. tibble:::vectbl_recycle_rows(...)

── 3. Error: Some column names of data are used in the function (@test-calendar.
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
  1. testthat::expect_is(...)
  5. sugrrants:::frame_calendar.default(...)
  7. sugrrants:::gen_reference.monthly(...)
  9. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 10. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 11. tibble:::tbl_subassign_col(x, j, value)
 12. tibble:::vectbl_recycle_rows(...)

── 4. Error: The argument calendar (@test-calendar.R#83)  ──────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
 1. sugrrants::frame_calendar(...)
 2. sugrrants:::frame_calendar.default(...)
 4. sugrrants:::gen_reference.monthly(...)
 6. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 7. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 8. tibble:::tbl_subassign_col(x, j, value)
 9. tibble:::vectbl_recycle_rows(...)

── 5. Error: The grouped data (@test-calendar.R#116)  ──────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
  1. dplyr::group_by(., Sensor_ID)
  9. sugrrants::frame_calendar(., x = Time, y = Hourly_Counts, date = Date)
 11. sugrrants:::frame_calendar.default(...)
 13. sugrrants:::gen_reference.monthly(...)
 15. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 16. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 17. tibble:::tbl_subassign_col(x, j, value)
 18. tibble:::vectbl_recycle_rows(...)

── 6. Error: The tsibble data (@test-calendar.R#126)  ──────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
  1. testthat::expect_equal(...)
 10. sugrrants::frame_calendar(., x = Time, y = Hourly_Counts, date = Date)
 15. sugrrants:::gen_reference.monthly(...)
 17. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 18. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 19. tibble:::tbl_subassign_col(x, j, value)
 20. tibble:::vectbl_recycle_rows(...)

── 7. Error: The identity 1 (@test-calendar.R#181)  ────────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
 1. sugrrants::frame_calendar(...)
 2. sugrrants:::frame_calendar.default(...)
 4. sugrrants:::gen_reference.monthly(...)
 6. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 7. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 8. tibble:::tbl_subassign_col(x, j, value)
 9. tibble:::vectbl_recycle_rows(...)

── 8. Error: The argument dir (@test-calendar.R#204)  ──────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
 1. sugrrants::frame_calendar(...)
 2. sugrrants:::frame_calendar.default(...)
 4. sugrrants:::gen_reference.monthly(...)
 6. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 7. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 8. tibble:::tbl_subassign_col(x, j, value)
 9. tibble:::vectbl_recycle_rows(...)

── 9. Error: The argument polar (@test-calendar.R#211)  ────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
 1. sugrrants::frame_calendar(...)
 2. sugrrants:::frame_calendar.default(...)
 4. sugrrants:::gen_reference.monthly(...)
 6. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 7. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 8. tibble:::tbl_subassign_col(x, j, value)
 9. tibble:::vectbl_recycle_rows(...)

── 10. Error: The output (@test-calendar.R#218)  ───────────────────────────────
Tibble columns must have consistent sizes, only values of size one are recycled:
* Size 28: Existing data
* Size 7: Column `day`
Backtrace:
 1. sugrrants::frame_calendar(...)
 2. sugrrants:::frame_calendar.default(...)
 4. sugrrants:::gen_reference.monthly(...)
 6. tibble:::`$<-.tbl_df`(`*tmp*`, "day", value = 1:7)
 7. tibble:::tbl_subassign(x, i = NULL, as_string(name), list(value))
 8. tibble:::tbl_subassign_col(x, j, value)
 9. tibble:::vectbl_recycle_rows(...)

══ testthat results  ═══════════════════════════════════════════════════════════
[ OK: 12 | SKIPPED: 0 | WARNINGS: 0 | FAILED: 10 ]
1.  Error: Multiple y's and NA's (@test-calendar.R#11) 
2.  Error: Variable scoping (@test-calendar.R#30) 
3.  Error: Some column names of data are used in the function (@test-calendar.R#58) 
4.  Error: The argument calendar (@test-calendar.R#83) 
5.  Error: The grouped data (@test-calendar.R#116) 
6.  Error: The tsibble data (@test-calendar.R#126) 
7.  Error: The identity 1 (@test-calendar.R#181) 
8.  Error: The argument dir (@test-calendar.R#204) 
9.  Error: The argument polar (@test-calendar.R#211) 
10. Error: The output (@test-calendar.R#218) 

Error: testthat unit tests failed
Execution halted

2 errors x | 0 warnings ✓ | 0 notes ✓

Naming of data

From @trianglegirl
There is a conflict in the data names of tsibble::pedestrian and sugrrants::pedestrian

Is it possible to deprecate/rename sugrrants::pedestrian somehow? Perhaps it could be replaced with an active binding to report it as deprecated in favour of some other name?

`prettify()`: add support for plotly

make prettify() S3 generic, and add support for plotly

"Dangling Monday" is plotted at top of calendar

Hi! Wonderful Package!

I noticed what may be a small bug where if there is a "dangling monday" for a given monthly calendar it seems to be plotted at the top of the calendar instead of at the bottom. See the attached example, in which Sept 30 is at the top instead of the bottom of the calendar.

My code:

alldata %>%
filter(Time.POSIX >= as_datetime("2019-09-01 00:00:00", tz="America/Los_Angeles"),
     Time.POSIX <= as_datetime("2019-10-01 00:00:00", tz="America/Los_Angeles"), 
     Response == "PM2.5_ATM_ug.m3") %>%  
mutate(Date = as_date(Time.POSIX),
     Time = 60*hour(Time.POSIX)+minute(Time.POSIX),
     Month = month(Time.POSIX, label = TRUE)) %>%  
ggplot(aes(x = Time, y = Value, color = Location)) +
geom_line() +
scale_x_continuous(breaks=c(0,360,780,1140), 
                 labels=c("12a","6a","12p","6p"),
                 limits=c(0,1440)) + 
labs(y = "PM2.5", x = "Time", title="Purple Air") +  
facet_calendar(~ Date, ncol=2) +
 theme_bw() +
theme(legend.position = "bottom")

Everything else I have plotted looks great,

Thanks, Neil

Logo contribution

Hello @earowang

I am a graphic designer. I contribute to open source software by designing logos in free time. I designed a logo for suggrants. What do you think?

Local scaling and polar coords

• When using free scale (daily) or one of the other local scale options, it should use a cutoff to remove small values. If a day has "effectively" zero values, then this should not be magnified into a trend.
  Polar coordinates does seem to work when x variable is 1-24

Is the default start of `facet-calendar` really Sunday?

In the documentation, the facet defines Sunday (7) as the start of the week:

sugrrants/R/facet-calendar.R

Line 10 in c108e8a

#' 1 means Monday, 7 means Sunday (default). You can set `lubridate.week.start`

However, the code indicates that the default value is actually 1, which is Monday?

sugrrants/R/facet-calendar.R

Line 39 in c108e8a

week_start = getOption("lubridate.week.start", 1),

My local implementations seem to confirm this.

Ignoring that, really love this function, and the package in general. Thanks for the work 😄

Error when using frame_calendar

Public holiday missing

Vic - AFL grand final eve public holiday missing for 2015 onward.

facet_calendar: formula cannot include expressions

Unlike ggplot facet functions, facet_calendar does not support usage of expressions for facetting. This is particularly useful in the time domain.

---

library(tsibble)
#> 
#> Attaching package: 'tsibble'
#> The following object is masked from 'package:stats':
#> 
#>     filter
library(lubridate)
#> 
#> Attaching package: 'lubridate'
#> The following objects are masked from 'package:tsibble':
#> 
#>     interval, new_interval
#> The following object is masked from 'package:base':
#> 
#>     date
library(ggplot2)
elec_mth <- tsibbledata::elecdemand %>% 
  filter_index(~ "2014-01-30") %>% 
  index_by(datehour = floor_date(index, "hour")) %>% 
  summarise(avg_supply = mean(Demand))

# Expressions are not evaluated
elec_mth %>% 
  ggplot(aes(x = datehour, y = avg_supply)) +
  geom_line() + 
  sugrrants::facet_calendar(~ as_date(datehour), scales = "free_x")
#> Error: Facet calendar only accepts (un)quoted variable and RHS formula.


# Works fine with facet_wrap
elec_mth %>% 
  ggplot(aes(x = datehour, y = avg_supply)) +
  geom_line() + 
  facet_wrap(~ as_date(datehour), scales = "free_x")

^{Created on 2019-01-01 by the reprex package (v0.2.1)}

Examples fail with dev ggplot2

...
>    p1 <- calendar_df %>% 
+      ggplot(aes(x = .Time, y = .Hourly_Counts, group = Date)) +
+      geom_line()
>    prettify(p1, size = 3, label.padding = unit(0.15, "lines"))
>    
>    # use in conjunction with group_by()
>    grped_calendar <- pedestrian %>% 
+      filter(Year == "2017", Month == "March") %>% 
+      group_by(Sensor_Name) %>% 
+      frame_calendar(
+        x = Time, y = Hourly_Counts, date = Date, sunday = TRUE
+      )
>    
>    p2 <- grped_calendar %>% 
+      ggplot(aes(x = .Time, y = .Hourly_Counts, group = Date)) +
+      geom_line() +
+      facet_wrap(~ Sensor_Name, nrow = 2)
Error in grouped_indices_grouped_df_impl(.data) : 
  Need at least one column for `hash()`
Calls: %>% ... group_indices.grouped_df -> grouped_indices_grouped_df_impl
Execution halted

Can you please help us figure out what's gone wrong?

Error returned in example for: frame_calendar()

calendar_df <- pedestrian %>%
filter(Sensor_ID == 13, Year == 2016) %>%
frame_calendar(x = Time, y = Hourly_Counts, date = Date, nrow = 4)
filter: removed 69972 out of 78755 rows (89%)

Error in loadNamespace(name) : there is no package called ‘tsibble’