rstudio/gradethis

Tools for teachers to use with learnr

RStudio

RStudio is an integrated development environment (IDE) for the R programming language. Some of its features include:

• Customizable workbench with all of the tools required to work with R in one place (console, source, plots, workspace, help, history, etc.).
• Syntax highlighting editor with code completion.
• Execute code directly from the source editor (line, selection, or file).
• Full support for authoring Sweave and TeX documents.
• Runs on Windows, Mac, and Linux, and has a community-maintained FreeBSD port.
• Can also be run as a server, enabling multiple users to access the RStudio IDE using a web browser.

For more information on RStudio please visit the project website.

Getting the Code

RStudio is licensed under the AGPLv3, the terms of which are included in the file COPYING. You can find our source code repository on GitHub at https://github.com/rstudio/rstudio.

Documentation

For information on how to use RStudio check out our online documentation.

For documentation on running your own RStudio Server see the server getting started guide.

See also the following files included with the distribution:

• COPYING - RStudio license (AGPLv3)
• NOTICE - Additional open source software included with RStudio
• SOURCE - How to obtain the source code for RStudio
• INSTALL - How to build and install RStudio from source

If you have problems or want to share feedback with us please visit our community forum. For other inquiries you can also email us at [email protected].

gradethis's Issues

Rename grading demo file

rename grading-demo/grading-demo.Rmd to gradethis-demo/gradethis-demo.Rmd

Both file and in gradethis_demo()

check_code should not require use of `standardise_call`

It's confusing and hard to test. There should never be a "it's voodoo" when explaining how it works.

Current:

check_result(args, grader_args, learnr_args) # checks result

Proposed:

ck_fn <- check_result(args) # returns function that accepts grader_args / learnr_args
ck_fn # will be called after "*-check" evaluation as `ck_fn(grader_args, learnr_args)`

This allows for any function to be produced in any way as long as a function is returned from the chunk that accepts grader_args and learnr_args. We do not need to be cautious of parameter expansion as grader_args is already included and is a list.

This would require functions like check_result to return a function that does the evaluation. Rough example:

new_check_result(...) {
  results <- list(...)

  function(grader_args, learnr_args) {
    # check results here

    graded(TRUE)
  }
}

This would clean up the logic inside grade_learnr and would be much easier to enforce / document

Getting standarized formals from s3 generics

You need to evaluate everything in order to determine which s3 function is being called.

rlang::fn_fmls(mean)
$x


$...

this mean that the call_standardise_formals will not work at the moment for mean since the generic signature is just mean(x, ...). For example, if student puts mean(1:3, na.rm=FALSE), and the solution code is mean(1:3)

Rename expect_message names

There are 2 versions of expect_message functions

The one used in test_test_result.R

expect_message <- function(x, message, correct = FALSE) {
  expect_s3_class(x, "grader_graded")
  expect_equal(x$correct, correct)
  expect_true(grepl(message, paste0(x$message, collapse = ""), fixed = TRUE))
}

and the rest

expect_message <- function(x, message) {
  expect_s3_class(x, "grader_graded")
  expect_true(!x$correct)
  expect_true(grepl(message, paste0(x$message, collapse = ""), fixed = TRUE))
}

They are similar, but they should be re-written to use the same function signature (and conversely, the same expect check)

Builds stall: github/pages Expected — Waiting for status to be reported

the github/pages build is not working. It is constantly in the "yellow" phase (i.e., it doesn't pass, but it also isn't failing...).
This is making it so I cannot merge PRs

Add `""` to addin function that insterts check chunks

Put in more boilerplate text

all params supplied to any glue statement should start with a `.`

.message etc

Check code doesn't work with long quoted expressions

tidyr::gather(who, key, value, new_sp_m014:newrel_f65, ____ = ____)

Submit package to CRAN

• Rename package (#18)
• check_* methods should return functions to remove standarize_call (#26 )

Comparing dataframes via `check_result` is causing an `evaluate_condi` error

Conversation from this message: #36 (comment)

learnr shows an error message and this is what is shown in the r console:

Error in evaluate_condition(resu, grader_args, learnr_args): Assertion on 'res' failed: Must be of type 'logical', not 'character'.

Reproducible bits below:

Works in console

billboard <- structure(list(year = c(2000, 2000, 2000, 2000, 2000, 2000), 
    artist.inverted = c("Destiny's Child", "Santana", "Savage Garden", 
    "Madonna", "Aguilera, Christina", "Janet"), track = c("Independent Women Part I", 
    "Maria, Maria", "I Knew I Loved You", "Music", "Come On Over Baby (All I Want Is You)", 
    "Doesn't Really Matter"), time = structure(c(13080, 15480, 
    14820, 13500, 13080, 15420), class = c("hms", "difftime"), units = "secs"), 
    genre = c("Rock", "Rock", "Rock", "Rock", "Rock", "Rock"), 
    date.entered = structure(c(11223, 10999, 10887, 11181, 11174, 
    11125), class = "Date"), date.peaked = structure(c(11279, 
    11055, 10985, 11216, 11244, 11195), class = "Date"), x1st.week = c(78, 
    15, 71, 41, 57, 59), x2nd.week = c(63, 8, 48, 23, 47, 52), 
    x3rd.week = c(49, 6, 43, 18, 45, 43), x4th.week = c(33, 5, 
    31, 14, 29, 30), x5th.week = c(23, 2, 20, 2, 23, 29), x6th.week = c(15, 
    3, 13, 1, 18, 22), x7th.week = c(7, 2, 7, 1, 11, 15), x8th.week = c(5, 
    2, 6, 1, 9, 10), x9th.week = c(1, 1, 4, 1, 9, 10), x10th.week = c(1, 
    1, 4, 2, 11, 5), x11th.week = c(1, 1, 4, 2, 1, 1), x12th.week = c(1, 
    1, 6, 2, 1, 1), x13th.week = c(1, 1, 4, 2, 1, 1), x14th.week = c(1, 
    1, 2, 2, 1, 2), x15th.week = c(1, 1, 1, 4, 4, 2), x16th.week = c(1, 
    1, 1, 8, 8, 3), x17th.week = c(1, 1, 1, 11, 12, 3), x18th.week = c(1, 
    1, 2, 16, 22, 7), x19th.week = c(1, 8, 1, 20, 23, 8), x20th.week = c(2, 
    15, 2, 25, 43, 20), x21st.week = c(3, 19, 4, 27, 44, 25), 
    x22nd.week = c(7, 21, 8, 27, NA, 37), x23rd.week = c(10, 
    26, 8, 29, NA, 40), x24th.week = c(12, 36, 12, 44, NA, 41
    ), x25th.week = c(15, 48, 14, NA, NA, NA), x26th.week = c(22, 
    47, 17, NA, NA, NA), x27th.week = c(29, NA, 21, NA, NA, NA
    ), x28th.week = c(31, NA, 24, NA, NA, NA), x29th.week = c(NA, 
    NA, 30, NA, NA, NA), x30th.week = c(NA, NA, 34, NA, NA, NA
    ), x31st.week = c(NA, NA, 37, NA, NA, NA), x32nd.week = c(NA, 
    NA, 46, NA, NA, NA), x33rd.week = c(NA, NA, 47, NA, NA, NA
    ), x34th.week = c(NA_real_, NA_real_, NA_real_, NA_real_, 
    NA_real_, NA_real_), x35th.week = c(NA_real_, NA_real_, NA_real_, 
    NA_real_, NA_real_, NA_real_), x36th.week = c(NA_real_, NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_), x37th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x38th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x39th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x40th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x41st.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x42nd.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x43rd.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x44th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x45th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x46th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x47th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x48th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x49th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x50th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x51st.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x52nd.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x53rd.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x54th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x55th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x56th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x57th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x58th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x59th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x60th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x61st.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x62nd.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x63rd.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x64th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x65th.week = c(NA_real_, 
    NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), x66th.week = c(NA, 
    NA, NA, NA, NA, NA), x67th.week = c(NA, NA, NA, NA, NA, NA
    ), x68th.week = c(NA, NA, NA, NA, NA, NA), x69th.week = c(NA, 
    NA, NA, NA, NA, NA), x70th.week = c(NA, NA, NA, NA, NA, NA
    ), x71st.week = c(NA, NA, NA, NA, NA, NA), x72nd.week = c(NA, 
    NA, NA, NA, NA, NA), x73rd.week = c(NA, NA, NA, NA, NA, NA
    ), x74th.week = c(NA, NA, NA, NA, NA, NA), x75th.week = c(NA, 
    NA, NA, NA, NA, NA), x76th.week = c(NA, NA, NA, NA, NA, NA
    )), class = c("tbl_df", "tbl", "data.frame"), row.names = c(NA, 
-6L))

user <- reshape2::melt(
  head(billboard),
  id.vars = c("year", "artist.inverted", "track", "time",
              "genre", "date.entered", "date.peaked"),
  variable.name = "week",
  value.name = "rank",
  factorAsStrings = TRUE
)
user$week <- as.character(user$week)

solution <- tidyr::gather(head(billboard), "week", "rank", x1st.week:x76th.week)

all.equal(tibble::as_tibble(user), tibble::as_tibble(solution))

Fails in the learnr document

```{r zuzwislfkfnsoesa, exercise = TRUE}
# student code
user <- reshape2::melt(head(billboard), id.vars = c("year", "artist.inverted", "track",
        time", "genre", "date.entered", "date.peaked"),
    variable.name = "week",
    value.name = "rank",
    factorAsStrings = TRUE
)
user$week <- as.character(user$week)
```

```{r zuzwislfkfnsoesa-check}
solution <- tidyr::gather(head(billboard), "week", "rank", x1st.week:x76th.week)

grader::check_result(
  grader::pass_if(~ all.equal(tibble::as_tibble(.result), tibble::as_tibble(solution)))
)
```

What is Dan's role?

aut? ctb? huh?

grader -> gradethis internal variable renames

After changing the package name to gradethis (#18 and #54 ) there are a lot of internal variable names that refer to grader things (e.g., grader_graded, grader_args, grader_condition) that use the old package name.

These should be renamed to use gradethis

strict_check does not handle quosures correctly

Line 61 of strict_check.R has this:

  } else if (suppressWarnings(user == solution)) {

However, at this point both user and solution can be quosures if strict_check is being called by grade_learnr, in which case the == fails. The code should check if these params are quosures and !! if they are.

cc @gvwilson @schloerke

test grade_learnr bug

see: #21 (comment) from #11

the tests for grade_learnr do not seem to work with messages within pass_if, and the only the random praise is being passed along.

Function renames

grade_ prefixes:

• check_code -> grade_code
• check_result -> grade_result
• test_result -> grade_test grade_conditions

Others:

• grading_demo -> gradethis_demo

docs are wrong for `grade_learnr`

#' @param solution_code R code submitted by the user
#' @param user_code Code provided within the “-solution” chunk for the exercise.

should be flipped

Comparing NA values when solution is not an atomic

There is a difference between 3 == NA and NA == NA. This is probably the notation (==) that will be used by instructors, e.g., pass_if(~ .result == df)

When == is used in a comparison there is a difference between when the user and solution code are both NA (maybe should return a TRUE match), and when only one of them returns NA. Right now they both return NA, and I need to set na.rm = TRUE to get a boolean value from all that is not NA if there is a missing value.

Does this mean we need to replace any calls to == with all.equal or identical?

here's a minimal example of what I'm trying to show:

> user <- c(1, 2, 3, NA)
> solution <- c(1, 2, 3, NA) ## both are missing
> user == solution
[1] TRUE TRUE TRUE   NA
> all(user == solution)
[1] NA

> user <- c(1, 2, 3, NA)
> solution <- c(1, 2, 3, 4) ##  the solutions don't actually match
> user == solution
[1] TRUE TRUE TRUE   NA
> all(user == solution, na.rm = TRUE)
[1] TRUE

3 parts to a message

praise/encouragement + question specific + correct/incorrect message

e.g.,

             question specific (e.g., from pass_if)
             v
"You did it! That's the correct answer. Next, we'll see how we can apply this to dataframes."
^                                       ^
praise/encouragement                    correct message

There's currently only 2. The "praise/encouragement" is overwritten by "correct".

strict_check does not correctly unpipe solutions with multiple pipes

From the Nested Data section of https://tutorials.shinyapps.io/list-columns:

fix readme badges

links and tags still point to grader

Might need envir_prep in evaluate_condi

might need envir_prep. idk for sure. Not the priority right now

Originally posted by @schloerke in https://github.com/_render_node/MDIzOlB1bGxSZXF1ZXN0UmV2aWV3VGhyZWFkMTgzNDcyMDcyOnYy/pull_request_review_threads/discussion

not sure if this is needed since the function is called within check_result. but can come back to it if the instructor puts in a function that makes calls to variables in the environment.

Submit answers or log to instructor

I need a way for students to submit their work to the teacher. Ideally, this would be integrated into a learnr button.

I'm not yet sure what exactly should be submitted. The log of the students work in the tutorial seems like a good candidate.

I'm not yet sure where the submission should go. One idea is that the submission function/button would place an entry in the master log that a teacher could then work with using grader tools.

Add ORCID infos

Example in DESCRIPTION

person("Barret", "Schloerke", , "[email protected]", "aut",
           comment = c(ORCID = "0000-0001-9986-114X"))

unpipe cannot handle deeply nested dots (.)

The code below should return sum(4, 1, 2, sqrt(4) or return an error that advises people to avoid nesting .'s in sub-functions (because the correct behavior seems zany: 4 is inserted both at the dot and as the first argument).

pipe1 <- quote(4 %>% sum(1, 2, sqrt(.)))
unpipe(pipe1)
# sum(4, 1, 2, sqrt(.))

LICENCE file

check_result API

Grader is going through some API changes in #10 . Thank you all for your input and patience!

Two big motivations are at play: keep it simple (@garrettgman ), keep it clean (@wch ).

---

After talking it through with @blairj09, I think we have a clean, simple solution for check_result.

• Two check object functions: passes and fails. (These names do not class with checkmate or testthat.)
• passes and fails can take:
  • a Value - equivalent to ~ identical(., value)
  • a Formula - Evaluate the formula with . representing the user result
    • By using a formula (purrrr style) with ., there is no non-standard evaluation trickery to understand (such as a random ans being available). As long as we state that . is the user result, the validation can be processed however the user writes.
    • Should return TRUE / FALSE
    • Throwing will cause an error in computing (bad for author)
  • a Function that takes an user result argument
    • Should return TRUE / FALSE
    • Throwing will cause an error in computing (bad for author)
• Once an equality (against a value) is found or TRUE (from a formula / function) is returned, computation will stop. The respective message will be displayed as a correct (passes) or incorrect (fails) result.
• correct or incorrect params will be the default message for any passes or fails that do not provide a message. They will be glueed with appropriate information.

Example:

check_result(
  fails(5, "bad answer!"),
  fails(~ . == 5, "bad answer!"),
  fails(~ isTRUE(all.equal(., 5)), "bad answer!"),
  fails(~ . %in% c(5,7), "bad answer!"),
  fails(function(x) x == 5, "bad answer!"),
  fails(function(x) x == 5),
  fails(somepkg::some_method, "bad answer!"), # function(x) ...

  passes(4, "good answer!"),
  passes(~ . == 4, "good answer!"),
  passes(~ . %in% c(4,6), "good answer!"),
  passes(function(x) x == 4, "good answer!"),
  passes(function(x) x == 4),
  passes(somepkg::some_method, "good answer!"), # function(x) ...

  correct = "You got it right!",
  incorrect = "You got it wrong."
)

---

check_result will be kept separate from test_result. All tests will be executed for test_result.

• test_result will take
  • a test object which will only accept formulas or functions as defined above (no values). The message supplied will be provided if any throw occurs.
  • a Formula - the thrown error will be reported
  • a Function - the thrown error will be reported
• correct and incorrect will be glueed with appropriate information.

Example:

test_result(

  ~ checkmate::assert_function(., args = "x"),
  test(~ checkmate::assert_function(., args = "x"), "Make sure to have an 'x' argument!"),
  somepkg::some_expectation, # function(x) ...
  pryr::partial(checkmate::assert_function, args = "x"),

  correct = "{num_correct} / {num_total} correct! {random_praise()}",
  incorrect = "{num_correct} / {num_total} correct. {random_encourage()}"
)

---

Again, these are not set in stone... just a followup from discussions over the past 48 hrs.

cc @gvwilson

Function to launch a learnr tutorial within an IDE frame

The function should take a tutorial ID (url?) and open the tutorial in the viewer pane.

Context:

How can we transition students from learnr tutorials to the IDE?
How can we provide more sophisticated tutorials that require the use of the IDE?

I would like to enable students to launch a learnr tutorial within the IDE viewer pane. The tutorial would be hosted online elsewhere, for example at shinyapps.io, and the student would access it through an iframe. This removes the need to provide the raw .Rmd to the student (which would contain the solutions).

Together with #2, this would create a complete assignment workflow.

CRAN has a `gradeR` package already (released 2019-06-05)

https://cran.r-project.org/web/packages/gradeR/index.html

going to have to rename this package, most likely

Error upon submitting answer in demo

When I submit an answer to the first question in grader::grading_demo(), the doc closes, and I see this at the console:

Warning in grader_args$user : partial match of 'user' to 'user_quo'
Warning: Error in checker: `grade_learnr` should receive a `graded` value from every `-check` chunk
  79: stop
  78: checker
  77: evaluate_exercise
  74: observeEventHandler
   3: <Anonymous>
   1: rmarkdown::run

CONTRIBUTING file

grader-demo not working

@using the grader-demo in the tutorials folder, when clicking Submit Answer (the correct one or any) I receive:

Error : Base operators are not defined for quosures. Do you need to unquote the quosure? # Bad: myquosure1 == myquosure2 # Good: !!myquosure1 == !!myquosure2

Do you know why does this happen and how could I solve it? @garrettgman

test_solutions() does not use code in global setup chunk when testing exercises

Object params is made in the document's global knitr setup chunk

> library(grader)
> demo <- system.file("tutorials", "solutions-demo/solutions-demo.Rmd", package = "grader")
> test_solutions(demo)
setup: ✔ package ‘purrr’ was built under R version 3.4.4 
success-1-solution: ✔ 
warning-2-solution: ✔ NaNs produced 
failure-3-solution: ✖ argument "x" is missing, with no default 
success-4-solution: ✖ object 'params' not found 
failure-5-solution: ✖ object 'knitr_result' not found 
success-6-setup: ✔ 
success-6-solution: ✖ object 'params' not found 
warning-7-setup: ✔ This is a warning message! 
warning-7-solution: ✔ 
failure-8-setup: ✖ Uh oh. An error! 
failure-8-solution: ✖ object 'two' not found 
success-for-9: ✔ 
success-9-solution: ✖ object 'params' not found 
warning-for-10: ✔ Uh oh. A Warning! 
warning-10-solution: ✖ object 'params' not found 
failure-for-11: ✖ Uh oh. An error! 
failure-11-solution: ✖ object 'params' not found 
success-for-9: ✔ 
success-12-solution: ✖ object 'params' not found 
success-for-9: ✔ 
success-13-solution: ✖ object 'params' not found 
warning-for-10: ✔ Uh oh. A Warning! 
warning-14-solution: ✖ object 'params' not found 
warning-for-10: ✔ Uh oh. A Warning! 
warning-15-solution: ✖ object 'params' not found 
failure-for-11: ✖ Uh oh. An error! 
failure-16-solution: ✖ object 'params' not found 
failure-for-11: ✖ Uh oh. An error! 
failure-17-solution: ✖ object 'params' not found 
usererr-18-solution: ✖ object 'params' not found 
usererr-19-solution: ⚠ exercise.setup chunk_opt is not a single string. 
success-6-setup: ✔ 
usererr-20-solution: ✖ could not find function "add_one" 

use code coverage and code linting

Add to .travis.yml

r_binary_packages:
  - jimhester/lintr
  - jimhester/covr
after_success:
  - Rscript -e 'covr::codecov()'
  - Rscript -e 'lintr::lint_package()'

Exec in R

usethis::use_coverage("codecov")
usethis::use_build_ignore(".lintr")

lintr::lint_package() %>% 
  as.data.frame %>% 
  dplyr::group_by(linter) %>% 
  dplyr::tally(sort = TRUE) %$% 
  sprintf(
    "linters: with_defaults(\n    %s\n    NULL\n  )\n", 
    paste0(linter, " = NULL, # ", n, collapse = "\n    ")
  ) %>% 
  cat(file = ".lintr")

• Remove the packages from Description's Suggests.
• Adjust the lint to allow for lines up to 100 characters. (link: https://github.com/jimhester/lintr#project-configuration)

lintr::lint_package(path = '.', relative_path = TRUE, with_defaults(line_length_linter(100)))

• Add the code coverage badge to the readme

strict_check() fails for exercises with multiple calls

Remove `~` from the pipe message sent to student

From #17, the first code returns

I see that you are using pipe operators (e.g. %>%), so I want to let you know that this is how I am interpretting your code before I check it: ~gather(who, key, value, new_sp_m014:newrel_f65, na.rm = FALSE) I expected na.rm = TRUE where you wrote na.rm = FALSE. Try it again. I have a good feeling about this.

We should remove the ~ that appears in the message as it will confuse students. In this case , the ~ appears before gather.

Discussion for this comment: #30 (comment)

Clean up stale branches

Stale branches, @garrettgman can these be deleted?

• attempt-1.0

Updated last year by garrettgman

• infix-checking-bug

Updated last year by garrettgman

• pseudo-pipe-approach

Updated last year by garrettgman

• outside-in

Updated last year by garrettgman

Hunt for edge cases

learnr tutorial chunks to show inconsistencies (aka bugs)

• #33 Remove ~ when returning unpiped result
• Add line breaks before and after code unpipe (#30 (comment))
• #28 Always comes back correct. It seems to not parse arguments after the : column slicing. But seems the pipe is also involved. (#30 (comment))
• Piping to identity is inconsistent between pipelines and ggplot2 (#30 (comment))
• keywords in functions are returned when a mistake is found, even if kwargs are not passed and values are only passed by position. (#30 (comment))
• #36 Something is causing learnr/grader to take a long time (and also fail) (#30 (comment))
• check_code can see if a function definition is correct, but if they differ it will fail (expected to return error message) (#30 (comment))
• Should check_code evaluate the call? Or should it be comparing the call? (#30 (comment))

make check_ code more generic and less tied into learnr

After #11 gets merged with #21, we should make the checking code more generic by not explicitly passing around grader_args and learnr_args and only use the values from those list objects.

Allow pass_if and fail_if to pass in a NULL message

be able to use

grader::pass_if(~ .result == correct_df) instead of only

grader::pass_if(~ .result == correct_df, message = "1337")

Add global option to customize message for empty user input

No checking function should handle empty user input

launch tutorial at startup

Solved here: rstudio/rstudio#1579 (comment)

To use it, set something like the below in your .Rprofile:

setHook("rstudio.sessionInit", function(newSession) {
  if (newSession)
    message("Welcome to RStudio ", rstudioapi::getVersion())
}, action = "append")

reshape2 causing a really long time to grade

Get the data:

wget https://raw.githubusercontent.com/hadley/tidy-data/master/data/billboard.csv

Load the data:

billboard <- readr::read_csv(here::here('data/billboard.csv'))

Example student code:

# student code
stu <- reshape2::melt(billboard,
                      id.vars = c("year", "artist.inverted", "track", "time",
                                  "genre", "date.entered", "date.peaked"),
                      variable.name = "week",
                      value.name = "rank",
                      factorAsStrings = FALSE)
stu$week <- as.character(stu$week)
stu

Check submitted args against default arguments

log(log(2, base = exp(1)), base = exp(1)) # both

I expected log(2) where you wrote log(2, base = exp(1)). That's okay: you learn more from mistakes than successes. Let's do it one more time.

log(log(2, base = exp(1))) # inner log

I did not expect your call to log() to include base = exp(). You may have included an unnecessary argument, or you may have left out or misspelled an important argument name. Let's try it again.

log(log(2), base = exp(1)) # outer log

I did not expect your call to log() to include base = exp(). You may have included an unnecessary argument, or you may have left out or misspelled an important argument name. Try it again; next time's the charm!

Fix url on top of github repo

Needs to point to gradethis not grader

https://rstudio-education.github.io/gradethis/

unpipe prints out a lot of output

> grader:::unpipe_all(head(who, 2) %>% gather(key, value, new_sp_m014:newrel_f65, na.rm = FALSE))

returns the follow results, which maybe related to #36, but also will explain why learnr also shows this output.

("Afghanistan"("Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan", "Afghanistan", 
    "Afghanistan", "Afghanistan", "Afghanistan"))(iso2 = "AF"("AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", 
    "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF", "AF"), 
    iso3 = "AFG"("AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", 
        "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG", "AFG"), 
    year = 1980L(1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 
        1980L, 1981L, 1980L, 1981L, 1980L, 1981L, 1980L, 1981L), 
    key = "new_sp_m014"("new_sp_m014", "new_sp_m1524", "new_sp_m1524", 
        "new_sp_m2534", "new_sp_m2534", "new_sp_m3544", "new_sp_m3544", 
        "new_sp_m4554", "new_sp_m4554", "new_sp_m5564", "new_sp_m5564", 
        "new_sp_m65", "new_sp_m65", "new_sp_f014", "new_sp_f014", 
        "new_sp_f1524", "new_sp_f1524", "new_sp_f2534", "new_sp_f2534", 
        "new_sp_f3544", "new_sp_f3544", "new_sp_f4554", "new_sp_f4554", 
        "new_sp_f5564", "new_sp_f5564", "new_sp_f65", "new_sp_f65", 
        "new_sn_m014", "new_sn_m014", "new_sn_m1524", "new_sn_m1524", 
        "new_sn_m2534", "new_sn_m2534", "new_sn_m3544", "new_sn_m3544", 
        "new_sn_m4554", "new_sn_m4554", "new_sn_m5564", "new_sn_m5564", 
        "new_sn_m65", "new_sn_m65", "new_sn_f014", "new_sn_f014", 
        "new_sn_f1524", "new_sn_f1524", "new_sn_f2534", "new_sn_f2534", 
        "new_sn_f3544", "new_sn_f3544", "new_sn_f4554", "new_sn_f4554", 
        "new_sn_f5564", "new_sn_f5564", "new_sn_f65", "new_sn_f65", 
        "new_ep_m014", "new_ep_m014", "new_ep_m1524", "new_ep_m1524", 
        "new_ep_m2534", "new_ep_m2534", "new_ep_m3544", "new_ep_m3544", 
        "new_ep_m4554", "new_ep_m4554", "new_ep_m5564", "new_ep_m5564", 
        "new_ep_m65", "new_ep_m65", "new_ep_f014", "new_ep_f014", 
        "new_ep_f1524", "new_ep_f1524", "new_ep_f2534", "new_ep_f2534", 
        "new_ep_f3544", "new_ep_f3544", "new_ep_f4554", "new_ep_f4554", 
        "new_ep_f5564", "new_ep_f5564", "new_ep_f65", "new_ep_f65", 
        "newrel_m014", "newrel_m014", "newrel_m1524", "newrel_m1524",
        "newrel_m2534", "newrel_m2534", "newrel_m3544", "newrel_m3544",
        "newrel_m4554", "newrel_m4554", "newrel_m5564", "newrel_m5564",
        "newrel_m65", "newrel_m65", "newrel_f014", "newrel_f014",
        "newrel_f1524", "newrel_f1524", "newrel_f2534", "newrel_f2534",
        "newrel_f3544", "newrel_f3544", "newrel_f4554", "newrel_f4554",
        "newrel_f5564", "newrel_f5564", "newrel_f65", "newrel_f65"),
    value = NA_integer_(NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_,
        NA_integer_, NA_integer_, NA_integer_))

test_result needs matching api and return value

Fix Grading demo

The original issue from #16 should be fixed, but there were internal API changes (mainly from using the condition object), so we need to fix the grader::grading_demo file (or replace it entirely with a fully-fledged learnr + grader example)

Should check chunks require checking code to be returned last?

It is fine to be forced to be returned last, but it should be documented

pryr calls to rlang?

Hadley brought up using rlang::call_standarise instead of using pryr. Not sure if the code in grader created before rlang, but should probably consider using the newer package

https://twitter.com/hadleywickham/status/1141091795440455686

Glue message for test_result

#57 breaks how messages are created for test_result. also related to #60