Obtaining assessment data (pass/fail/trials)

[rstudio]

Obtaining assessment data (pass/fail/trials)

[jjallaire]

Yes, what you are describing is definitely a scenario want to support well. If you make use of an Exercise Checker (https://rstudio.github.io/tutor/exercises.html#exercise_checking) then the results of the check are included in the call to the event recorder.

[dtkaplan]

#coatless One prototype exercise checker is the checkr package, available via

devtools::install_github("dtkaplan/checkr")

There's a vignette with the package that gives a quick overview and examples.

The newest version of checkr contains an (optional) login/verification component, so that you can have a login even you you are running the tutorial on a generic shiny server.

Shinyapps.io is great, but you need the "standard" account level which at $99/mo would cause many instructors pain. The login component I implemented uses Google spreadsheets, so just about any instructor can set up and maintain the accounts and access the submissions.

I wrote checkr with a focus on checking R code, but this can be expanded to cover other modes of submission, for example, entering equations or fill-in-the-blanks. I'd be interested to hear what modes you would be interested in having checked.

[coatless]

@jjallaire: Excellent! I'll try to throw something together.

@dtkaplan: Thanks for the information, I'll definitely look into checkr. From the exercise-template vignette, it seems like each problem could be isolate to its own .Rmd file and then be combined so that each student could have a "random" ordering of questions?

Presently, I'm able to use an education-licensed Shiny Server Pro. So, I'll likely study how results are sent to google spreadsheets and perhaps make a modification to stream the results into a database.

Regarding your assessment types question, I'm looking at allowing wide berth of assessments from allowing students to:

• symbolic equations (input fields transformed with mathquill)
• matching
• explicitly mark points on a graph (think identify(x, y))
• capturing different student models

The last of which I've worked on a bit extensively on over the past year or so (https://github.com/coatless/stat429-fa15-autograder).

[dtkaplan]

@coatless Putting each exercise or question into it's own Rmd is working very well for me. I do this using the knitr child system. Among other things, it let's me debug individual problems (which relates mainly to getting checkr to give a useful response to different kinds of errors.