adrienjoly/js-test

💯 Exerciseur / outils d'évaluation d'étudiants -- NOT MAINTAINED ANYMORE

⚠ This project is not maintained anymore. Feel free to fork and make it your own ^^

JS-TEST

A toolkit to help JavaScript teachers evaluate their students' skills more efficiently.

Alternative short description: a file-based way to produce, publish, collect and grade student assessments.

In the frame of my JavaScript course at EEMI, I developed an online exam application for students, using Google Polymer and Firebase. It's designed to display different variants of each exercise to prevent fraud, and automatic student evaluation/grading, for both quizzes and coding exercises.

It's made of:

• a simple Markdown-based language to define quizz and coding exercises;
• a web app that students will use to give their answers;
• scripts to deploy, evaluate, collect (e.g. from Google Forms) and grade students' answers automatically.

🇫🇷 Annonce de sortie en Français: Un exerciseur en ligne + script pour corriger et noter le code de vos étudiants

▶️ Demo video: (click to play)

▶️ Interactive demo: (click to try)

Features

• 100% paperless examination/evaluation/training software, 100% online;
• designed for automated testing and evaluation of students' answers;
• exams/quizzes/exercises defined as Markdown files;
• variants based on student id;
• students identified using their Google Apps account;
• answers submitted to a Firebase account, can be exported to a JSON file;
• students scores can be evaluated in real time, from a web dashboard;
• beautiful UI thanks to Google Polymer and Material Design.

Use cases

I've been using js-test:

• to measure the understanding of my students; (quizz at the end of a class)
• to grade my students; (exams)
• to help my students self-assess and train their skills individually.

Setup: install js-test

Run the following steps in a bash-compatible terminal:

$ git clone https://github.com/adrienjoly/js-test.git && cd js-test
$ nvm use # to use the right node.js version
$ npm install # to install dependencies
$ npm run create # to generate sample exercises and configuration to `/exam-data`
$ vi /exam-data/ex.01.quizz.template.md # to edit quizz questions
$ vi /exam-data/ex.02.code.template.md # to edit coding exercise(s)

Create an assessment and test it locally

Video walkthrough: How I create automated student exams using Markdown - YouTube

After editing the exercise files (see above), run the following commands:

$ npm run build # to compile exercise data into `/public/scripts/exam-data.js`
$ npm start # to run the web server, until you press ctrl-c

Then, open localhost:8080 to test the assessment.

Read how to write assessments (a.k.a. exam templates) and examples to learn how to write quizzes and coding exercises.

Deploy an assessment to Firebase

After editing and testing the exercise files locally (see above):

1. Create a new project on Firebase
2. Click on "Add Firebase to your web app"
3. Copy the values of the given apiKey, databaseURL and messagingSenderId properties to the FIREBASE_CONFIG object of your /exam-data/exam-config.js file (i.e. overwrite existing values)
4. Still in your /exam-data/exam-config.js file:
   • set backend.type to firebase,
   • un-comment the FIREBASE_CONFIG object.
5. Run npm run build to re-compile your configuration into /public/scripts/exam-data.js
6. Run npm run deploy-firebase (or npm run deploy-firebase-instances) to push the project to production on Firebase Hosting
7. Test your Firebase project from the given URL, by simulating a student's submission

ℹ️️ After disabling PUBLIC_TEST_MODE in /exam-data/exam-config.js, don't forget to toggle the active property (to 1 or 0) of your Firebase database, to (de-)activate student access to the assessment.

⚠️ Security notice: You may want to protect your Firebase database to prevent accidental or malicious data alteration/loss, as explained in /public/scripts/app-firebase.js.

Evaluate and grade answers submitted to Firebase

At all times, you can:

• Go to the "Database" section of your Firebase project's dashboard to see the submitted answers
• Export the submitted data as a JSON file, for backup and/or offline evaluation and grading

After having run the steps above, you can either:

• Run npm run eval (npm run eval-instances) to compute the scores of answers stored in your Firebase database;
• Or npm run eval-firebase-dumps to do it offline, from a JSON export of your Firebase database.

In both cases, resulting grades will be stored in the /exam-data/scores.csv file.

Evaluate and grade answers submitted to Google Forms

As an alternative to using Firebase as a back-end, it's possible to ask students to submit their answers by email, or to Google Forms. This alternative has been useful with schools which don't provide Google-based authentification for their students.

Phase 1 - Setup and deploy the assessment website

1. In the /exam-data/exam-config.js file:
   • set PUBLIC_TEST_MODE to true,
   • set backend.type to 'email-submit',
   • set backend.EMAIL_SUBMIT_CONFIG to readfile('public/data/submitted.md'),
   • un-comment the FIREBASE_CONFIG object.
2. In the /public/data/intro.md file, write a message to let students know that refreshing (or changing) the page will cause them to loose their answers.
3. In the /public/data/submitted.md file:
   • include a HTML link to a Google Forms where your students will paste their answers in JSON format, with the target="_blank" attribute, to prevent them from loosing their answers during that process.
   • also include <pre>{{hashedAnswers}}</pre> to display the JSON code in question.
4. Run npm run build then commit the pending changes to the git repository that holds your js-test copy and your exercises.
5. Run npm run deploy-firebase, as specified in the procedure above. (i.e. you will need to setup a Firebase app first)

Phase 2 - Collect answers from students

1. At the time of the exam, provide the resulting URL to students, so they can fill their answers.
2. At the end of the exam, go to the "answers" tab of the Google Forms, click the icon to save them into a Google Spreadsheet, then open that spreadsheet.
3. Export the spreadsheet as a csv file, then use https://www.csvjson.com/csv2json to convert it into a JSON file. Download that file in your js-test directory, and rename it as student-submissions.csv.json.
4. Run node src/extract-student-submissions-from-spreadsheet-json.js ./student-submissions.csv.json => this creates a students directory that contains one JSON file per student submission.

Phase 3 - Evaluate and grade student answers

1. Run npm run eval-student-submissions to evaluate and grade these student submissions and store their results in exam-data/email-submission.

   If you see any error during this process, it may mean that your testing code needs some fixes. Student mistakes and runtime errors are reported in their own file, not in stdout.

2. When you're done, send these result files back to students, so they can read the feedback and grade of their work.

Checkout this YouTube playlist for examples.

Enable student identification

ℹ️️ These instructions are not complete yet.

• Change Google's client ID (from /exam-data/exam-config.js) so that students can identify using your own google apps domain name (or localhost),
• build and deploy again to your assessment's hosting account. (e.g. Firebase or Heroku)

Roadmap

• Add variants to Quizzes.
• Allow per-variable variants. => more combinations.
• Add access rules. (per student group and timeslot)
• read more: ROADMAP.md, Kanban board and issues

js-test's Issues

Prevent the on-tap event from firing when clicking outside of the choices.

is it still a problem?

Generate one shared Google Doc per evaluated student sheet, so that they can comment on it

make sure that exam data files (js and md) are loaded in their latest version => avoid caching

explanation: we want to make sure that the students always see the most recent update of their exam!

Import issues from https://github.com/adrienjoly/js-exam/issues

Prevent students from connecting an exam URL 1 hour after activation

to avoid next students to see the test before their timeslot, even if other students finish early and show them the URL

Feature: support markdown formatting in quizz answers

Render a distribution chart to see number of students per range of grade

Create a new front-end / UI

Objectives:

• compatibility => any student should be able to use it, whatever their OS, browser and privacy settings
• robustness => let's avoid relying on 2-way binding, polyfills and other technical tricks that may cause bugs
• testability => build it in a way that makes it fast to test, so we can enable continuous integration
• modularity => it should allow to plug-in different back-ends and/or UI elements, e.g. to enable self-assessment with granular feedback
• simple deployment => before adding any plug-ins, the UI should be a static webapp, so we can deploying easily and quickly to a wide range of hosting platforms, including free ones. (e.g. github pages, netlify, now.sh, firebase...)
• foster contributions => use a tech stack that is popular, to make it motivating and accessible for other developers to contribute

Also, we should probably create a separate repo for the UI, and focus this one on the writing of exercises + the collection and evaluation of student submissions.

See the list of related issues, below, including closed ones. (consider this issues as an epic that should cover all these requirements)

display runtime jailed/eval errors in terminal while `npm run eval-firebase-dumps`

as of today, runtime jailed/eval errors are displayed that way in stderr:

Tue, 27 Mar 2018 17:00:21 GMT jailed:sandbox DYNAMIC PLUGIN:95
  const canonicalLines = studentFct.toString()
                                   ^

TypeError: Cannot read property 'toString' of undefined
    at evaluateStudentCode (DYNAMIC PLUGIN:95:36)
    at DYNAMIC PLUGIN:128:3
    at ContextifyScript.Script.runInContext (vm.js:35:29)
    at ContextifyScript.Script.runInNewContext (vm.js:41:15)
    at Object.runInNewContext (vm.js:93:38)
    at executeJailed (/Users/adrienjoly/Dev/eemi/js-test-eemi-2017-2018/node_modules/jailed-node/sandbox/sandbox.js:151:8)
    at execute (/Users/adrienjoly/Dev/eemi/js-test-eemi-2017-2018/node_modules/jailed-node/sandbox/sandbox.js:165:3)
    at process.<anonymous> (/Users/adrienjoly/Dev/eemi/js-test-eemi-2017-2018/node_modules/jailed-node/sandbox/sandbox.js:53:7)
    at emitTwo (events.js:106:13)
    at process.emit (events.js:191:7)

...and are merged with stdout before being logged in eval.log files, because I did not find a proper way to:

• provide useful context (i.e. student code) when errors happen
• and display just these errors in the terminal while running npm run eval-firebase-dumps

create a validation script that runs code evaluation tests on provided solutions, to make sure they work

• without variants
• then for each variant!

Feature: script to detect students who copied from another variant (plagiarism)

WIP: https://github.com/adrienjoly/js-test/blob/master/src/checkGroupFile.js

Allow per-variable variants. => more combinations

Evaluate code style/quality automatically (e.g. with eslint)

That would make evaluation scripts less verbose, and save some time to exam writers.

create separate backend scripts: realtime-eval and auto-eval (that evals only on submit)

Setup automatic versionning, based on conventional commit messages

Like in openwhyd/openwhyd#216

Provide a way for students to read their copy (questions and answers) after submission

pour prochain partiel: trouver un moyen pour que les etudiants puissent discuter des questions / réponses après l'examen. => copie de la copie après rendu?

Code-Eval: Find a way to display the position of the error in the student's code

cf 8c43286

Offer to deploy static version to netlify, now.sh or github pages

Because Heroku can be overkill.

BUG: points for a correct code answer were only counted in incognito mode

depuis https://js-revisions.herokuapp.com/ (commit 538cf29)

Le 10 janvier 2017 à 12:20, Pierry LIM [email protected] a écrit :

Bonjour,

Le site ne m'a pas compté les points sur le code javascript

Par exemple sur la question 16:

Mon code:

if (nombre === 4){
resultat = 'egal';
}

et la solution:

if (nombre === 4) {
resultat = 'egal';
}

Ou comme dans la question 17:
Mon code:

var reponse = prompt('Ta couleur préférée ?');
if (reponse === 'bleu') {
  alert('OK');
} else if (reponse === 'gris') {
  var reponse2 = prompt('clair ou foncé ?');
    if (reponse === 'clair') {
      alert('comme le ciel');
    } else if (reponse2 === 'foncé') {
      alert('c\'est bien !');
    }
} else {
  alert('je connais pas');
}

et la correction:

var reponse = prompt('Ta couleur préférée ?');
if (reponse === 'bleu') {
  alert('OK');
} else if (reponse === 'gris') {
  var reponse2 = prompt('clair ou foncé ?');
  if (reponse2 === 'clair') {
    alert('comme le ciel');
  } else if (reponse2 === 'foncé') {
    alert('c'est bien!');
  } 
} else {
  alert('je connais pas');
}

Je pense que le problème vient du site.

Je viens de tester en navigation privée et ça marche.

Adrien:

As tu vu qu'il fallait cliquer sur une autre question et attendre un peu pour que les points soient comptés?

Pierry:

Oui j'ai bien pris soin d'attendre que ça se corrige mais le code valait 0 point sur chrome.

En navigation privée, j'ai copié le même code mis à la question 16, j'ai eu les 3 points.

Support variants in Quizzes

BUG: realtime-eval gives zero points to question, except in incognito mode

experienced by Pierry LIM, more than once.

mention in UI that 3rd-party cookies must be enabled

(otherwise, nothing happens after google's login pop-up closes)

Feature: display eval results and logs in the front-end, to help student understand what's wrong

The autoeval backend is already able to run evaluation tests in the browser and to display resulting grades next to each quest.

But it would be nice to also display the log of the evaluation process, with feedback.

Enhancement: move intro.md and rendu.md to /exam-data/, and compile their content into /public/scripts/exam-data.js

so that all the exam-specific files (content) are stored in /exam-data, and compiled to exam-data.js (=> consistency)

Share your eslint config

Hi Adrien,

You should share your eslint config, as this, dev is standardized

move test-specific title, firebase credentials and settings in one external file

use https://github.com/convoo/login-fire to login (instead of google login)

... because google login requires third-party cookies and fails silently. (cf other issues)

Feature: store student replies in localstorage, when using auto-evaluation or email-submission mode

... so that they can come back to their copy later, and still see their answers.

=> then, we should provide a way to clear the copy, so that students can start an exam over again.

Provide a way for teachers to make sure that a student has submitted their copy before leaving the room in advance

pour prochain partiel: demander aux etudiants de prouver que le rendu a été fait avant de quitter la salle (pour pas qu'ils rendent dehors, à côté des potes)

Bug: also apply variants on solutions

Idea: allow anonymous student submissions => get rid of front-end

Idea expressed in French:

Je suis carrément ouvert à expérimenter une interface de saisie 100% anonyme (c.a.d. sans login google), à condition que je puisse continuer de faire corriger le travail rendu de manière automatique et de générer des variantes de sujet pour chaque étudiant (ex: en fonction de son numéro anonyme), pour détecter le copiage.

Je me suis dit qu'on pourrait s'y prendre comme ça par exemple:

• on me fournit en avance la liste des numéros qui seront utilisés pour le rendu anonyme de chaque étudiant
• je génère un sujet en PDF pour chacun de ces étudiants (les variantes sont calculées par modulo sur le numéro associé à chaque copie)
• le jour J: on imprime et distribue les sujets PDF => chaque étudiant récupère le sujet associé à son numéro de rendu anonyme. il saisit son numéro de rendu anonyme + ses réponses dans un Google Forms générique
• après l'examen, j'exporte les réponses du google forms au format spreadsheet => ma moulinette sait à quel numéro étudiant est associé chaque réponse, et donc quel était la variante de cet étudiant (pour séléctionner les bons paramètres de correction auto)

Optimisation: Bundle the web app into one JS file

js test: bundler l'app avec https://github.com/Polymer/polymer-bundler

BUG: js-code.herokuapp.com stays inactive on safari

Store students' email address + provide script to send them their eval log

combine bower dependencies in 1 js file, to allow 25+ students to access firebase hosting at the same time

Provide a fallback for students which browser have high privacy settings

Idée js-test: pour éviter problèmes de connexion (cookies, local storage désactivés), envoyer lien personnel à chaque étudiant => attention, s'assurer que seul l'étudiant puisse ouvrir ce lien, en particulier si les réponses sont persistees dans le cloud

BUG: sometimes, the submit button does not lead to the submission page

the page keeps loading (chrome's loading favicon) and the CPU heats. infinite loop?

Provide a way for teacher to add/remove points, and take them into account in the total grade

dashboard: sort students per score

BUG: Page cannot be printed

that would be handy to students, for keeping their answers, and solutions

Share your editor config

Hi Adrien,

You should share your editor config file, as this, dev is standardized

Feature: disable google login button if google auth config was not provided

This would reduce clutter of the exam's web UI, for exams that were setup to be accessible publicly.

Feature: prevent page freeze on user-submitted infinite loops

dashboard: only evaluate score of student who made an update, don't refresh everyone on every update!

this would improve performance and reduce consumption of resources.

Web interface to evalute students scores in real time

• for quizzes (no variants, no code evaluation)
• for code exercises (variants + code evaluation)

BUG: cannot login if localstorage / sessionstorage (privacy) was disabled on Chrome

identified from https://js-code.herokuapp.com (js qcm 5) by alexandre blanc.

cause:

when localstorage was disabled by the user on Chrome, the google-signin element fails to load:

644096210-idpiframe.js:12 Uncaught DOMException: Failed to read the 'sessionStorage' property from 'Window': Access is denied for this document.(…)

same problem cited there:

• Google Sign-in doesn't do anything when I block 3rd party cookies => "gapi.auth2 requires third-party cookies to be allowed" (source, nov 2016)
• Add Google Sign-in without using iframe, no solution given
• Google Sign-in does nothing when 3rd party cookies are disabled => solution: detect if third party cookies are enabled
• Client doesn't work if third-party session storage is disabled
• similar: Google signin not working in safari private mode => solution: detect storage capability and warn user

solutions:

1. re-enable localstorage on Chrome: http://stackoverflow.com/questions/24456891/iframe-in-chrome-error-uncaught-securityerror-failed-to-read-the-sessionstora

2. add accounts.google.com to exception list: GoogleWebComponents/google-signin#99 (instructions en français)

BUG: Line breaks are ignored, in quizz solutions

'use strict' prevents student function from being tested

Feature: add a Google Spreadsheets back-end

This would be easier to use for teacher than Firebase.

BUG: `active` does not seem to be reactive on Safari (a refresh was needed)