ricecomp215-staff / ricechecks Goto Github PK

View Code? Open in Web Editor NEW

11.0 4.0 1.0 752 KB

Gradle-based Java AutoGrader

License: Apache License 2.0

Java 28.25% Kotlin 71.32% Shell 0.44%

java gradle autograder education

ricechecks's People

Contributors

Stargazers

Watchers

Forkers

jpratt-mru

ricechecks's Issues

Set up many more demo repositories

We seem to have the basics working now. Still need to sort out how to do "standalone mode".

Switch from Jackson to Kotlin Serialization?

https://github.com/Kotlin/kotlinx.serialization

Supports JSON out of the box.
3rd party plugin supports YAML.
Unclear about support for XML, which we need.

Deprecated features of Gradle

Currently this:

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.4/userguide/command_line_interface.html#sec:command_line_warnings

The problem seems to be the way we're calling project.javaexec:

gradle.buildFinished { ignoredResult ->
    project.javaexec {
        classpath = configurations.ricechecks
        main = "edu.rice.autograder.AutoGraderKt"
        args = [ "--project", gradeProject,
                 "--config", gradeConfig,
                 "grade" ]
    }
}

Possible solution: finalizedBy:
https://discuss.gradle.org/t/how-to-call-task-execution-from-outside-task/25971

automatic uploads to Sonatype

Here's a script that doesn't seem to do the job:

uploadArchives {
    repositories {
        mavenDeployer {
            beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }

            repository(url: "https://oss.sonatype.org/service/local/staging/deploy/maven2/") {
                authentication(userName: ossrhUsername, password: ossrhPassword)
            }

            snapshotRepository(url: "https://oss.sonatype.org/content/repositories/snapshots/") {
                authentication(userName: ossrhUsername, password: ossrhPassword)
            }

            pom.withXml { fixPomXml(it) }
        }
    }
}

Deleted for now since it was throwing errors on Travis (and I don't want to set up username/password environment variables there.).

If we're going to release versions of RiceChecks on a regular basis, we need to get the uploading process properly automated. Manual upload sucks.

Simplify the annotation tags based on context

Example: if GradeTopic was attached to a class, then any Grade within that class that doesn't name a topic could "inherit" the stated topic and project.

Upsides: cleaner annotations.

Downsides: even more error checking and validation.

We probably want the lambdas' coverage stats to be accumulated alongside their containing classes, rather than being treated as separate classes for coverage purposes. This will require paying attention to the mangled names.

Weird version numbers for google-java-format

Rather than "0.8", instead it's some crazy Docker string (a file directory with a carriage return in it!).

Tentative fix seems to work. Need to roll out a new Jar, etc.

JDK8 support

Created a branch to experiment with this:
https://github.com/RiceComp215-Staff/RiceChecks/tree/jdk8-support

Changed all the examples to compile against Java8
- Updated build.gradle files (notably, switching to a much older error-prone plugin)
- Got rid of all the var declarations
- List.of(...) to Arrays.asList(...)
Changed the core Kotlin code to use older Java8 APIs
- Some APIs are too new, all easily downgraded

It's going to be all-JDK8 or all-JDK11. No easy way to mix them. Among other issues, we need to make sure all the generated libraries use the "version 53" classfiles, otherwise this happens in a JDK8 project trying to read JDK11 classfiles:

> Task :standaloneSort:compileJava FAILED
/Users/dwallach/IdeaProjects/RiceChecks/standaloneSort/src/main/java/edu/rice/sort/InsertionSort.java:9: error: cannot access GradeCoverage
import edu.rice.autograder.annotations.GradeCoverage;
                                      ^
  bad class file: /Users/dwallach/.gradle/caches/modules-2/files-2.1/edu.rice.ricechecks/ricechecks-annotations/0.5/780f06a5a0184415e1eab7e3ca34756690d8ee75/ricechecks-annotations-0.5.jar(/edu/rice/autograder/annotations/GradeCoverage.class)
    class file has wrong version 55.0, should be 53.0

Does the annotations. https://github.com/education/autograding/blob/master/src/output.ts
Tests the annotations. https://github.com/education/autograding/blob/master/src/__tests__/output.test.ts

Relevant GitHub APIs:

https://developer.github.com/v3/checks/runs/

Prior code that I wrote that works with the check-suites data:

https://github.com/danwallach/github-classroom-utils/blob/master/github_completion_times.py

Implementation strategy?

The main RiceChecks doesn't need to change, since it should run as-is on a student machine or in a cloud CI. Grading output in JSON format is already written to the filesystem. That will stick around.
A new RiceChecks tool would be run from GitHub Actions that would read the grading JSON and would manipulate annotations on the commit.
Kinda unclear: maybe we want to set up a "check run server" to run the unit tests on every commit and configure RiceChecks to run as a check suite?
- https://developer.github.com/apps/quickstart-guides/creating-ci-tests-with-the-checks-api

IntelliJ test-runner gets confused if the Java compiler errors

ClassNotFoundError, etc, rather than stopping.

tweak build.gradle to convert ErrorProne errors into warnings
tweak build.gradle to make builds fail correctly
- but somehow the autograder still gets to run?

Set up RiceChecks demonstration for anybody to play with

Create GitHub Classroom with three example projects (sort, rpn, regex)
Make sure Travis works (shouldn't be a problem for public repos?)
Make sure that GitPod.io works properly, which makes for great demo
- Needs a Dockerfile specifying JDK11
- Terminal command works: sdk use java 11.0.2-open
- Setting Dockerfile doesn't effect IDE: gitpod-io/gitpod#356
- .gitpod.yml: https://www.gitpod.io/docs/41_Config_Gitpod_File/
- Docker image: https://www.gitpod.io/docs/42_Config_Docker/

Currently no Kotlin support: gitpod-io/gitpod#83