texadactyl/jacotest.go

User-level Testing Companion to Jacobin

jacotest.go's Issues

Display go version

During showHelp, jacotest should show the version of go used to build the executable.

Windows only sometimes sees "*** ERROR, runner: cmd.run(jacobin main.class) returned"

This requires some research using a nix VM to install Windows.

Create installation instructions

and test them!

Retry on SQL INSERT duplicate key errors

sqlite3.go is affected:

• Wait 1 second.
• Get a new UTC date and time.
• Rebuild the SQL command-text.
• Rerun the SQL command.
• If it fails a 2nd time, that's fatal.

Note 1st time failure as a nonfatal error.
Note recovery in 2nd attempt as successful.

Adjust for multiple report files

Now that report files are date and time stamped, its getting a bit messy to have them all resident at the highest-level directory. So, a reports subdirectory was created; all reports will be resident therein.

Something is rotten in github actions w.r.t. time zone

Test case zippy uses a zip file created on an Ubuntu Linux system such that all the entries show precisely a last modification timestamp of 2023-05-06T20:37:34Z (15:37:34 USA CT). This was set up prior to zipping with a recursive touch using find.

When I run zippy on my home Ubuntu system, all is well.

However, when I run zippy under Github Actions, I saw messages like this:
ERROR :: For name 0.main.log, expected stamp = 2023-05-06T20:37:34Z but observed 2023-05-06T15:37:34Z

Observed "15:37:34Z" ??? Yes, if one could time travel back 5 hours.

Switching to using a CRC methodology in place of the time stamp:

• avoid the time zone dilemma
• CRC is computed over the entire uncompressed zip file entry

Test JACOBIN-0263-gc gives intermittent error on Windows 10

When running test JACOBIN-0263-gc, I get an intermittent error on Windows 10 running HotSpot from JDK17 (Termurin). Attached image shows the output.

If I had to guess, it might be because the total elapsed time is zero--although the error occurring in <init>, which is the constructor, makes me most uncertain about my diagnosis.

jacotest database query for the "Last Two" comparison

An important query is to compare the last 2 test case history items. Thus, it is important to query with date and time descending.

Add Github Actions workflow

use docker to execute

O/S: ubuntu_latest

O/S dependencies:

• openjdk-17-jdk

Other dependencies:

• golang 1.19

Question about RUN_REPORTS*.md

@platypusguy

Are the RUN_REPORTS*.md files still useful?
I ask because I never use them.

Show which version of Java is in use

The subject line says it all!

Issues raised by Om Andrew

Om Richard,

I finally got some time off from all this excess vacationing and spent it happily getting into Jacotest.

Above all, thank you for this original test suite that I can run without blowing up my GitHub Actions and thereby incurring on the project home page, the dreaded, red "Failed to run properly" badge.

Here are initial thoughts and suggestions. I thought I'd send them in an email and you can break them out into tasks on GitHub if you want or if you prefer on a free instance of YouTrack. I like the latter more than GitHub b/c we can speak freely about deficiencies without making our project look deficient. Anyway, please take them in exactly the same spirit as your many thoughtful comments pointing out deficiencies and possible improvements in Jacobin.

Simple stuff

In the run reports:

• In markdown <br> is not a valid tag, so those tags in the run report look strange.
• I would make the test case field fixed length, say 20 chars wide. We can always make sure that tests are distinguishable within the first 20 chars. Having it be fixed width, it's easy to scan the report for failures.
• In the report as currently generated with -j java, after the line for negtest-runner-timeout, I get the following line:
  ||| |
• Finally, I would suggest that the non-Jacobin run, not be called java, as both JVMs are in theory executing Java. I think OPENJDK or just JDK are more descriptive.

Failed on every test

• When I run Jacotest against Jacobin, I get "FAILED" on every test. That appears to be b/c it can't find the Jacobin executable. I would suggest testing for the Jacobin executable before the test run. If it can't be found, then say that in the report and abort the run. There's no point running the tests after that.
• When Jacotest can't find Jacobin, it generates an empty log file for the failed test. Which I think is another reason to avoid running the tests at all if the executable can't be found.

Running Jacotest

If I copy jacobin.exe to the main Jacotest directory and juse -j jacobin on the command line, the tests run, but every Executing statement is followed by:

*** ERROR, runnder: cmd.run(jacobin main.class) returned:

Once again, all the log files are empty, so I can't tell what's wrong.

---

Again, thank you for Jacotest. I look forward to using it regularly, once I can get it to run on my system.

Feel free to move these comments to a different forum and we can continue the dialog there, or we can carry on via email--as your preference dictates.

The use of "java" as a JVM name is misleading

"I would suggest that the non-Jacobin run, not be called java, as both JVMs are in theory executing Java. I think OPENJDK or just JDK are more descriptive."

Amen to this too. I was lazy in simply using the JDK executable file name.

Need a run report

My proposed low-cost approach is to output a github markdown file (.md), suitable for viewing on the github site.

Must test for the existence of OpenJDK or Jacobin executable

"When I run Jacotest against Jacobin, I get "FAILED" on every test. That appears to be b/c it can't find the Jacobin executable. I would suggest testing for the Jacobin executable before the test run. If it can't be found, then say that in the report and abort the run. There's no point running the tests after that."

Amen.