release-candidate/vscode-ocaml-expect-inline

This extension lets you run OCaml Expect-tests and inline tests using the native Test Explorer

OCaml Expect and Inline Test Explorer for Visual Studio Code

This extension lets you run OCaml PPX Expect and PPX Inline Test with the (native) Test Explorer UI.

• Features and drawbacks
  • Drawbacks
• Getting started
  • Dependencies
  • Installation
  • Q & A
    • Q: What do the groups in the Test Explorer view mean?
    • Q: How can I (re-) discover all tests?
    • Q: Where can I see the output of the test run(s)?
    • Q: How can I change which test extension's tests are run by the Run Tests button in the upper right of the Test Explorer?
    • Q: What does the red circle with a dot in the middle mean?
    • Q: Where can I see the log of the extension?
    • Q: Why does the extension not work when using Dune in watch-mode dune -w | --watch or dune ... --passive-watch?
• Configuration
• Changes
• Contributing
• License

Features and drawbacks

• uses dune to compile and run the tests
• support for expect PPX tests and inline PPX tests
• filtering of tests by name
• parses the test list output of the test runners to fill the Test Explorer view: faster than grepping every source file for test cases and the test tree view is consistent with the test runners
• retries running dune if another instance has locked the project until dune can acquire the lock or a timeout occurred
• Uses VS Code's native Test Explorer (no additional extension needed)

Drawbacks

• needs dune
• when running tests, every test is run on its own, sequentially
• Uses VS Code's native Test Explorer UI

Getting started

Dependencies

• Visual Studio Code version 1.65 (February 2022) or higher
• PPX Expect or PPX Inline Test
• Dune: the extension uses Dune to build and run the test runners.

Attention: you must be in a trusted workspace. Tests (test runners) can execute arbitrary code, so you do not want to run them in untrusted directories!

Installation

Either

• install the extension directly from the Visual Studio Code Marketplace Expect and Inline Tests
• install the extension directly from the Open VSX Registry Expect and Inline Tests
• or download the extension from the latest release at GitHub
• or build the extension yourself by cloning the GitHub Repository and running yarn install and yarn package in the root directory of the cloned repo.

Q & A

What do the groups in the Test Explorer view mean?

How can I (re-) discover all tests?

Where can I see the output of the test run(s)?

How can I change which test extension's tests are run by the Run Tests button in the upper right of the Test Explorer?

What does the red circle with a dot in the middle mean?

Where can I see the log of the extension?

Q: Why does the extension not work when using Dune in watch-mode dune -w | --watch or dune ... --passive-watch?

Q: What do the groups in the Test Explorer view mean?

A: Every workspace folder in the current project has it's own node, Workspace: WORKSPACE_NAME. If the project is a single workspace, only one of these exists. Below the workspace node the test cases are grouped by filename. If you are using another Test Explorer extension, like Alocotest Test Explorer, the workspace nodes are contained in a root node of each testing extension - Expect and Inline PPX Tests (Alcotest tests in Alcotest Tests)

Q: How can I (re-) discover all tests?

A: Push the Refresh Tests button in the upper right of the Test Explorer view.

Q: Where can I see the output of the test run(s)?

A: You can either click the Show Output button in the upper right corner of the Test Explorer to see the output in a new terminal window, click on Go To Test to the right of a failed test in the test explorer and then Peek Error or Peek Test Output or hover over the Error Lens output in the source file - this preview is too narrow, so the test output is mangled.

Q: How can I change which test extension's tests are run by the Run Tests button in the upper right of the Test Explorer?

A: Click the down arrow to the right of the Run Tests button, there you can set the profiles using Select Default Profile(s).

Q: What does the red circle with a dot in the middle mean?

A: That means that dune returned an error (not a failed test). Mostly because of code that does not compile.

Q: Where can I see the log of the extension?

A: In the OUTPUT tab of the Panel, you have to select the extension named Expect and Inline Tests in the upper right drop-down menu.

Q: Why does the extension not work when using Dune in watch-mode dune -w | --watch or dune ... --passive-watch?

A: Dune uses a file lock (default path: _build/.lock) to coordinate multiple Dune processes. Dune in watch mode does not release the file lock at all, so no other Dune process can run at the same time. The meaning of the two buttons in the error message is:

• Cancel - stop the running Dune process: cancels running Dune, but the extension is not going to work until the other Dune process releases the lock.
• Retry - retry running Dune: Either stop the Dune process in watch mode or wait until all other Dune processes have finished their work. If no other Dune process is running, click Retry.

For a future version of Dune the plan is to be able to use the Dune process in watch mode with another dune running e.g. dune [rpc] exec and get the output on the 'client' Dune process.

Configuration

• expectppx.dunePath - Set an absolute path or a path relative to the project root of the Dune executable. Default: dune - use the one in the local Opam environment or in PATH.
• expectppx.excludeRunners - A list of inline test runner names to be excluded from test discovery an startup or refresh, e.g. because they take too long to finish. E.g. ["inline_test_runner_fsevents_tests.exe"] to exclude the test runner of the fsevents_tests library.
• expectppx.discoverOnStartup - Boolean. Set this to false if you do not want to run or compile all expect and inline test runners on startup to discover tests. If you want to rediscover all test by running all inline test runners, use the Refresh Tests button in the upper right corner of the Test Explorer.
• expectppx.discoverInSources - Boolean. Whether to parse source files on open and save for tests and update the Test Explorer tree. Should be set to true if expectppx.discoverOnStartup is false.

Changes

See file CHANGELOG.md.

Contributing

See file CONTRIBUTING.md

License

OCaml Expect and Inline Test Explorer for Visual Studio Code is licensed under MIT license. See file LICENSE

vscode-ocaml-expect-inline's Issues

Flicking when running all tests

Hi,

Running the tests in the dune repo for example doesn't quite work:

Screencast.from.2023-03-18.23-24-16.webm

There appears to be a lot of flickering and it isn't obvious what is actually happening.

For reference, I have the OCaml platform extension also installed.

It would be nice if this extension used dune rpc also so that it can connect to a dune already in watch mode. At the moment, Dune disallows multiple dune processes in the same project due to one of them acquiring a lock.

feature request: show diffs in errors messages

When we have diffs in error messages they are simply rendered as plain text:

It would be nice if they were rendered diffs.

Cannot cancel tests when waiting for Dune lock

When the extensions are waiting for Dune to run the tests, it appears that they are waiting for the lock to be released. However when pressing cancel the tests are not cancelled.

Request test builds from running Dune build in watch mode

When there is a dune build in watch mode, the extension should instead do an rpc request to dune rather than run its own dune.

The VSCode's Testing tab does not show anything

Describe the bug
The VSCode's Testing tab does not show anything

My dune project's directory has this directory hierarchy (.ml files are omitted):

ProjectName
- dune-project
- ProjectName.opam
- bin
  - dune
- lib
  - dune
- test <- this is just a dummy directory, containing nothing
  - dune

I think dune is correctly configured because my VSCode can successfully typecheck variables and highlight compilation errors from .ml files.

To Reproduce
Steps to reproduce the behavior:

1. Install the plugin and wait
2. Click the 'Testing' tab on the left

Expected behavior
A list of tests are shown

Screenshots

Logs

If applicable, add logs of the extension (Output of Expect and Inline Tests), the Extension Host (Ouput of Extension Host) and Window (Output of Window).

I would like to report, but where can I find these?

Example OCaml project

Environment (please complete the following information):

• Version of the extension: v0.2.0
• Version of VS Code: 1.76.1
• OS: [e.g.Linux] WSL2, Ubuntu

Additional context