randycoulman/mix_test_interactive

Interactive watch mode for Elixir's mix test. https://hexdocs.pm/mix_test_interactive/

mix test.interactive

mix test.interactive is an interactive test runner for ExUnit tests.

Based on Louis Pilfold's wonderful mix-test.watch and inspired by Jest's interactive watch mode, mix test.interactive allows you to dynamically change which tests should be run with a few keystrokes.

It allows you to easily switch between running all tests, stale tests, or failed tests. Or, you can run only the tests whose filenames contain a substring. Includes an optional "watch mode" which runs tests after every file change.

Installation

The package can be installed by adding mix_test_interactive to your list of dependencies in mix.exs:

def deps do
  [
    {:mix_test_interactive, "~> 2.0", only: :dev, runtime: false}
  ]
end

Usage

Run the mix task:

mix test.interactive

Your tests will run immediately (and every time a file changes).

If you don't want tests to run automatically when files change, you can start mix test.interactive with the --no-watch flag:

mix test.interactive --no-watch

After the tests run, you can use the interactive mode to change which tests will run.

Use the p command to run only test files that match one or more provided patterns. A pattern is the project-root-relative path to a test file (with or without a line number specification) or a string that matches a portion of full pathname. e.g. test/my_project/my_test.exs, test/my_project/my_test.exs:12:24 or my.

If any pattern contains a line number specification, all patterns are passed directly to mix test.

p pattern1 pattern2

Use the s command to run only test files that reference modules that have changed since the last run (equivalent to the --stale option of mix test).

Use the f command to run only tests that failed on the last run (equivalent to the --failed option of mix test).

Use the a command to run all tests.

Use the w command to turn file-watching mode on or off.

Use the Enter key to re-run the current set of tests without requiring a file change.

Use the q command, or press Ctrl-D to exit the program.

Passing Arguments To Tasks

Any command line arguments passed to the mix test.interactive task will be passed through to the task being run, along with any arguments added by interactive mode. If I want to see detailed trace information for my tests, I can run:

mix test.interactive --trace

mix test.interactive will detect the --stale and --failed flags and use those as initial settings in interactive mode. You can then toggle those flags on and off as needed. It will also detect any filename or pattern arguments and use those as initial settings. However, it does not detect any filenames passed with --include or --only. Note that if you specify a pattern on the command-line, mix test.interactive will find all test files matching that pattern and pass those to mix test as if you had used the p command.

Running A Different Mix Task

By default, mix test.interactive runs mix test. Through the mix config it is possible to run a different mix task. mix test.interactive assumes that this alternative task accepts the same command-line arguments as mix test.

# config/config.exs
use Mix.Config

if Mix.env == :dev do
  config :mix_test_interactive,
    task: "custom_test_task"
end

The task is run with MIX_ENV set to test.

Clearing The Console Before Each Run

If you want mix test.interactive to clear the console before each run, you can enable this option in your config/dev.exs as follows:

# config/config.exs
use Mix.Config

if Mix.env == :dev do
  config :mix_test_interactive,
    clear: true
end

Excluding files or directories

To ignore changes from specific files or directories add exclude: regexp patterns to your config in mix.exs:

# config/config.exs
use Mix.Config

if Mix.env == :dev do
  config :mix_test_interactive,
    exclude: [~r/db_migration\/.*/,
              ~r/useless_.*\.exs/]
end

The default is exclude: [~r/\.#/, ~r{priv/repo/migrations}].

Compatibility Notes

On Linux you may need to install inotify-tools.

Desktop Notifications

You can enable desktop notifications with ex_unit_notifier.

Acknowledgements

This project started as a clone of the wonderful mix-test.watch project, which I've used and loved for years. I've added the interactive mode features to the existing feature set.

The idea for having an interactive mode comes from Jest and its incredibly useful interactive watch mode.

Copyright and License

Copyright (c) 2021-2023 Randy Coulman

This work is free. You can redistribute it and/or modify it under the terms of the MIT License. See the LICENSE.md file for more details.

mix_test_interactive's Issues

Update README for new version

Just a quick note that the README still shows 1.0 but 2.0.0 is current.

Support for running test/describe by name

Hi,
Thanks for the great tool!
I'd been to contribute if you think it's a good idea to add functionality this functionality from mix test:

mix test --exclude test --include 'test:test file upload on upload for all sets submit for all' file_upload_test.exs

Not sure if we can get list of all tests somehow - maybe with test.trace? or maybe mix.test provides some facility to do so.

Time out if input is received while tests are running

** (exit) exited in: GenServer.call(MixTestInteractive.InteractiveMode, {:command, "\n"}, 5000)
    ** (EXIT) time out
    (elixir 1.11.3) lib/gen_server.ex:1027: GenServer.call/3
    (mix_test_interactive 1.0.0) lib/mix_test_interactive.ex:23: MixTestInteractive.loop/0
    (mix 1.11.3) lib/mix/task.ex:394: Mix.Task.run_task/3
    (mix 1.11.3) lib/mix/cli.ex:84: Mix.CLI.run_task/2

Steps to reproduce:

• mix test.interactive
• press enter while tests are running, and if it takes more than 5 seconds, task exits

Allow customization of command used to run tests

Currently, the use of mix to run the tests is hard-coded. The actual task to run is configurable, but not the mix part.

This makes certain use cases very difficult, or even impossible. See #59 and #62 for examples.

A possible solution would be to allow a new command (or command/args) configuration option that takes either a string or an array of strings, defaulting to mix. Then, when running the tests, mix_test_interactive would use the provided command to run the tests.

Examples:

# config/config.exs
use Mix.Config

if Mix.env == :dev do
  config :mix_test_interactive,
    command: ["elixir", "--sname", "foo", "-S", "mix"]
end

# config/config.exs
use Mix.Config

if Mix.env == :dev do
  config :mix_test_interactive,
    command: "path/to/my/test_runner.sh"
end

Support for ash

I was wondering if there is any interest in supporting ash in addition to the current support for bash. I imagine a lot of people in the community use Alpine images and adding another shell might not be the most appealing option.

I'd be happy to work on this if you'd accept a PR.

Support dynamic changing of filters

mix test supports a number of options for filtering which tests will be run: --include, --exclude, and --only.

Currently, if any of those options are provided to mix test.interactive, they will be forwarded on to mix test on each run.

It would be a nice addition to mix test.interactive to be able to dynamically change these filters in interactive mode without having to restart.

I don't use tags on my tests normally, so I'm not sure of the best way to implement such a feature. I would love feedback from those who would use such a feature. How should this work? What use cases need to be supported?

Some questions:

• Does it ever make sense to use both --only and --include/--exclude at the same time? Or are they mutually exclusive?
• It looks like sometimes --exclude should be specified before --include, but other times, --include should come before --exclude. How could mix test.interactive decide which order to use?
• How should this feature interact with the settings from ExUnit.configure? mix test.interactive would not likely be able to change those settings entirely. It would only be able to override them via the command-line options.

Thanks!

Running other tasks like Credo

Is there any way to run things like Credo or Dialyxir from mix_test_interactive?

Config file option to specify all/stale/failed

Hi! 👋 Great tool.

Would it be possible to set the default group with a configuration file? For example:

config :mix_test_interactive, group: :stale

How to work with --name and --sname

Thanks for the project.

If I start my test with:

elixir --sname foo -S mix test.interactive

And I inspect node() from the test, I get :nonode@nohost.

On the other hand, when I launch the test like so:

elixir --sname foo -S mix test

I get :foo@my_host

Is there an option to make mix test.interactive work with --sname and --name?

Switch between file and line number

Hi, thanks for the library! I think this is a feature request, but with pointers I'd be happy to look into implementing it.

I often want to switch between running all tests in a file and running test at a given line. Is there an easy way to achieve this? I think it'd only make sense if the currently running tests are all in one file.

Config-less operation

Some of mix_test_interactive's behavior can be modified using application configuration. For example, the clear, exclude, and task options can only be configured in app config.

Projects generated with mix new no longer include a config by default (for good reasons), which makes it harder to customize mix_test_interactive.

Ideally, there would be a way to customize mix_task_interactive without using application configuration. The most obvious answer to this would be to also support command-line arguments for configuring these settings.

In order to do this, there are some problems to solve:

• Conflicts between mix_test_interactive's command-line arguments and mix test's arguments. For example, mix test already takes an --exclude argument that would conflict with mix_test_interactive's config setting. We might want to use a prefix for our own arguments like mti- or _ or similar.
• Specifying complex argument values, like the array needed by exclude or the command setting proposed in #63. We could attempt to parse an Elixir array from those arguments, but would have to find a way to do that in a secure way that didn't allow for malicious code.