sourcefrog / cargo-mutants Goto Github PK
View Code? Open in Web Editor NEW:zombie: Inject bugs and see if your tests catch them!
Home Page: https://mutants.rs/
License: MIT License
:zombie: Inject bugs and see if your tests catch them!
Home Page: https://mutants.rs/
License: MIT License
If cargo mutants
is run from a subdirectory of a Rust source tree, with no -d
, it errors. It could instead find the enclosing project, perhaps using cargo locate-project
.
Clap 3 seems to have addressed some of the weird API overgrowth in earlier versions, and also to be more actively maintained than Argh. In particular this would allow syntax like --cargo-arg=--release
for #59: Argh insists on --cargo-arg --release
.
Perhaps have it mark a file when it stops, or hold a lock while it's running.
cargo mutants
command does not generate mutants for workspace members on a workspace project. Let's say I have a workspace project which contains 2 binary and 1 lib cargo crates. When I execute the cargo mutants
on the root, there are no mutants generated, so the workspace member crates are not included in the generation of the mutants.
I prepared a GitHub repo for reproducing the problem:
Clone the following workspace test project test-repo-for-cargo-mutants
cargo mutants
Mutants are generated for all the projects in the workspace.
Note: I would expect this to happen because when for example I execute cargo test
, then all the tests of workspace member crates are executed. The same for cargo build
.
No mutants are generated
==PS==
Anyway, awesome job with this crate, thank you!
So I've been running mutants on my code base all day, and while showing only the failing mutants and current one does keep the output nice and terse it does mean I have no idea how many more are left to run before it's finished. Something like N/total
on the in-progress line would helpful for gauging if it's worth waiting for it to finish or whether I should consider killing it and adding some ignores in the codebase ๐
Any extra arguments to cargo mutants
are passed to cargo test
, but it would be helpful in some cases to also pass custom arguments to cargo build
(or cargo check
if that's what's used.)
I believe that it would be handy to exclude specified paths from the mutants generation.
Let's say that I want to exclude things like configuration logic or some test helper logic files, located within the src
directory. Right now the default behaviour of cargo mutants
command is that it tries to generate mutants for every file in the src
directory and in their child directories.
I could envision having a command line parameter as follows:
cargo mutants --exclude <regexes-for-paths-to-be-excluded>
Git clone test repo
cargo mutants --exclude "(\/src\/tests)" "(\/src\/config)"
Every file which is located in either /src/tests
or /src/config
directories are excluded from mutants generation.
Because of the default behaviour of cargo-mutants
(and due to the non-existency of this command line parameter) there are mutants generated for the files both in /src/tests
and /src/config
In some situations where there are many possible mutants or they're slow to test, it might be good to say "test at many as you can in n minutes".
After the time expires cargo mutants should allow any current tests to finish and then return a result on what it found so far.
cargo mutants
does many incremental builds and test runs. In most trees, most of the time is spent running the tests.
Tests will typically run faster with --release
; in some crates it will be much faster.
So that should at least be an option (#59) but according to the principle of "make it easy" perhaps --release
should be the default.
There are at least two potential downsides:
debug_assert
to catch some bugs. I think this would be an unidiomatic use of Rust, but it might well happen.I think those could both reasonably be handled by having an option to go back to debug builds.
Although cargo and the test framework try to use many cores, they often go through phases of being single-threaded.
The overall mutation tests would go faster if we ran multiple cargo builds in parallel.
Conceivably up to NCPUs builds could be run in parallel. But on some machines that might cause thrashing or too much memory pressure. Possibly we should also look at available memory. There should at least be an option to limit it. As a starting point perhaps NCPUs/2 is reasonable.
This requires making multiple scratch directories. However, we should not make more than the number of discovered mutants.
CargoTask
with a poll
method--jobs, -j
and thread it through.lab
code:
-j2
works.Mutant::unapply
.Deferred until later:
cargo-mutants has limited dependencies on Cargo and could in principle be used with any other build tool.
This bug is primarily blocked on someone providing a realistic example of a Rust tree that builds with something other than Cargo. A minimized example would be good, but it should correspond to some real tree to give confidence that any solution actually works at scale. Preferably there would be an open source tree as a real example on which this could be tested.
cargo locate-project
.cargo metadata
. For Bazel we could potentially look at bazel query
?#72 adds some knowledge of cargo workspaces. These very loosely correspond to Bazel packages.
cc @kupiakos
The cargo mutants
command fails when there is a local dependency for a binary project.
This happens in a bigger project of us, but I created a test repo for the sake of easy reproduction.
Steps:
./main
directorycargo mutants
commandExpected behavior:
Actual behavior:
Freshen source tree ... ok in 0.015s
Copy source and build products to scratch directory ... 14 MB in 0.006s
Unmutated baseline ... FAILED in 0.007s
*** baseline
*** run /home/dmoka/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo check --tests
*** cargo result: Exited(101) in 0.007sency of package `main v0.1.0 (/tmp/cargo-mutants--eL1pHT.tmp)`
Caused by:
failed to load source for dependency `utils`
Caused by:
Unable to update /tmp/utils
Caused by:
failed to read `/tmp/utils/Cargo.toml`
Caused by:
No such file or directory (os error 2)
Note: when cargo run
is executed in the ./main
directory, the binary is executed properly with the local utils
dependency included. So I assume that it should have something to do with the cargo-mutants crate itself.
System: Ubunt 22.04
If you need any more information, let me know, thank you!
Sometimes tests fail in GitHub actions due to timeouts, e.g. https://github.com/sourcefrog/cargo-mutants/runs/7932363158?check_suite_focus=true.
It's not always the same test. It might be more common on Windows but it does happen elsewhere.
I think this is because the CI machines are over-committed and sometimes slow down.
Options in order of preference:
It's possible that this is not just slow but actually somehow getting deadlocked?
failures:
---- cargo_mutants_in_replace_dependency_tree_passes stdout ----
thread 'cargo_mutants_in_replace_dependency_tree_passes' panicked at 'Unexpected failure.
code-3
stderr=```""```
command=`"D:\\a\\cargo-mutants\\cargo-mutants\\target\\debug\\cargo-mutants.exe" "mutants" "--no-times" "--no-copy-target" "--no-shuffle" "-d" "testdata/tree/replace_dependency"`
code=3
stdout="Copy source to scratch directory ... done\nUnmutated baseline ... ok\nFound 2 mutants to test\nsrc/lib.rs:6: replace is_even -> bool with false ... TIMEOUT\n2 mutants tested: 1 caught, 1 timeouts\n"
stderr=""
', /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\ops\function.rs:248:5
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library\std\src\panicking.rs:584
1: core::panicking::panic_fmt
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library\core\src\panicking.rs:142
2: core::panicking::panic_display<assert_cmd::assert::AssertError>
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\panicking.rs:72
3: assert_cmd::assert::AssertError::panic<assert_cmd::assert::Assert>
at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\assert_cmd-2.0.4\src\assert.rs:1036
4: core::ops::function::FnOnce::call_once<assert_cmd::assert::Assert (*)(assert_cmd::assert::AssertError),tuple$<assert_cmd::assert::AssertError> >
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\ops\function.rs:248
5: enum$<core::result::Result<assert_cmd::assert::Assert,assert_cmd::assert::AssertError> >::unwrap_or_else<assert_cmd::assert::Assert,assert_cmd::assert::AssertError,assert_cmd::assert::Assert (*)(assert_cmd::assert::AssertError)>
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\result.rs:1484
6: assert_cmd::assert::Assert::success
at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\assert_cmd-2.0.4\src\assert.rs:156
7: cli::cargo_mutants_in_replace_dependency_tree_passes
at .\tests\cli.rs:906
8: cli::cargo_mutants_in_replace_dependency_tree_passes::closure$0
at .\tests\cli.rs:901
9: core::ops::function::FnOnce::call_once<cli::cargo_mutants_in_replace_dependency_tree_passes::closure_env$0,tuple$<> >
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\ops\function.rs:248
10: core::ops::function::FnOnce::call_once
at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library\core\src\ops\function.rs:248
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
failures:
cargo_mutants_in_replace_dependency_tree_passes
test result: FAILED. 54 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in [143](https://github.com/sourcefrog/cargo-mutants/runs/7932363158?check_suite_focus=true#step:6:144).12s
This is a really nice crate and works pretty great!
I have one issue: I really don't want mutants
to be a non-dev dependency. Do you think it's possible to define an attribute which can be checked, so for example the following example would work? This probably won't need the mutants
crate at all.
#[cfg_attr(mutants, mutants::skip)]
fn function_to_skip(&mut self);
Another possibility would be to set the test
-attribute when creating mutations, so this would work (this would only need a dev-dependency
):
#[cfg_attr(test, mutants::skip)]
fn function_to_skip(&mut self);
It would be handy to have a mutation to remove statements that call a single function or method. I see many times in bigger projects (written without TDD) that such things are not covered by tests.
For example, let's say I have the following code:
impl EventBus {
pub fn deposit_event(event_id: u32) {
print!("event with id {} is emitted", event_id);
}
pub fn do_logic(event_id: u32) {
//some logic
Self::deposit_event(3);
}
}
It would be nice to have a mutation of this code where we would delete the following line in the do_logic
method:
Self::deposit_event(3);
What do you think, it is feasible to do such a thing?
If I run cargo mutants, the following mutant is generated
src/main.rs:12: replace EventBus::deposit_event with () ... NOT CAUGHT in 0.248s
It rules out many cases, but does not rule out the situation where we have tests for public API deposit_event
, but not for calling it within other public API do_logic
cargo mutants
There is a mutant generated for removing a statement calling a specific function, something like:
src/main.rs:15: remove Self::deposit_event(3) ... NOT CAUGHT in 0.248s
No such mutation is supported by command cargo mutants
Right now, the entire function path is highlighted with a single color in the terminal output. This makes it difficult to read, especially when the path is deeply nested or contains complex <impl Trait for T>
parts.
I propose using the basic set of ANSI escape code colors to highlight different parts of the path, similar to how an IDE would. I believe the most important parts to distinguish by color are the brackets (<>
) and double colons (::
). Nice "could haves" would be different colors for keywords (impl
, for
), module names, type names and function names.
As far as I know, it should be possible to identify these parts of the path context-free, with the exception of type names and function names (here its possible to make some pretty good guesses though).
Split out from #20:
It would be good to have --no-copy-target
or similar, to omit copying /target
.
This would be helpful in isolating any test dependencies on the original build directory, and make more sure that the builds are all hermetic. The downside is that at least the initial build may be slower.
Also in that case, there's probably no point doing a build in the source tree.
So a pattern I've seen a number of places in rust code (and have in my own code in places) is:
impl SomeStruct {
pub fn new() -> Self {
Self::default()
}
}
In cargo-mutants it will take this function and apply the Default
mutagen to it creating code that is equivalent to the source code and then register as not caught. This could be solved by #[mutants::skip]
so it's not a must have but I figured it might be a reasonably simple/nice optimisation for working out the number of mutants. Similarly if someone has a function which returns a bool but is just a single literal the only valid mutagen is the negation of that constant not the constant itself
Fuzz testing randomly varies the input to the program and sees if the program will crash. Mutation testing deterministically varies the program (in ways that will probably still compile) and sees if your tests detect the change.
Probably pseudo-random edits to the program are introducing bugs that should be caught by tests, and so if no tests fail that might be a gap in test coverage. (Or, it might be that the edit is semantically harmless, or it changes something that is very hard to test hermetically, such as making something slowerโฆ)
fuzz testing is generally used to tell you that your program will crash/hang/spin on bad input (i.e. there are bugs in your code, if you canโt trust the input); mutation testing tells you that some presumably-bad edits to your code would not be caught โ i.e. if a hapless contributor proposed some patches they would not be caught by CI
fuzz testing e.g. with cargo-fuzz for Rust requires that you have an interface to โrun the code on this blob of bytesโ, e.g. โparse this blob of bytes as an MP3 fileโ; mutation testing doesnโt make any assumptions about the structure and doesnโt need you to write a harness: you can just run it on any existing crate with hermetic tests.
fuzz testing is necessarily sampling-based because there are so many possible input files once you get beyond a couple of bytes (256 << n_bytes); mutation testing can in principle test every mutation it can generate. (Although for large crates that might be a large number, itโs still probably only in the thousands-millions, not astronomical.)
Originally posted by xd009642 February 13, 2022
So this may be worth creating an issue for, but this is largely an idle thought I had yesterday. Mutation testing improves on things like coverage by making sure the tests are actually useful and not just hitting a ton of lines/conditions but not checking any of the values. Now if we already have code coverage results for our tests and can see some functions aren't tested at all we could save time by not applying mutations to them - after all none of the mutations would be caught.
For maximum usability this should probably take the form of accepting an optional argument of some open coverage format like an lcov report or cobertura.xml which a number of coverage tools already output.
Yeah, good idea. mutagen
optionally does something live this according to its documentation, but I have not looked at the implementation.
We could take it a step further by understanding which tests run which function under test. Functions not reached by any test we know are just apparently not tested. Functions that are reached by some tests, we can mutate and then run only the relevant tests. This would potentially be dramatically faster on some trees.
That said, I think there are a few things that might make this annoying to implement reliably, but perhaps my preconceptions are out of date. In the past, getting coverage files out of Rust historically to be a bit platform-dependent and fiddly to set up in my experience. And, historically the output was in a platform-dependent format that required external preprocessing. Both of these are in tension with my goal for a very easy start with cargo-mutants.
However there is now https://blog.rust-lang.org/inside-rust/2020/11/12/source-based-code-coverage.html providing -Z instrument-coverage
, which is moving towards stabilization as -C instrument-coverage
.
So if this ends up with a way to just directly get a platform-independent coverage representation out of cargo
this might be pretty feasible.
Coverage may still raise some edge cases if the test suite starts subprocesses, potentially in different directories, as both cargo-mutants and cargo-tarpaulin seem to do. Will we still collect all the aggregate coverage info? But, we could still offer it for trees where it does work well. And maybe it will be fine.
There might also be a hairy bit about mapping from a function name back to the right cargo test
invocation to hit it. But that also can probably be done: if nothing else perhaps by just running the test binary directly...
Possibly this could be done with https://github.com/taiki-e/cargo-llvm-cov
Perhaps the easiest-to-use structure is missed.txt
, caught.txt
, etc.
It needs to coordinate with or be routed through Nutmeg so that it doesn't scramble the progress bar.
So for my project I have to run with cargo test -- --test-threads 1
as the project uses unix APIs that aren't thread-safe within a process so running the integration tests in parallel causes failures. This means my unmutated run fails and I can't try cargo-mutants
on my project. Not sure if there's a way to pass this that I've missed, but if there's not it feels like a useful thing to add (and I'd be willing to help ๐ )
e.g.
Unmutated baseline ... ok in 30 + 40 + 10 s
For example Option<Result<String>>
could generate various values within the tree of possibilities.
Conceivably this might generate too many mutations for some functions in which case we might need a cap.
Some trees might have configured e.g. unused variables to be an error, for that real Go-in-Rust feeling.
These are likely to fail to build when mutated, but this isn't a useful signal. It'd be better to turn off these warnings for mutated builds.
There seem at least two options:
#[allow:all]
in the mutated function?RUSTFLAGS=-Aall
RUSTFLAGS=--cap-lints=allow
I believe that we could improve the names of our automated tests, and setting up a guideline for it, to
The ShouldWhen
and the GivenWhenThen
template are the most popular ones, personally I am a big fan of ShouldWhen
template as it is quite flexible in most of the cases. But I am open to any other suggestions :)
Of course just as a guideline, and not a rigid rule. But eventually we could add this guideline to CONTRIBUTING testing section
Just to pick some examples:
list_mutants_well_tested_exclude_folder_filter
would become
exclude_option_should_exclude_folder_when_specified_with_slash
Or
copy_testdata_doesnt_include_build_artifacts
would become
build_artifcats_should_not_be_included_when_testdata_is_copied
Or
error_when_no_mutants_found
would become
mutants_generation_should_return_error_when_no_mutants_are_found
What do you think?
cargo-mutants currently assumes that src/**/*.rs
will find all the source files in a tree, which is typical but not necessarily true for Rust source trees. https://github.com/Nukesor/pueue is one example where it is not true.
Perhaps a better approach here is to look at the output of cargo metadata
to find out where the source is? Or, possibly to look at the commands that are run during a build in the source tree...
Lines 95 to 122 in 687590c
Requires changing cp_r
to let a callback tell it to stop (sourcefrog/cp_r#1).
Is there an option somewhere to disable running doctests? I am attempting to run this on a fairly large project with over 50 doctests, but it causes each mutant to take a long time to run. That, combined with a lot of mutations (over 300) makes for a long run-time.
For comparison, running normally takes ~15 seconds for each mutation, which ends up being about an hour and a half. However, if I modify my library's Cargo.toml
to disable doctests (setting doctests = false
), the time for each mutant reduces to about 1-1.5 seconds each. That ends up being only approximately 7.5 minutes, which is much more desirable, especially if I want to run this in CI.
Is there a way a flag can be added to disable doctests? Or perhaps one already exists? In general, my doctests tend to test things already covered in my unit tests, and serve more as examples in the documentation, so I don't think much is lost by excluding them.
Is there a way to get some descriptive text into the list? It might not be possible for plugin commands.
https://github.com/kupiakos/papermario-solver/blob/main/Cargo.toml builds a WASM application as a cdylib
. Cargo-mutants 0.2.11 doesn't treat cdylib
targets as containing sources that should be mutated, so it finds no files and no mutants.
*lib
targetsFor example, in a PR, it would be interesting to see new mutants that are missed relative to the baseline, or mutants that are newly caught.
This could be done by reading a file from mutants.out
in the old tree and doing some kind of intelligent diff. The line numbers are likely to have changed but we could probably match on the function name and mutation.
What you probably actually want to debug is target/debug/cargo-mutants mutants --exclude ...
, after building it.
Note you must have mutants
as argv[1]
because that's the protocol for cargo subcommands.
This will run your new code as the direct child process and so the debugger should work.
I have used the VS Code debugger although I probably don't reach for it as often as would be optimal because I find the UI of configuring targets in json a bit weird. But this should work with anything that can launch and debug a child.
Originally posted by @sourcefrog in #65 (comment)
If someone's iterating on trying to improve coverage it might be helpful to only try mutants that were not caught on the last run, maybe by automatically looking in mutants.out
.
However you can get close to this today by narrowing it with -f
.
Closely related to #57.
I'll pick this up, based on ideas for cl3joly but in a separate branch.
This could be done in either a positive or negative sense:
Basic idea is to:
--iterate
command-line option (it would not make sense to have in the config.)caught.txt
in that output directoryIn this approach if you also change the filters, for example focussing on one file at a time, then after that change it will test everything, which seems OK.
It seems like ecryptfs has a limit of 143 chars for file name. That causes cargo mutants to fail sometimes.
Would you accept a PR with some limit imposed here
Line 36 in 26199b3
Hi.
I created a simple fizz buzz program as example.
Executing 'cargo mutants' leads to the result
$ cargo mutants
Freshen source tree ... ok in 0.184s
Copy source and build products to scratch directory ... 7 MB in 0.326s
Unmutated baseline ... ok in 1.892s
Auto-set test timeout to 20.0s
Found 2 mutations to test
after executing
git add .
git commit -m "Example"
in a git Bash, executing 'cargo mutants' leads to s different result.
$ cargo mutants
Freshen source tree ... ok in 0.154s
Copy source and build products to scratch directory ... failed
error copying source tree . to C:/Users/Foo~1/AppData/Local/Temp/.tmpfEmxQg: copy source tree to lab directory
Caused by:
0: writing file: C:\Users\Foo~1\AppData\Local\Temp\.tmpfEmxQg\.git\objects\07\4cf47c98083e4f871d3da4bc11050869ce2ec2: Zugriff verweigert (os error 5)
1: Zugriff verweigert (os error 5)
Error: copy source tree to lab directory
Caused by:
0: writing file: C:\Users\Foo~1\AppData\Local\Temp\.tmpfEmxQg\.git\objects\07\4cf47c98083e4f871d3da4bc11050869ce2ec2: Zugriff verweigert (os error 5)
1: Zugriff verweigert (os error 5)
I tried it with different projects, but the result is always the same. Before committing everything is fine, after committing I get Access Denied (os error 5).
It would sometimes be useful to check for mutants in just one or a few files, e.g.
cargo mutants -f mutate.rs
Probably this should match a whole path from the root if it contains slashes, and just the filename if it does not.
General interpretation of cfg
conditions seems difficult but we could at least skip #[cfg(windows)]
on Unix and so on. At the moment they are mutated and reported as missed.
This could perhaps use https://docs.rs/cargo-platform/0.1.2/cargo_platform/.
Cargo also seems to expose cfg variables to build scripts, but perhaps not otherwise. This might not be usable. https://doc.rust-lang.org/cargo/reference/environment-variables.html
https://docs.rs/target-spec/latest/target_spec/ exists and might be useful.
While there are issues with killing the test process while in progress, for some repositories this isn't an issue, and timeouts could be enabled by the command line
Perhaps, cargo mutants
should only list mutants that were missed, as that's really the actionable output. -v
could show everything that was tested.
list_mutants_in_all_trees
runs --list
on everything under testdata/tree
, but this easily gives false negatives when switching between branches, because untracked target
files can be left behind, causing the directory to still exist when the expected data does not.
One way would be to simply remove this test.
Another smaller step would be to run it only on directories that contain a Cargo.toml
and so that seem to "really" exist.
Currently, if I want to use #[mutants(skip)]
, I need to add mutants
to Cargo.toml
as a dependency
. The readme at least promises that there is no effect on the generated code by this. I think it should be sufficient to add it as a dev-dependency
.
Minor issue, thanks for the tool!
Block merging changes that don't have canonical formatting.
These options need to be passed consistently to all of check, build, and test.
For example, time-rs/time can't build its tests unless --all-features
is specified. time-rs/time#497
More generally people might want to test some crates with specific features.
#59 is about having custom options specifically for the build, but these seem important enough that they might be worth handling specifically.
--release
would be another good one to add, either for crates where it significantly helps the test suite speed, or for crates like syn
where the test suite insists on --release
. (Tangentially, it might actually be a good idea in general, depending on the balance of time between linking and running the test suite...)
This could be tested with a specific testdata tree that fails to build unless a given feature is given.
Hi me again ๐
, I have the following test which works always outside cargo-mutants but failed in the unmutated baseline test on the final assert. The manifest dir stuff is all correct in the run as it works for other tests of a similar pattern. The only thing that stands out as unusual in this test is the modifying of the CARGO_HOME
environment variable. I'm going to ignore it for now to see if I can get through to running mutants and if I can I'll look into it more afterwards. But just making this issue in case you can see anything that stands out initially.
#[test]
fn cargo_home_filtering() {
let new_home =
Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/data/HttptestAndReqwest/new_home");
let previous = env::var("CARGO_HOME");
let mut config = Config::default();
config.test_timeout = Duration::from_secs(60);
config.set_clean(false);
let restore_dir = env::current_dir().unwrap();
let test_dir = get_test_path("HttptestAndReqwest");
env::set_current_dir(&test_dir).unwrap();
config.manifest = test_dir;
config.manifest.push("Cargo.toml");
env::set_var("CARGO_HOME", new_home.display().to_string());
let run = launch_tarpaulin(&config, &None);
match previous {
Ok(s) => env::set_var("CARGO_HOME", s),
Err(_) => {
let _ = Command::new("unset").args(&["CARGO_HOME"]).output();
}
}
let (res, _) = run.unwrap();
env::set_current_dir(restore_dir).unwrap();
assert_eq!(res.iter().count(), 1);
}
Also repo link: https://github.com/xd009642/tarpaulin
Following on from #58 it would be nice to have options to control which functions are mutated.
Perhaps a regexp against the entire generated function name.
For example you might want to exclude all impl Debug
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.