Comments (4)
Some notes for my future self:
- I don't think this necessarily needs to be tied to archive/reuse work, because you could in principle also provide a directory to redirect
NEXTEST_BIN_EXE_<my-app>
to. - There is some orchestration that needs to be done here. For example, I suspect
cargo-dist
makes separate archives for each binary. It's a bit hard to build all the orchestration into nextest itself because it doesn't control both ends of the process. - Note that
CARGO_BIN_EXE_
is not set at run time, only at build time.NEXTEST_BIN_EXE_
is set at runtime. This means that tests will have to useNEXTEST_BIN_EXE_
at runtime. - The integration tests will have to support nextest. This can't really be backported to
cargo test
in an easy fashion. - Does this bring up use cases for exposing the
NEXTEST_BIN_EXE_
variables to more kinds of tests via a config option? Need to think about this. One challenge is that in the regular flow, binaries might not actually be built if there's no corresponding integration test or benchmark. Maybe defer this to the future. cargo-dist
could set its ownCARGO_DIST_EXE_
environment variables. The advantage of building this into nextest is that any tests containingNEXTEST_BIN_EXE_
would work transparently withcargo-dist
. The disadvantage is that projects would have to switch to nextest.
from nextest.
One use case that we need to think about is integration tests that use multiple binaries:
- Imagine a multi-crate workspace with two crates, binary-a and binary-b.
- You want to write an integration test that runs both binary-a and binary-b.
- You pretty much have to make it a third crate that's test-only, and run
cargo build
from within the tests for that crate.
Not supporting this would diminish the value of nextest and cargo-dist, I think.
An alternative might be that nextest lets you say: if you're building a particular crate's tests, also build this other binary -- then set NEXTEST_BIN_EXE_
for those tests. However, nextest doesn't currently do this, and supporting it can be rather delicate (eg what features to build the binary with? What if you want to use different set of flags/debug vs release/etc)?
Another option is to support this in setup scripts (#683). That requires even more design work.
from nextest.
After some discussion it became clear that point 6 was especially salient. Users would have needed to modify their code to conditionally check for NEXTEST_BIN_EXE_* anyway, so we can just do the same thing with our own var:
let my_app = std::env::var("OVERRIDE_CARGO_BIN_EXE_my-app")
.unwrap_or_else(|_| env!("CARGO_BIN_EXE_my-app").to_owned());
With this you can cargo test
like normal, but then have a preprocessing step that sets OVERRIDE_CARGO_BIN_EXE_my-app
before running cargo test
to inject prebuilt bins!
from nextest.
I'm going to close this as resolved for now, but I'll reopen if I see more room for tighter integration.
from nextest.
Related Issues (20)
- Ability to specify a custom cargo profile? HOT 2
- `build.rs` line change slows tests HOT 2
- Ability to enable features in the configuration HOT 2
- Allow user-specific configuration
- Allow to display stdout, while silencing test harness. HOT 2
- Add support for excluding paths from an archive
- Archive extra files, relative to paths other than the target directory
- Add a way to include extra files on a per-test-binary and per-platform basis
- Add a "testimonials" page (please comment here!) HOT 3
- [windows] lots of tests running in parallel may cause "leaky" warnings HOT 3
- v0.9.70 using significantly more stack HOT 7
- Idea: network isolation between tests HOT 14
- Add support for `cargo test`s --skip and --exact command line flags HOT 1
- `cargo nextest run --workspace` fails with DLL missing if a macro lib exists HOT 14
- nextest chooses arbitrarily not to run some tests it's found HOT 6
- Consider conditions to not include libstd in archives
- [meta] tell users what configuration their tests are running with
- Human-readable configuration output HOT 1
- Machine-readable configuration output
- Feature Request: Conditional skipping test based on setup scripts
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nextest.