burntsushi / quickcheck Goto Github PK
View Code? Open in Web Editor NEWAutomated property based testing for Rust (with shrinking).
License: The Unlicense
Automated property based testing for Rust (with shrinking).
License: The Unlicense
There's more than a few compile errors resulting from using the latest nightly:
$ rustc --version
rustc 1.0.0-nightly (dfc5c0f1e 2015-02-18) (built 2015-02-19)
$ cargo build
Compiling quickcheck v0.1.39 (file:///home/caspar/src/rust-quickcheck)
src/arbitrary.rs:150:29: 150:44 error: the parameter type `A` may not live long enough [E0310]
src/arbitrary.rs:150 let chain = single_shrinker(None).chain(x.shrink().map(Some));
^~~~~~~~~~~~~~~
src/arbitrary.rs:150:29: 150:44 help: consider adding an explicit lifetime bound `A: 'static`...
src/arbitrary.rs:150 let chain = single_shrinker(None).chain(x.shrink().map(Some));
^~~~~~~~~~~~~~~
src/arbitrary.rs:150:29: 150:44 note: ...so that the type `core::option::Option<A>` will meet its required lifetime bounds
src/arbitrary.rs:150 let chain = single_shrinker(None).chain(x.shrink().map(Some));
* etc etc for another 85 lifetime-related error messages *
src/tester.rs:189:30: 189:36 error: type `core::result::Result<std::thread::JoinGuard<'_, T>, std::io::error::Error>` does not implement any method in scope named `join`
src/tester.rs:189 .join()
^~~~~~
src/tester.rs:446:29: 446:35 error: type `core::result::Result<std::thread::JoinGuard<'_, T>, std::io::error::Error>` does not implement any method in scope named `join`
src/tester.rs:446 match t.scoped(fun).join() {
^~~~~~
error: aborting due to 88 previous errors
Could not compile `quickcheck`.
I had a shot at fixing it without any success (still getting my head around lifetimes and associated syntax) so I figured the next best thing is reporting it.
Related: maybe it'd be worth fixing up http://www.rust-ci.org/BurntSushi/quickcheck ? Seems like it hasn't kicked off a new build in almost a year; it's not particularly clear on that page but on the main rust-ci.org page it claims it was unable to build and needs the owner to authenticate, or something along those lines.
I very quickly ran into the four-argument limit, so it'd be lovely to have Testable instances for functions with more arguments.
Alternatively implement (see below).Arbitrary
for all arrays using the FixedSizeArray
trait from the standard library (requires nightly).
In a bygone era, I invented the Shrinker
trait as a proxy so the shrink
method of Arbitrary
could return any iterator without regard to its underlying type. This was because Box<Iterator<A>>
didn't itself implement Iterator<A>
.
Now that DST is here and Rust has a more coherent story regarding trait object safety, it would be nice to remove this proxy trait and return Box<Iterator>
instead.
I started working on this, but I ran into some bugs: rust-lang/rust#20605 and rust-lang/rust#20953. Once those are fixed, I think we can drop Shrinker
.
This can happen due to invalid casts from quickcheck::Gen::size
of type usize
to various other integer types with a value range smaller than usize
(cf. quickcheck::arbitrary::unsigned_arbitrary).
Test case:
#[test]
fn test() {
fn prop(_: u8) -> bool { true }
QuickCheck::new()
.gen(StdGen::new(rand::thread_rng(), 1024))
.quickcheck(prop as fn(u8) -> bool)
}
Currently, shrinking numbers is done by filling up a vector with all possibilities and returning it as an iterator. It would be great if this were changed so that each shrunk value wasn't generated until the iterator advanced.
When I try to compile quickcheck_macros
as a Cargo dependency for something else, I get:
Build failed, waiting for other jobs to finish...
Could not compile `quickcheck_macros`.
--- stderr
src\lib.rs:29:48: 29:70 error: mismatched types: expected `Box<syntax::ext::base
::ItemModifier+'static>`, found `fn(&mut syntax::ext::base::ExtCtxt<'_>, syntax:
:codemap::Span, Gc<syntax::codemap::Spanned<syntax::ast::MetaItem_>>, Gc<syntax:
:ast::Item>) -> Gc<syntax::ast::Item>` (expected box, found extern fn)
src\lib.rs:29 ItemModifier(expand_meta_quickch
eck));
rustc
version is rustc 0.12.0-pre-nightly (79a5448f4 2014-09-13 20:36:02 +0000)
, quickcheck_macros
version is v0.1.0 (https://github.com/BurntSushi/quickcheck#c5a719a3)
. Windows 64-bit (hence the unwieldy line breaks, blame the Win32 console!).
Thanks!
Using stable Rust trying to install quickcheck_macros gives:
Compiling quickcheck_macros v0.2.24
/home/users/russel/.cargo/registry/src/github.com-0a35038f75765ae4/quickcheck_macros-0.2.24/src/lib.rs:8:1: 8:45 error: #[feature] may not be used on the stable release channel
/home/users/russel/.cargo/registry/src/github.com-0a35038f75765ae4/quickcheck_macros-0.2.24/src/lib.rs:8 #![feature(plugin_registrar, rustc_private)]
which more or less makes this potentially wonderful testing tool useless. Or have I missed something?
Ok, so my problem is wanting to implement shrink()
for a type made up of other Arbitrary
types:
struct Foo {
bar: Bar,
baz: Baz,
}
impl Arbitrary for Foo {
fn arbitrary<G: Gen>(gen: &mut Gen) -> Foo {
Foo {
bar: Bar::arbitrary(),
baz: Baz::arbitrary(),
}
}
fn shrink(&self) -> Box<Iterator<Item=Foo>> {
let bars = self.bar.shrink().chain(Some(self.bar));
let bazs = self.baz.shrink().chain(Some(self.baz));
Box::new(bars.cartesian_product(bazs).map(|(bar, baz)| Foo { bar: bar, baz: baz }))
}
}
I think this shrink()
impl is very elegant, but it doesn't work because cartesian_product
(from itertools
) requires Clone
on the second iterator (of course). This is a well known problem with trait objects; making shrink()
return Box<Iterator + Clone>
isn't a very satisfying solution, since there's a question of "where do I stop? what about Sync + Send
etc?. Furthermore, closures are never
Clone` :(. Making the iterator type of shrink an associated type
pub trait Arbitrary {
type Shrink: Iterator<Item=Self>;
fn shrink(&self) -> Self::Shrink;
/* ... */
}
means a default impl can't be provided (at least not without specialisation), but it would solve the issue. I know I can implement a shrink iterator by storing bar
and baz
itself, and creating new iterators all the time, but that is a lot of boilerplate. Is there some general solution to this problem of composing Arbitrary
types?
I have a type which wraps an i32
, and wrote an Arbitrary
impl like so:
trait Arbitrary<PgDate> {
fn arbitrary<G: Gen>() -> Self {
PgDate(i32::arbitrary())
}
}
When I actually output what it's running with, the values are only ranging from 100
to -100
, instead of 100 completely random numbers as I'd expect. Additionally, shrink
appears to be flawed on these types. My understanding was that quickcheck would test for known problematic values for a given type. For i32
I'd expect that to at minimum be 1
, 0
, -1
, i32::MAX
and i32::MIN
, but when I add the shrink
like so:
fn shrink(&self) -> Box<Iterator<Item=Self>> {
Box::new(self.0.shrink().map(PgDate))
I just see the same range of -100 to 100. This caused a critical bug that occurs for any values less than -2451545
to go unnoticed.
When building on the latest nightly, I get the following error message:
$ cargo test
Downloading rand v0.3.12
Downloading log v0.3.4
Downloading quickcheck v0.2.24
Downloading winapi v0.2.5
Downloading libc v0.2.2
Downloading quickcheck_macros v0.2.24
Compiling winapi v0.2.5
Compiling libc v0.2.2
Compiling log v0.3.4
Compiling advapi32-sys v0.1.2
Compiling rand v0.3.12
Compiling quickcheck v0.2.24
Compiling quickcheck_macros v0.2.24
/Users/hawk/.multirust/toolchains/nightly/cargo/registry/src/github.com-88ac128001ac3a9a/quickcheck_macros-0.2.24/src/lib.rs:23:5: 23:28 error: unresolved import `rustc::plugin::Registry`. Could not find `plugin` in `rustc` [E0432]
/Users/hawk/.multirust/toolchains/nightly/cargo/registry/src/github.com-88ac128001ac3a9a/quickcheck_macros-0.2.24/src/lib.rs:23 use rustc::plugin::Registry;
^~~~~~~~~~~~~~~~~~~~~~~
/Users/hawk/.multirust/toolchains/nightly/cargo/registry/src/github.com-88ac128001ac3a9a/quickcheck_macros-0.2.24/src/lib.rs:23:5: 23:28 help: run `rustc --explain E0432` to see a detailed explanation
error: aborting due to previous error
Could not compile `quickcheck_macros`.
To learn more, run the command again with --verbose.
On the latest nightly:
$ rustc --version
rustc 1.7.0-nightly (81ae8be71 2015-12-09)
$ cargo --version
cargo 0.7.0-nightly (1af03be 2015-12-08)
It seems like Option<bool>
would be a simple way to discard tests vs. having to return a TestResult
directly.
For example, this Arbitrary
implementation for a simple tree will overflow the stack more often than when running tests. I'm not sure how to bound the depth of the recursion here.
Looks like the most recent Rust broke things. Building my repo https://github.com/FranklinChen/type-directed-tdd-rust which uses quickcheck
resulted in
src/arbitrary.rs:60:29: 60:40 error: explicit lifetime bound required
src/arbitrary.rs:60 impl<A> Iterator<A> for Box<Shrinker<A>> {
^~~~~~~~~~~
src/arbitrary.rs:75:35: 75:46 error: explicit lifetime bound required
src/arbitrary.rs:75 pub fn empty_shrinker<A>() -> Box<Shrinker<A>> {
^~~~~~~~~~~
src/arbitrary.rs:89:44: 89:55 error: explicit lifetime bound required
src/arbitrary.rs:89 pub fn single_shrinker<A>(value: A) -> Box<Shrinker<A>> {
^~~~~~~~~~~
src/arbitrary.rs:372:25: 372:36 error: explicit lifetime bound required
src/arbitrary.rs:372 fn new(x: A) -> Box<Shrinker<A>> {
^~~~~~~~~~~
src/arbitrary.rs:407:25: 407:36 error: explicit lifetime bound required
src/arbitrary.rs:407 fn new(x: A) -> Box<Shrinker<A>> {
^~~~~~~~~~~
Currently, a vector is shrunk by producing all possible shrunk values immediately. It would be much better if they were generated lazily with an iterator.
Perhaps std::iter::Unfold
will be useful for this.
I have a quickcheck test using Vec<EnumWithManyVariants>
and my tests rarely fail. It would be nice if I could set an environment variable to make quickcheck try more possible inputs.
It might be possible to add a syntax extension (maybe similar to https://github.com/jakub-/rust-instrumentation/blob/654a074452b6556ade26b4d86948ce743b4f102c/examples/timing.rs#L52) so that code like
#[test]
fn is_sorted() {
fn prop() -> bool { ... }
quickcheck(prop)
}
Could be written like
#[quickcheck]
fn is_sorted() -> bool { ... }
Would be great to be able to generate these arbitrarily with features = ["unstable"]
. It seems like this should probably impl Arbitrary
for every reasonable owned type in the std library.
Running cargo test
in the quickcheck_macros
directory with the latest rust nightly 4874ca36f.
Compiling quickcheck_macros v0.1.30 (file:///Users/sam/Dropbox/Projects/quickcheck/quickcheck_macros)
tests/macro.rs:9:1: 9:14 error: functions used as tests must have signature fn() -> ()
tests/macro.rs:9 #[quickcheck]
^~~~~~~~~~~~~
tests/macro.rs:18:1: 18:14 error: functions used as tests must have signature fn() -> ()
tests/macro.rs:18 #[quickcheck]
^~~~~~~~~~~~~
tests/macro.rs:22:1: 22:14 error: functions used as tests must have signature fn() -> ()
tests/macro.rs:22 #[quickcheck]
^~~~~~~~~~~~~
tests/macro.rs:25:1: 25:14 error: functions used as tests must have signature fn() -> ()
tests/macro.rs:25 #[quickcheck]
^~~~~~~~~~~~~
error: aborting due to 4 previous errors
Could not compile `quickcheck_macros`.
To learn more, run the command again with --verbose.
Would it be a good idea to update the .travis.yml
file to also test the macros?
Shall this be incorporated?
Using rustc 1.0.0-nightly (3d0d9bb6f 2015-01-12 22:56:20 +0000):
fn reverse<T: Clone>(xs: &[T]) -> Vec<T> {
let mut rev = vec!();
for x in xs.iter() {
rev.insert(0, x.clone())
}
rev
}
#[test]
fn reverse_identity() {
fn prop(xs: Vec<i32>) -> bool {
xs == reverse(reverse(xs.as_slice()).as_slice())
}
quickcheck(prop);
}
produces
error: the trait `quickcheck::tester::Testable` is not implemented for the type `fn(collections::vec::Vec<i32>) -> bool {tests::reverse_identity::prop}`
quickcheck(false)
compiles; this appears to be specific to the fn impls, including the zero-arg fn.
Currently INFINITY
, NEG_INFINITY
, NAN
and possibly -0.0
are never generated by quickcheck. These values are valid floats, and should be tested for.
Current git version seems to hang after a test failure, continuously spawning new threads.
The #[quickcheck]
macro doesn't work when the quickcheck
crate is linked as a path other than ::quickcheck
:
#![feature(plugin)]
#![plugin(quickcheck_macros)]
#[cfg(test)]
mod test {
extern crate quickcheck;
#[quickcheck]
fn bar() -> bool { true }
}
Errors:
src/lib.rs:8:5: 8:15 error: failed to resolve. Maybe a missing `extern crate quickcheck`?
src/lib.rs:8 #[quickcheck]
^~~~~~~~~~
src/lib.rs:8:5: 8:18 error: unresolved name `quickcheck::quickcheck`
src/lib.rs:8 #[quickcheck]
^~~~~~~~~~~~~
It's clear why this is happening from the macro's implementation, but it would be nice for it to be supported. One solution would be to pass arguments to the #[quickcheck]
macro like
#[quickcheck(self, quickcheck)]
fn bar() -> foo { true }
and turn the sequence of identifiers into a path, defaulting to ::quickcheck
when none are given. It should also work when quickcheck
is used under a different name:
extern crate "quickcheck" as baz;
#[quickcheck(self, baz)]
fn bar() -> foo { true }
This would involve making use of the third argument (the &ast::MetaItem
) passed to expand_meta_quickcheck
, which is currently ignored.
make test
succeeds, but cargo test
fails to compile with quickcheck macro dependency issues. I spent a few minute poking around, but couldn't figure out how to fix the issues.
$ cargo test
Fresh quickcheck_macros v0.1.0 (file:///Users/dan/src/rust/quickcheck)
Compiling quickcheck v0.1.0 (file:///Users/dan/src/rust/quickcheck)
src/lib.rs:23:1: 23:25 error: can't find crate for `quickcheck`
src/lib.rs:23 extern crate quickcheck;
^~~~~~~~~~~~~~~~~~~~~~~~
It would be very convenient to add a way to export failing test cases to allow to reproduce them and make a regression test suite.
The failing test should be exported either as:
This feature is more or less presented in the following video:
https://www.youtube.com/watch?v=x7O2Hkq983Y
And is used on:
http://www.quickcheck-ci.com/
Details about why my my data structure failed.
UNABLE TO SHOW RESULT OF PANIC.
(I am still learning Rust, so bear with me if my code is dumb or I miss something obvious.)
I am trying to define simple, stable-compatible quickcheck tests for a trait, that can be reused to test different data structures that implement it. In this case, I am trying to test a persistent heap.
Data structure + generic test.
Unfortunately, one of my two implementations is broken somewhere, but the only information I get out of quickcheck is Error: "UNABLE TO SHOW RESULT OF PANIC."', <path>/quickcheck-0.2.21/src/tester.rs:113
.
The data structure in question has #[derive(Debug)]
and I have tested that println("{:?}", s)
works for this instance of the data structure, so I don't know why I am not getting any other information about the panic, and I don't know what to do to make the situation better.
Is it possible to also define edge cases beside the random data without implementing my own Arbitrary type?
Of course for such cases I could just use normal testingโฆ
Right now, String
s only have ASCII alphanumeric characters. I tried to use it to generate strings with spaces in it and was surprised when my code was never called ๐บ
It would be cool if there were more of the Unicode set included. Even awesomer might be if we had some tuning knobs where we could say "more punctuation, less letters", perhaps using the Unicode character classes?
Thanks for the fun-to-use library!
I wanted to write my own Arbitrary impls but apparently I cannot access the quickcheck::arbitrary
module?
Please consider this test case:
extern crate quickcheck;
use quickcheck::{Arbitrary, Gen};
#[derive(Clone)]
pub struct Foo(u8);
impl Arbitrary for Foo {
fn arbitrary<G: Gen>(g: &mut G) -> Foo {
Foo(g.gen())
}
}
Compilation with rustc 1.3.0-nightly (0c052199b 2015-07-11)
gives:
src/lib.rs:10:13: 10:20 error: source trait is private
src/lib.rs:10 Foo(g.gen())
^~~~~~~
I think it is necessary to export not only Gen
but also it's parent trait Rng
:
diff --git a/src/lib.rs b/src/lib.rs
index f13fcdc..ab1698b 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -10,6 +10,7 @@
#[macro_use] extern crate log;
extern crate rand;
+pub use rand::Rng;
pub use arbitrary::{
Arbitrary, Gen, StdGen,
empty_shrinker, single_shrinker,
It would be nice to be able to use quickcheck with (e.g.) Wrapping<u8>
. I'm not sure I can fix this in my own code, adding
impl Arbitrary for Wrapping<u8> {
fn arbitrary<G: Gen>(g: &mut G) -> Wrapping<u8> {
Wrapping(Arbitrary::arbitrary(g))
}
}
generates a compile error because neither Arbitrary
nor Wrapping
were defined by me.
At the moment when passing a function to quickcheck
, its type has to specified:
fn prop() -> bool {
true
}
quickcheck(prop as fn() -> bool);
However, bare functions can be passed as generic items implementing Fn
traits without any casting:
fn run<F, T>(f: F) -> T where F: Fn() -> T {
f()
}
fn prop() -> bool {
true
}
fn main() {
let t = run(prop);
println!("{}", t);
}
If we implemented the Testable
trait for F where F: Fn(A, ...) -> T
instead of the concrete fn(A, ...) - T
types, it would be possible to just do:
quickcheck(function);
I think this is a usability win, especially with #[quickcheck]
unusable under 1.0 (people are likely to be writing quickcheck quite a bit).
I had a quick go at implementing this, but ran into trouble with the Fun
trait. I couldn't work out a way to call the function (passed as &self
) in a move
closure passed to safe
. It may be possible, or else more extensive restructuring may be required.
It would be great to have an example for implementing shrink
with a custom type.
I have tests that fail very rarely. When this happens on Travis, I'd really like to reproduce the issue locally.
If quickcheck randomly chose a seed, but printed that seed value on failure, then I could replicate test failures.
On the search page for "shrink", some of the links lead to 404s. Can they be styled differently?
http://burntsushi.net/rustdoc/quickcheck/?search=shrink
http://burntsushi.net/rustdoc/collections/string/struct.String.html#method.shrink
I'm having errors for one test. I'm changing code and running cargo test
, but it's including the 300 other tests. Is there any way to flag the test to be the only one run? Something like #[only(quickcheck)]
?
I want quickcheck to have a compiler plugin that automatically generates Arbitrary impl for structs and enums composed of Arbitrary types, like with serialisation.
That should shrink
extern crate quickcheck;
use quickcheck::Arbitrary;
#[derive(Clone,Debug)]
struct Mumu {
a : u32,
b : String,
}
#[derive(Clone,Debug)]
enum Qqq {
Lol,
Ror,
Kokoko(Mumu),
Totot(u32),
}
impl Arbitrary for Mumu {
fn arbitrary<G : quickcheck::Gen>(g:&mut G) -> Mumu {
Mumu {a : Arbitrary::arbitrary(g), b : Arbitrary::arbitrary(g) }
}
fn shrink(&self) -> Box<Iterator<Item=Self>> {
Box::new ( (self.a.clone(), self.b.clone()).shrink().map(|(aa,bb)| Mumu { a: aa, b:bb}))
}
}
impl Arbitrary for Qqq {
fn arbitrary<G : quickcheck::Gen>(g:&mut G) -> Qqq {
let y = g.next_u32() % 4;
match y {
0 => Qqq::Lol,
1 => Qqq::Ror,
2 => Qqq::Kokoko(Arbitrary::arbitrary(g)),
3 => Qqq::Totot(Arbitrary::arbitrary(g)),
_ => panic!(),
}
}
fn shrink(&self) -> Box<Iterator<Item=Self>> {
match self {
&Qqq::Totot(ref x) => Box::new(x.shrink().map(|s| Qqq::Totot(s))),
&Qqq::Kokoko(ref x) => Box::new(x.shrink().map(|s| Qqq::Kokoko(s))),
_ => quickcheck::empty_shrinker(),
}
}
}
#[test]
fn it_works() {
fn qqq(x : Qqq) -> bool {
if let Qqq::Kokoko(Mumu { a : v, b: _ }) = x {
return v != 4
}
true
}
quickcheck::QuickCheck::new().tests(1000_000_000).quickcheck(qqq as fn(Qqq) -> bool);
}
to
extern crate quickcheck;
use quickcheck::Arbitrary;
#[derive(Clone,Debug,Arbitrary)]
struct Mumu {
a : u32,
b : String,
}
#[derive(Clone,Debug,Arbitrary)]
enum Qqq {
Lol,
Ror,
Kokoko(Mumu),
Totot(u32),
}
#[test]
fn it_works() {
fn qqq(x : Qqq) -> bool {
if let Qqq::Kokoko(Mumu { a : v, b: _ }) = x {
return v != 4
}
true
}
quickcheck::QuickCheck::new().tests(1000_000_000).quickcheck(qqq as fn(Qqq) -> bool);
}
Sometimes I write too many "discard conditions" (along with too many arguments), or sometimes I get those conditions wrong, and quickcheck ends up discarding all the 10k
tests, and flagging the unit test as passed. IMO, either case is a bug, because I'm not actually testing anything, and I should fix the discard conditions.
Right now, the only way to detect this issue is to use RUST_LOG=quickcheck
and look out for "Passed 0 Quickcheck tests" messages.
I would like to see some way to fail the unit test if too few quickcheck tests passed. Preferably, using a environment variable, say QC_PASSED_TESTS_THRESHOLD=25 cargo test
will fail the unit test if less than 25 quickcheck tests passed, if the variable is unset, the threshold will be zero, which is the current behavior.
Thoughts @BurntSushi ?
rust-lang/rust/pull/19955 killed TrieMap
and a few other collection types. quickcheck
provides an Arbitrary
impl for TrieMap
, but not for any other of the removed collections. What would make more sense - following stdlib and killing the TrieMap
impl altogether, or using the collect crate as dependency?
This is a question issue, and I hope that's an acceptable use in this project.
I try to use quickcheck to write tests for function accepting two strings. I haven't been able to adapt the example test to my use case. This is what I have. Is quickcheck able to provide random parameters for strings (or &str
s)?
extern crate quickcheck;
// at least difference of the sizes of a and b
#[test]
fn prop_test_() {
use quickcheck::quickcheck;
fn at_least_size_difference(a: &str, b: &str) -> bool {
let size_a = a.chars().count() as i32;
let size_b = b.chars().count() as i32;
let diff = (size_a - size_b).abs();
edit_distance::edit_distance(a, b) > diff;
}
quickcheck(at_least_size_difference as fn(a: &str, b: &str) -> bool);
}
Compiling quickcheck_macros v0.2.26
Running `rustc /tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs --crate-name quickcheck_macros --crate-type dylib -C prefer-dynamic -C opt-level=3 -C metadata=df62799bae73490e -C extra-filename=-df62799bae73490e --out-dir /tmp/ramp-master/target/release/deps --emit=dep-info,link -L dependency=/tmp/ramp-master/target/release/deps -L dependency=/tmp/ramp-master/target/release/deps --extern quickcheck=/tmp/ramp-master/target/release/deps/libquickcheck-ef66e0544cfbee5c.rlib --cap-lints allow`
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10:1: 10:21 error: found possibly newer version of crate `std` which `syntax` depends on [E0460]
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10 extern crate syntax;
^~~~~~~~~~~~~~~~~~~~
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10:1: 10:21 note: perhaps this crate needs to be recompiled?
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10 extern crate syntax;
^~~~~~~~~~~~~~~~~~~~
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10:1: 10:21 note: crate `std` path #1: /home/odroid/rust-1.8.0-system-alloc-LFS/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libstd-db5a760f.rlib
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10:1: 10:21 note: crate `std` path #2: /home/odroid/rust-1.8.0-system-alloc-LFS/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libstd-db5a760f.so
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10:1: 10:21 note: crate `syntax` path #1: /home/odroid/rust-1.8.0-system-alloc-LFS/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libsyntax-db5a760f.so
/tmp/.cargo/registry/src/github.com-48ad6e4054423464/quickcheck_macros-0.2.26/src/lib.rs:10:1: 10:21 note: crate `syntax` path #2: /home/odroid/rust-1.8.0-system-alloc-LFS/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libsyntax-db5a760f.rlib
error: aborting due to previous error
Could not compile `quickcheck_macros`.
Dinghy is almost two nights old.
README.md:
N.B. When using quickcheck (either directly or via the attributes), RUST_LOG=quickcheck enables info! so that it shows useful output (like the number of tests passed). This is not needed to show witnesses for failures.
RUST_LOG
no longer controls the output of the log
crate itself. The old behavior can be replicated with env_logger
, or the logging behavior can be changed (and the documentation be updated).
Downloading rand v0.3.11 (reasonable - quickcheck is a randomized test)
Downloading log v0.3.2 (reasonable - it's omnipresent)
Downloading winapi v0.2.4 ( ??? )
Downloading quickcheck v0.2.24
Currently, quickcheck doesn't provide a way to test properties that must trigger a task failure to pass. For example, if you want to test that bound checking is working, you would write the following property:
#[quickcheck]
fn out_of_bounds(length: uint, index: uint) -> TestResult {
let v = Vec::from_elem(length, 0u8);
if index < length {
TestResult::discard()
} else {
// v[index] ?
}
}
But you are missing a piece: You need some function that returns TestResult::passed()
if v[index]
triggers a task failure, or TestResult::failed()
if v[index]
succeeded.
That's why I propose adding the must_fail
function to the library, where must_fail
is defined as:
use quickcheck::TestResult;
use std::comm;
use std::io::ChanWriter;
use std::task::TaskBuilder;
// This should be a `TestResult` static method
fn must_fail<T: Send>(f: proc(): Send -> T) -> TestResult {
// A "/dev/null" sink for the stderr of the child task
let (tx, _) = comm::channel();
let dev_null = box ChanWriter::new(tx) as Box<Writer + Send>;
let failed =
TaskBuilder::new().
// Redirect the stderr of the child task to "/dev/null"
stderr(dev_null).
try(f).
is_err();
TestResult::from_bool(failed)
}
This function provides the missing functionality to complete the out_of_bounds
property:
#[quickcheck]
fn out_of_bounds(length: uint, index: uint) -> TestResult {
let v = Vec::from_elem(length, 0u8);
if index < length {
TestResult::discard()
} else {
must_fail(proc() v[index])
}
}
Which now can be tested:
running 1 test
test out_of_bounds ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
And if you wrongly implement the test, for example by reversing the index < length
condition, then the test would fail as expected:
running 1 test
test out_of_bounds ... FAILED
failures:
---- out_of_bounds stdout ----
task 'out_of_bounds' failed at '[quickcheck] TEST FAILED. Arguments: (1, 0)', src/lib.rs:83
failures:
out_of_bounds
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
@BurntSushi What do you think? Should this functionality be added to quickcheck?
Strings typically generted look like this:
๒ฝน๒ชฌ๒๓ฅซน๑ผ๒ถน๑๓ฟกฅ๑ฐซฉ๔ก๐ป๒
ช๒ธ๒๒ช๓ดฅ๔
ฒด๓ฉี๐กซ๓ผบ๒บฉช๐พบ๐ต
๐ป๑ฆ
๑ณฟ๓แค๑พ๑บฅฟ๒๓ฌญ๐น๑ฃ๑ฅ๒๒ค๔ฌ๓จ๓จซ๒บ๑ฝ๑ทฎ๓ป๐ฟ๑นฉต๒ฌ๑ฃ๓
๓๐ผ ๓ท๒ฝ
๑กฐ๐ฃ๓นฃต
๔
ค๓ถ๑ผฎณ๐ฎ๒๑ฐ๒ฆ๒นป๑ซ๓พช๒ขเตพ๐ฌซค๑ค๔พท๒ค๑ฎ
ๅผ
๒ฅ
ฝ๑จญด๓ถก๑ซฏญ๐งผไ แด๔ท๓ป๑๓ต๐ฏญ๔ผง๒ทบ๑๑ชง๓ฏฑฉ๑ฉด๐ญ๐จ๑กฅ๓ข๑บข๐ต็ป๓๑
๒ปจ๑ทฎ๓ธง๓ตธ๑ฝ๒กฅ๑ผฎ๒ผค๑ญ๑ทฌ๔ญก๒ผ ๑ฅ๓ข๒กฝ๔๑ฎ๑ฟซ๑ฟ๒๑๑ข๒ฏผ๑น๔ช๒๓ฌ ๓งธ๓๐ ฅค๑กญ๐งผ๑ฆผ๑ถฐ๑ ๐ป๒
๒บ๓ถผ๐ฐทน๐พ๒ฎญ๒ฅ๓ค๒ซ๓๓๐ฐป๑ซ๐ด
๐ ผ๓นๅพ๐๐ ๐พญญ๐ข๑ฅก๑ซ๒ซฏ๓จ๏จฑ๓ฐฒบ๒ณ๐๑กดถ๔น๒แผ๔
๓ฏ๓ค๓ท๔๑ฆ๋ด๒ฌฉ๑ฉญ๒ฒ๓ฅ๐ฐ๔๑ฅ๒
ณ๐ฟด๔๑ฐ๑ฏ๔กฉ๒ต ๑ญ๔ฃ๒
๒๓ฌฆ๑ฉ๓ฎ๒ฝ๓๑๑๓๑ฅ๓๐ฉ๑กจญ๒ก ๒ช๓ต๐ซฏ๑ฉฝ๐กซ๒ ๐คทธ๒บ๒ฉ๒ฎ๒นฃได๐ป๓ธ
ฉ๔ฌ๐ท๑ฒ๑ชจ๒๔ป๑๓น
๑ขฉ๒๐ฅฟ๒
๑ฌง๐๑ฎ๒ฟฎา๎ฅญ๓ข๓ ข๑ญ๓ด๓
ธ๒
๐ณ๑ชฉ๑ข๒ญง๓ฆฆ๓ชฆ๓ฑํ๑ง็ฏฌ๓ฆฒซ๒งน๑ฟ๑ฃช๒พ๐ก๓ซฌด๒ฝซ๑๑ฏ๓นฉ๑ฆ ๓ผน๐ฆผ๓นฒ๑๑ ๑ฝ๔ค๑ค๓๑ผ๔ผ๓ฎฏ๐ฏ๓ฆ๑ตต๒ฏ๓ฉ
๒๒ฏ ๑ฌ๋ก๑บ๑ตบพ๐งจ๑พ๒ฌ๓ทฎ๓ ขง๒ฆผ๐นฅ๓ฎพ๒ผข๔๒ธฌ๓ฎฃ๒ชพ๒ฌท๑ฏ๑๓ญค๓ฐฅ๑ฎ๓ฒซ๓ปป๒ปฌ๔๒ญ๒ซ๓จถ๓ฝ๒นน๒ผนจ๒จต๑ฐ
๑ปฃ
๑๔ฅฉ๑ต๑ถ๐ฌ้ญ๑ฆน๓คต๐ฆช๐ฎก๓ฎฝ๐ ๑ฅฝ๔ฅฆ๔ฒน๓ก๒๐ต๐ณฉ๋ฒ๓ฌฅ๒ถฐ๑ททข๒ฌ๑๒นช่
๓ญ๒๒ฒ๓บฟ๑ฒฅ๐นฏ๓ฉฌ
๑พง๓ซฎ๒ฟ็ชฏ๑จ๔ฒถ๑ฝถ๒ฉชฃ๒นก๔๒๒๐ ทบ๑ฆ๑ญญธ๑ฝ๐๐จบฟ๒บ๐ฉ๔ฝ๑ธถ่ผฌ๑ธ๑
๑ขณ๑ง๑๐ณ๐ช
๐ฅน ๐ผณ๓พข๐
ฟ๔ญท๑พฃ๒๑ญณ๒ธ๓๒ถฃ๐ฟ๔ข๓ถฟ๐ฒ๑ช๑๑ฒปฆ๑ญฉ๓ณ๐พบฒ๐ฏ๑ธ๑ผ ๑ญณ๑ท๒๔๐ธ๓ฟฒ๒๐ฐคซ๒ซฅ๒ผฒ๐ฅธ๑ฅพฅ๓ฎฟ๒ณฑ๐๓ณ๔ จ๐ณ๓ท๒๒ฉฐ๓บฑ๑ฐๅดณ๓๐ฏป๒ฟฎฅ๐ฏ๑ฅป๓ฟญ๒ซ๑ฆฝ๑ ท๐ธ๑ตณณ๓ฟงป๔ฏ๓น๒ชพ๓ฝ๓ตท๒ซ๑ธ๓ณ๑ถ๑ฅตพ๓ง๒ฎซฐ๐ช๑ผ๔พข๐ถ๒ฎด๐ฐผ๒๑บฟญ๒ฃธ
๒ต๒ข๒ฏช๑พ๐งฅ๓ผ๒ฌฟ๓ฐ๔ฏ๒๑ ค๑ฌด๐ฒ๓นฉ๒พฃ๑ฐข๓ญฏ๑ ต๑นฆ๓ณผ๑กซ๓บฒข๑พฅ๑ฟท๓พฃๅ๓ฝ๒ผช๒ฐญ๑ ฎบ็ฆน๓พฃฃ๐ผต๒น๑ช๓ซจฌ๒ซ๓ฆ๑ซ๓ฌฟ๒ป๑ฏถ๓๐ท๐ฒฆ๓ปด๑ฎ๒ถฐ๑ฅข๑ฐตช๑ข๒ปขๅ๓ขท
๓ฝ๑๑ผ๒ญถธ๔
๓ญ๐ฏ๐จ๓ถง๒ชฟ๐จฝ๒๐ฑขผ๒ฉฑผ๒ซฎด
๐ปฅป๒ฟ๓ฐ๐ฑท๐๐ผ๔ผ
๐ฒ๔๓๔ซข๐๑บ๐ฌฝ๒ข๎ญ๒ ฌ๔ธซ๔ต๐ฎ๓ฅบ๔ถ๒ตค๐บ
๑๑ฅ ๔ฝ๓นซ ๒กง๓ธก๒ฐ๒ฐ๑ฅ๑ฃ๐ฉ๒ฐ
๒ตฏ๑ฃ๓ข๒๒จธ๐พ๐ท
ฃ๐ตผค๐ฆ๒๐ฝฃ๒ด
๑๑ถด๓ซ๑นฎ๑ฃฏ๐ตฃ๓พฃ๒ด๓ฟ๐ฆถ๓ฏซ๓๑ณฎ๑ถจ๓กทฃ๐ธซ๐ณ๑ช๓ฎถ๐บพใฎ๑ก๓ถญ๏ฏ๓๓ฟงบ๔๓ค๓พ๒ฃฆ๑ฆ๒ฏ ๔๑ ๒บฑ๐ฆ๑ฃค๓ขง๒ฑ๑ฝณ๓ถฑฎ๓ถผ๓ฟ๒พ๒ตต๒ผซ๓ฌ๒๓ฑฆก๑๒ฎ๓ผ๒ธ๓บบ๐จฝ๓ฒ๑ถ๓ฟญ๓ฃ ฏ๒ฑฉ๑ฐซๆ๑ณฅ๑ผทต๐ ๑ป ๓ฝพ๐ฟ๓๑ฏ๒ฆ ๐ฒ
๐ณธ๒ธด๎ค๓ฉ๑ฝฐ๑จช๑๒คต๓ณท๐ฑฒ๒ต๓ถต๑ก๓ก๒ฎต๑ฌ๑ซกต๑ ฌณ๑ฌถ๐กฑญ๑
ผบ๒พ๑ฌ๊บ๐น๐ตใญ๑ญ๔ซ๓๑ข๒ฝฒ๑ฟจ๑ฐ๑ฟด๒ขฃ๔ฃ๒ฐฎ๓ฟฑ๓กฃ๓น๐ฌฅฟ๑๐ญฆ๒ก
๓ฐบฟ๓ถ๐ฌท๓ฑ๓ฌ๓พชข๑๑ฎ๑ง๒พฝผ๑จคณ๐ธพ๐ฃ๓ธ๓นด๐ฌ๐ผฆ๔ทฑ๒ฉค๐ธ๑ฅบผ๑ฑ๑ฑ๓ฒฃ๐ดจ๓ดน๐นช๓๓จด๒๒ฟต๑ซ๓ฏ๒ฏฒ๑ต๑ข๓นคจ๒ข๑จฉ๒ ๑ปจธ๐ปฆ๓ฆฉ๑ฌชธ๓ปดด๑๒ฉ๑ผ๒ฃ๓ต๓ฃนฅ๒ญ๒ฆ๑บฒ๑๓ฎ๑ซ๐ฒน๒ผก๒ทง๑ฟฉฌ๓ญ๒ช๐๑ป๓ซจ๐กฑใ
๒ผก๓ฏปญ๔ค๑๓ธดญ๓ฐ๑ฑฑํฆ๓๐ขป๒ท๑ฎต่ช๓ฃข๐ซญช๐ฒบ๓ฟ๑
๒ฟง๓ฟด๑ซฑ่กฏ๒ญ๒ดา๓ด ๐ฑฏ๑ฐ๒๑ฎจฌ๓ชพช๒ฎ๐พจ๑ฑพ๑ก ํข๒ก๒ฝถ๒ดธ๐
ก๒๐ ฎ๓ถ๒ก๓ฟ๔ ฐ๒ฟข๑จฌ๐ฏขฌ๒ญซ๔๑ฎ๓๑กค๐ฎบ๑ซ๐๑ทญ๐ชข๒ฌฃ๒กก๑ก๒๒ถ๐ฆทด๑ถต๒ฎต์ฅ๐ฒช
๑ฟ๐ฆ๒ตป๓ ๓ผพ๐จ๓นฒ๐ธฏ๐ฆๆท
๓ตฎ๐ฐ๑ฆ๓ซ๓ฒ๓๓นฎ๑งท๑งต๑ ถ๒นซ๓ฐ๒๔ข้ฎ๒ฃป๑ฉน๐ฑ๐ฅฅซ๑ฉ
It fails (or takes long) to find simple case like a when something starts with a space, special character or like this.
I think normal characters should be preferred to deeply Unicode things.
I think by default there should be tiers of characters classes:
And character generator may aim for, for example, 10% of buggy chars, 40% of [a-zA-Z0-9_], 40% of basic Unicode characters, 10% of everything else.
I have another question about writing QuickCheck tests.
My function under test, edit_distance
, has another property, which I want to quickcheck.
It is zero if and only if the strings are equal.
This requires that both parameters reasonably often are equal. I have a naive implementation here:
febeling/edit-distance@30532c2#diff-475a9530a9c1e888d097445199b74c2fR75
But when I run it and prepare the code a bit, I have seen that the the condition 'strings are equal' is never met. I guess that's a common problem in quickcheck test. Is there a good way to deal with this case?
The README.md links to http://burntsushi.net/rustdoc/quickcheck/fn.quickcheck_config.html which now 404s. Where is configuration documented now?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.