clar-test/clar

What tests are made of.

clar's Issues

Run preprocessor before processing test declarations

Right now it's not possible to use an #ifdefs to let certain tests compile certain tests conditionally, because the .c file is processed as-is, before running it through the C preprocessor.

clar didn't support for python3

Idea on output

Wouldn't it be a bit better to use colored ● (green - pass/red - failure/yellow - skipped)?

Selecting a suite should look in all categories

If you run ./libgit2_clar -s49 where that suite is part of the "network" category, it won't find it unless you also add -iall or -inetwork. If I know which suite I want to run explicitly, it shouldn't matter which category it's in. As the while suite belongs in one category, it's not like I can use it to run suite 49 except for the network tests.

So I think -s should imply -iall internally. I might take a stab later, for now I'm just complaining.

Print back trace upon failure

It'd be useful when clar would print out a stack trace that lead up to the failed test.

clay.py is not executable

The README instructs to use ./clay.py , but it does not have the executable bit.

Problems getting started with clar (stale documentation)

I'm a bit confused on how to get clar set up in a project of mine.

The "mixer" (clar.py) was removed in 161e8cd but is still referenced in the documentation. Is the generate.py script analogous to clar.py?

It was mentioned in a comment that the documentation is out of date. Once I get more familiar, I would love to help.

Thanks,
Chris

Specifier for a precise match using `-s`

When using -s, it will match a prefix. For example -sfoo will match tests that start with foo. This is useful for matching categories like -sfoo::one, -sfoo::two, -sfoo::three, etc.

However, sometimes you want to be able to provide an exact match because you may have a test named foo::do_this_thing and foo::do_this_thing_and_another.

Now you have no way to run foo::do_this_thing since the prefix match will find both.

Now, this suggests a poor naming convention but test users may not be the app developer, and we should support them. Add a way to literally specify a thing, maybe using a $ at the end, eg -sfoo::do_this_thing$.

clar didn't support for python3

b'#ifndef __CLAR_TEST_H__\n#define __CLAR_TEST_H__\n\n#include <stdlib.h>\n\nvoid clar__assert(\n\tint condition,\n\tconst char *file,\n\tint line,\n\tconst char *error,\n\tconst char *description,\n\tint should_abort);\n\nvoid cl_set_cleanup(void (*cleanup)(void *), void *opaque);\nvoid cl_fs_cleanup(void);\n\n#ifdef CLAR_FIXTURE_PATH\nconst char *cl_fixture(const char *fixture_name);\nvoid cl_fixture_sandbox(const char *fixture_name);\nvoid cl_fixture_cleanup(const char *fixture_name);\n#endif\n\n/**\n * Assertion macros with explicit error message\n */\n#define cl_must_pass_(expr, desc) clar__assert((expr) >= 0, __FILE__, __LINE__, "Function call failed: " #expr, desc, 1)\n#define cl_must_fail_(expr, desc) clar__assert((expr) < 0, __FILE__, __LINE__, "Expected function call to fail: " #expr, desc, 1)\n#define cl_assert_(expr, desc) clar__assert((expr) != 0, __FILE__, __LINE__, "Expression is not true: " #expr, desc, 1)\n\n/**\n * Check macros with explicit error message\n */\n#define cl_check_pass_(expr, desc) clar__assert((expr) >= 0, __FILE__, __LINE__, "Function call failed: " #expr, desc, 0)\n#define cl_check_fail_(expr, desc) clar__assert((expr) < 0, __FILE__, __LINE__, "Expected function call to fail: " #expr, desc, 0)\n#define cl_check_(expr, desc) clar__assert((expr) != 0, __FILE__, __LINE__, "Expression is not true: " #expr, desc, 0)\n\n/**\n * Assertion macros with no error message\n */\n#define cl_must_pass(expr) cl_must_pass_(expr, NULL)\n#define cl_must_fail(expr) cl_must_fail_(expr, NULL)\n#define cl_assert(expr) cl_assert_(expr, NULL)\n\n/**\n * Check macros with no error message\n */\n#define cl_check_pass(expr) cl_check_pass_(expr, NULL)\n#define cl_check_fail(expr) cl_check_fail_(expr, NULL)\n#define cl_check(expr) cl_check_(expr, NULL)\n\n/**\n * Forced failure/warning\n */\n#define cl_fail(desc) clar__assert(0, __FILE__, __LINE__, "Test failed.", desc, 1)\n#define cl_warning(desc) clar__assert(0, __FILE__, __LINE__, "Warning during test execution:", desc, 0)\n\n/**\n * Test method declarations\n */\nextern void clar_on_init(void);

Tap protocol output

Feature request: reporting (e.g. jUnit xml)

It would be nice if clar could generate a structured report of the ran tests, e.g. jUnit format.