westerndigitalcorporation/stat

Standalone unit-testing framework for software written in C

stat's Issues

BUG: Get-Spy-Data API returns NULL instead of failing if no call was really performed

If no call to a certain derived test-double is performed, every API that returns on-call statistics (e.g. spy-data) should fail on read attempt instead of returning any data.

STAT: Add additional user-case examples to improve documentation and to test corner cases

Additional examples of user-case setup along with some corner case scenarios might provide a good improvement for documentation and also as additional tests for handling of different corner cases.

Might consider to split this task to two separate efforts/tickets.

Incremental build fails in certain scenarios

Testing the latest version (1.2.0) on a real product revealed the following issues:

• Incremental build failed due to overkill checking-option (/WX) that was added for linker
  • It caused the makefiles with duplication of source declarations to fail due to duplication of object files
• There is a bug in the logic responsible for extraction of dependecies that are used by incremental build
  • The compiler command-line doesn't include definitions specified for the test-packages, causing it to fail
• A workaround is needed for certain cases of bad-syntax encountered in user-makefiles

Makefile parsing inconsitency with comments

If a comment is interleaved with the values of a macro defintion, makefile parser doesn't ignore its contents.
This behavior conflicts with IDE-writers, since it attempts to treat the comment as a valid entry and fails on it.

For instance:

SOURCES = \
  file_x.c\
  # file_y.c\   <--this comment will be treated by parser as a file, and it shouldn't
  file_z.c

Ocasional failure in incremental build

NMAKE shows an inconsistent behavior in dealing with assignment of values to environment variables. This caused to invariant behavior of the incremental build, which is affected by the amount of times certain environment variable is set (not as part of a batch file). Therefore, sometimes incremental build works correctly, but sometimes it fails.

Migrate from NMake to GMake

GMake (a.k.a. GNU Make tool) is a cross-platform Make tool. Migrating to it, will simplify the procedure of entending the framework to platforms other than Windows.

Make: Critical issue with dummy interfaces

A new quite critical bug has spotted within makefiles that disturbs it work:

• If there is a dummy header file within the dummies folder any test-package that needs a production version of this file will end up using the dummy version and crash

MOCK: Modify NUMERIC-type macros to operate natively with negative/float values too

We used STAT_ADD_NUMERIC_MOCK to mock a variable that is from float type.
For some reason, we didn't get the correct value.
after a debug session, we saw that the converting from float to unsign was incorrect, and required the same method like in STAT_ADD_MOCK when the value is copied as void.

Adjust the framework to make it run on Linux

MSVS: Compilation errors are not clickable in Visual-Studio GUI

In the output window of Visual-Studio when one clicks on a link with the filename and the line of a complication error, the IDE fails to open the location of the file.
Instead, it fires an error that the file cannot be found:

Make: Performance degradation on Windows platform

After the framework has migrated to GMake tool, a new design of the core makefiles causes a significant performance degradation on Windows platform.

Support for directory view when using VS 2019 is missing / not working

makestat -vs creates a vs 2019 solution, when vs2019 is installed on the computer.
Currently, this solution is not opened with correct directory view - to show the files arranged by their actual folder. instead files are presented flat on the solution explorer.

BUG:FWK: Framework doesn't fail test-package on complication fails if running incremental build

If the previous framework run for a test-package succeeded to compile and to pass, then the subsequent incremental run won't fail in case the compilation fails, but will run previously compile test-package executable.

Temporary workaround: add a '-cc' command-line argument to each framework re-run.

Add CI workflow for regression on Linux platform

Add some IDE support for debugging on Linux platform (e.g. VS Code)

STAT: Add support for standalone MS Build Tools

Today, on Windows STAT supports full MSVS variations (i.e. build tools with IDE). To support better DevOps and lower expenses it has been requested by some users to add support standalone MS Build Tools (i.e. w/o IDE).
This is especially relevant now, when a support for VSCode has been added to the framework.

Pass framework unit-tests on Linux

STAT: Support for incremental build

The current version of NMAKE script is very basic that rebuilds each test-package in full.
It would be nice to add the framework ability to provide several options of building:

• incremental build
• rebuild
• clean and rebuild (currently the only target that issued by default)

Add support for GCC on Linux

To enable the framework on Linux it must be able to compile on Linux. GNU Compiler was chosen as a default one for Linux platform.

Produce symbolic link to source

Add a support to STAT that will produce symbolic link to source to some dedicated folder (like I do with the headers) and compile them from there, this isolating source files from header files.