greg4cr/labelsearch

Search-Based Generation of Label-Covering Test Cases for C Programs

labelsearch's Issues

Recursive generation of structs

Structs often contain pointers to other structs. Those may contain further pointers. When do we stop generating? We need some form of depth control.

For generation - do a coin flip - do we go another level down?
Maybe keep a counter, have a max gen depth? Adjust probability as we go down.

During mutation, we could add another layer, null out an existing layer, or mutate an existing layer.

Compatibility with Check?

Check is a popular unit testing framework for C programs: https://libcheck.github.io/check/

Should LabelSearch tests be compatible with Check?

Generator does not produce correct input for structs that contain structs

Traced down to the fact that structs return in the form:
//structInput69107: struct tagEntry structInput69107 = {.key = 221, .value = 99};

However, the struct generator also dislikes returns with "//" in them. Easy to fix, but can't get to it today. Making a note to remember.

String generation

Currently, char* results in a single character. We should support generation of full strings.

Revise test case representation to support mutation

We should make it easier to mutate test cases by isolating mutatable elements. Also consider how to identify those when importing a file.

Generator fails to handle function definitions that return pointers or complex types

File "LabelSearch/generation/DependencyGraph.py", line 114, in visit_FuncDef
function.append(node.decl.type.type.type.names)
AttributeError: 'TypeDecl' object has no attribute 'names'

Change how non-compiling obligations are handled

Obligations that do not compiled when instrumentation is performed are currently replaced with constants. Instead, they should be replaced with flags (met/not met instead of continuous score).

Generator fails to properly handle structs with pointers as members

AttributeError: 'Struct' object has no attribute 'names'

Lack of support for extern/__inline

The C parser, pycparser, does not correctly parse functions marked as extern or __inline. However, the C filtering done by frama-c often leaves these in the file. This is obviously not ideal.

Options:

• Change how we parse C.
• Perform some kind of pre-processing to alter or remove these. We need them for test execution, but they should be ignored during generation anyways.
• Look at whether we can alter the output of frama-c.

Function calls that reference function pointers are not recorded properly

Functions of the type (*PrintFunc)(...)
The arguments matter for the simple def/use analysis. However, right now, they cause the parser to crash.

Support for enum type

LabelSearch does not currently support the enum type. This should change.

Gather code examples

Ideally with requirements too.

Test generation needs to know how to handle const/volatile.

Currently, the generator just states that it can't handle the type for, i.e., const char.