wc-duck / datalibrary Goto Github PK

View Code? Open in Web Editor NEW

42.0 42.0 8.0 2.64 MB

Open Source Data Library for data serialization.

License: Other

Python 0.59% Lua 1.79% C 16.63% C++ 80.22% Batchfile 0.06% Shell 0.02% CMake 0.68%

datalibrary's People

Contributors

Stargazers

Watchers

Forkers

matricks zeyelth jameslinus skyformat99 lundmark tisten elementalandy

datalibrary's Issues

Typelibrary compiler fails if last line is not blank

If you have your typelibrary definition that ends with a line that has the ending scope in it (a simple line with the last } on it) the typelibrary compiler fails with:

50> expected '{' but got '@' at 114
50> at line 0, col 0:
50> @�n��
50> ^
50> failed to load ty

This should either be checked and given a proper error rather than printing a bad buffer, OR it should be accepted.

Defaults for entire subtypes does not work

{
    "module" : "test",
    "types" : {
        "mytype_1" : {
            "members" : [
                { "name" : "submember1", "type" : "string[]" },
                { "name" : "submember2", "type" : "uint32" }
            ]
        },
        "mytype_2" : {
            "members" : [
                { "name" : "member1", "type" : "mytype_1", "default" : { "submember1" : [ "apa" ], "submember2":1337 }}
            ]
        }
    }
}

Should work but the string-array for submember1 seems to be set incorrectly.

Add json5 support in text-format

http://json5.org

Assigning a pointer in dl-txt to 0 does not work as expected

There should be an error that DL doesn't find the pointer pointed to (since 0 is never used)

error C2024: 'alignas' attribute applies to variables, data members and tag types only

Getting this compile error with VS 2015 on the following generated code:

struct resource_array
{
#if defined( __cplusplus )
    static const uint32_t TYPE_ID = 0xE9279B86;
#endif // defined( __cplusplus )

    struct
    {
        const char** data;
        uint32_t count;
    #if defined( __cplusplus )
              const char*& operator[] (size_t index)       { return data[index]; }
        const char*& operator[] (size_t index) const { return data[index]; }
    #endif // defined( __cplusplus )
    } file;
    struct
    {
        DL_ALIGN(8) uint64_t* data;
        uint32_t count;
    #if defined( __cplusplus )
              DL_ALIGN(8) uint64_t& operator[] (size_t index)       { return data[index]; }
        const DL_ALIGN(8) uint64_t& operator[] ( size_t index ) const { return data[index]; }
    #endif // defined( __cplusplus )
    } id;
    int32_t max; 
};

Specifically lines like this:

DL_ALIGN(8) uint64_t& operator[](size_t index) { return data[index]; }

dl_tests_dl.cpp is getting to big for MSVS!

break the file into smaller parts.

Add support for specifying structs as arrays.

Currently all instances in the text-format is specified as structs. This is the best solution in most cases but for example vec3:s etc is really cumbersome.

Today that would be:
{ "x" : 1, "y" : 2, "z" : 3 }

Try that with a mat4x4 and you will just give up ;)

By adding support for parsing structs from arrays one can get a lot nicer.

[ 1, 2, 3 ] or mat4x4
[ 1, 1, 1, 1,
1, 1, 1, 1,
1, 1, 1, 1,
1, 1, 1, 1 ]

This should be possible by just adding the rules
if an array is encountered where a struct is expected the array will be parsed as if the members where in order in the array and all members need to be set.

dl_instance_store instance pointer should be const void *

Same goes for dl_util_store_to_file etc.

Enums as members does not compile in c

{
    "module" : "stuff",
    "enums" : {
        "my_enum" : {
            "THIS_IS_ENUM" : 0
        }
    },
    "types" : {
        "my_thing" : {
            "members" : [ 
                { "name" : "this_is_it", "type" : "my_enum" }
            ]
        }
    }
}

generates a struct that looks like this:

struct my_thing
{
    my_enum this_is_it;
};

Which unfortunately does not compile in c. I assume that the same thing happens when you do the same thing with structs. I'd suggest either adding the "enum" / "struct" keyword to the types (preferably) or do the c-style of declaration with typedef enum my_enum {} my_enum;

Arrays of struct-pointers becomes static pointers.

            "members" : [
                { "name" : "mystuff", "type" : "previously_declared_struct_type*[10]" },
            ]

generates the following code:

const struct previously_declared_struct_type* mystuff;

Is this really intended? I would assume that we would get this:

struct previously_declared_struct_type* mystuff[10];

dl_tlc should not open output file until input file has been parsed

Otherwise it cleans / overwrites the output file even if the input file is corrupt / not possible to parse.

readme I horribly out of date!

Arrays of structs with string-arrays as members broken!

Creating type a with a member as a string array and then creating an array of these breaks the strings.

example:
"type_a" : {
"members" : [
{ "name" : "strings", "type" : "array", "subtype" : "string" }
]
}

"type_b" : {
"members" : [
{ "name" : "structs", "type" : "array", "subtype" : "type_a" }
]
}

DL fails to pack/load specific file

I have the following type library and data file

https://gist.github.com/joeldevahl/91a6c4a3568531cc3053

and doing a dl_pack on cube.mesh with the compiled type library gives me a crash in dl_internal_store_string in the strlen.

datalibrary is at commit a59f48e

dl_reflect.h has a bad interface.

The current reflection interface is far from good and should be redone. Today the interface forces the user to manage memory by them self when a simpler interface would work just as well.

Example:

dl_reflect_type( dl_ctx_t ctx, int type_index, dl_type_info* info );

Add support for array and inline array of pointer.

Add support for "apa_[]" and "apa_[6]"
Prerequisite is to rewrite json-parse since that would simplify this feature ALOT!

Data deduplication support

It would be nice if DL would support data deduplication when serializing as a user flag when the user does not care about data being unique.

Add some code-coverage metrics.

And integrate with coveralls?

Expose enum type declarations

such as this:

enum MyTestEnum : uint8_t {
MY_TEST_ENUM_VAL = 1,
};

Including defined struct-types in arrays generates invalid c-code

The following json:

{
    "module" : "test",
    "types" : {
        "test_type" : {
            "members" : []
        },
        "test_type2" : {
            "members" : [
                { "name" : "test_instance", "type" : "test_type[]" }
            ]
        }
    }
}

Seems to generate the following struct declarations:

struct test_type
{
#if defined( __cplusplus )
    static const uint32_t TYPE_ID = 0xE077D27A;
#endif // defined( __cplusplus )

};

struct test_type2
{
#if defined( __cplusplus )
    static const uint32_t TYPE_ID = 0xFBE19D9B;
#endif // defined( __cplusplus )

    struct
    {
        test_type* data;
        uint32_t count;
    #if defined( __cplusplus )
              test_type& operator[] ( size_t index )       { return data[index]; }
        const test_type& operator[] ( size_t index ) const { return data[index]; }
    #endif // defined( __cplusplus )
    } test_instance;
};

Where the type-pointer in the sub-struct for the array-data says "test_type* data" which should probably be "struct test_type* data;". This does not compile in c. It does compile in c++

Feature: custom pointer types

I think it would be really nice if you could do this in your type declaration:

{ "name" : "blah", "type" : "struct mystruct*[5]" }

This means that the struct can hold a list of pointers so that you don't have to do void*'s and can keep type safety somewhat reasonable.

Crash. Packing strings.

I have the following types:

"dl_material_mapping" : 
{
    "members" :
    [
        { "name" : "material", "type" : "string" },
        { "name" : "submesh", "type": "int8" }
    ]
},
"dl_model" : 
{
    "members" :
    [
        { "name" : "mesh", "type": "string" },
        { "name" : "materials", "type": "dl_material_mapping[]"}
    ]
}

Now when I pack the following data file dl_pack.exe crashes.

{
  "type": "dl_model",
  "data": {
    "mesh": "1234",
    "materials": [
      {
        "submesh": 2,
        "material": "apan.ymesjh"
      }
    ]
  }
}

If I change the line

"mesh": "meshname",

"mesh": "meshna",

it doesn't crash.

Feature: enum-values as size on static arrays

This would be very nice

{
    "module" : "my_module",
    "enums" : {
        "some_enum" : {
            "SOME_THING",
                        "SOME_THING2",
                        "NUM_THINGS"
        }
    },
    "types" : {
        "my_type" : {
            "members" : [ 
                { "name" : "some_things", "type" : "some_enum[NUM_THINGS]" }
            ]
        }
    }
}

So that if you go in and modify the enum, the result will still be good and not missing to increase the value on your static array.

Add support for unions.

Support for union types has been requested from both users and myself, so that should be added. Here is my suggestion for interface.

typelib:

    {
        // add new section "unions" to typelib.
        "unions" : {
            "my_union_type" : {
               // this would be the actual name of the union, see below. ( not the best name for the key =/ )
               "name" : "v", 

                // defined as dict, it could also be defined as a [] to match "types"?
                "members" : {
                    "member_1" : "other_defined_type1",
                    "member_2" : "other_defined_type2"
                }
            }
        }
    }

this would generate this type-header

struct my_union_type
{
    dl_typeid_t type; // type currently present in union
    union
    {
        other_defined_type1 member_1;
        other_defined_type2 member_2;
    } v;
};

However I have no really good idea for how to specify this in the text format yet =/ Lets say that we have a type with a member "union_member" of the above defined type, like this:

struct the_type
{
    union_type union_member;
};

how would that be specified in text-data ( how would the text-data specify what member to actually initialize? Maybe like this?

{
    "the_type" : {
        "union_member" : {
            "member_1" : {
                // ... other_defined_type1 data here ...
            },
            // ... setting more than one member would be an error ...
        }
    }
}

Generate accessors for union-types in headers?

Investigate if it is a good idea to, when compiling as c++, add, optional to use, accessors for union types to get and set members.

something like:

    enum my_union_type
    {
        my_union_type_item1 = ...;
        my_union_type_item2 = ...;
    };

    struct my_union
    {
        union
        {
            int32_t item1;
            struct apa item2;
        } value;
        my_union_type_item1 type;

       // things above is already generated.

        int32_t& item1()
        {
            ASSERT( type == my_union_type_item1 );
            return value.item1;
        }

        int32_t& as_item1()
        {
            type = my_union_type_item1;
            return value.item1;
        }

        struct apa& item2()
        {
            ASSERT( type == my_union_type_item2 );
            return value.item2;
        }

        int32_t& as_item2()
        {
            type = my_union_type_item2;
            return value.item2;
        }
    };

Biggest questions:

is this a good idea? ( I think so )
are the names ok, is there any better names?
how to handle the ASSERT() since that is something that you want to define as a user and I have no good idea on how to inject the def.

Usage of union in type is valid in dl, but not in generated header

Using a union in a type is valid in DL but generates a header where the union type is declared after the struct that uses it.

I can work around this by using a pointer to the union in the type but preferably I would like it built in. One solution would be to sort the types and unions on dependencies (types and union only depending on PODs first etc.)

Support multi-dimensional arrays

Add support for multi-dimensional arrays such as "uint32[][]", "uint32[5][]" and uint32[][6]".
Prerequisite is to rewrite json-parse since that would simplify this feature ALOT!

DL does not generate errors for "old" definitions of types.

If I try to feed dl with an instance generated from an old header-file (mismatch sizes/members etc), then DL does not generate an error. Instead it freezes. Some sort of version-tagging would be nice.

Add "tag" support on structs, members, enums and enum-members

It should be possible to add "tags" to each of the above mentioned parts of a typelibrary for the user to use as they see fit. ( mostly for editor/tools use ).

A tag should be any string and dl will put no meaning to these. On each of the aboved items a list of these tags should be valid.

Tags use-case examples:
"file" to indicate that a string should be a file
"range(0,100)" to say the range of an int.
"slider" should be edited by a slider in some editor.
Note: This will NOT be implemented by dl, only the tag-support.

strtof() multiple defined symbol at linktime

It's a part of the standard library and simply defining it for all msc-versions won't work on msvc2015 (at least).

Comments to types and members

Since we parse our header-files to extract certain information and generate things, it would be very beneficial if we could add comments to types and members. Such as:

// my_comment_1
enum A {
...
};

// my_other_comment
struct struten {
float mymember; // my_member_comment
};

Pointer patching in dl_instance_load fail on sertein conditions.

A struct with the format:

struct Parent
{
struct Child
{
some_type* ptr;
}
}

ptr will not be patched since both parent-instance and child-instance has the same offset in the stored data.

Defaults do not propagate down into subtypes

Example:

{
    "module" : "test",
    "types" : {
        "mytype_1" : {
            "members" : [
                { "name" : "submember1", "type" : "string[]", "default" : [ "apa" ] },
                { "name" : "submember2", "type" : "uint32" },
            ]
        },
        "mytype_2" : {
            "members" : [
                { "name" : "member1", "type" : "mytype_1", "default" : { "submember2":1337 }}
            ]
        }
    }
}

Fails on two parts:
1 The default for member1 in mytype_2 tries to apply a default-value that is the size of the entire member.

2 Defaults for the type mytype_1 is never applied to member1 in mytype_2

Rework how pointer-data is represented in text-data.

Currently pointers in text data is an integer:

0 - root-item
1 > items in the sub_data-section where each key is the matching number as a string. Such as

"subdata" : {
"1" : { ... },
"2" : { ... },
}

The bad thing here is the discrepancy between string and int depending on position in text-data and its hard to read.

My idea is to replace pointer-data as strings all over the board, so when setting a pointer-member it becomes:

{
"my_ptr_member1" : "my_first_sub_ptr",
"my_ptr_member2" : "my_second_sub_ptr",
"my_ptr_member2" : "root"
}

and in subdata

"subdata" : {
"my_first_sub_ptr" : { ... },
"my_second_sub_ptr" : { ... }
}

One potential problem with this, but some thing I accept, is that the pointer-names will not be kept when unpacking, so when doing a dl_txt_unpack() the pointers will be named "ptr1", "ptr2" etc.

Investigate if going towards hjson is a good idea.

https://hjson.org/

There has been some requests of "loosening" the json input to DL and I can absolutely understand why.

The only thing that I do not like about it is that by doing that DL will not be readable by "any conforming json parser" such as pythons json module etc. However that might not be a problem and it is worth more to have a simple format for humans.

The format would still be writable by any tool so that should be fine.

If there is anyone reading, feel free to comment :)

Using an undefined type results in unpretty error message

{
    "module" : "example",
    "types" : {
        "my_type" : {
            "members" : [
                { "name" : "x", "type" : "float" }
            ]
        }
    }
}

Yields

Traceback (most recent call last):
  File "src/external/dl/tool/dl_tlc/dl_tlc.py", line 91, in <module>
    tl = dl.typelibrary.TypeLibrary( options.input )
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 313, in __init__
    self.read( open(lib, 'r').read() )
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 324, in read
    self.from_text( lib )
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 412, in from_text
    self.__calc_type_order()
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 379, in __calc_type_order
    if ready_to_remove( temp, type ):
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 362, in ready_to_remove
    base_type_name = member.type.base_type().name
AttributeError: 'NoneType' object has no attribute 'base_type'

Struct order bug in generated header file.

The following tld file generates a c-header file where the structs are place in an incorrect order which leads to compile errors.

{
"module_name":"test_model",

"module_types":
{
    "vertex_element_desc" :
    {
        "members" :
        [
            { "name" : "usage", "type" : "string"},
            { "name" : "format" , "type" : "uint32"},
            { "name" : "offset" , "type" : "uint8"},
            { "name" : "size" , "type" : "uint8"}
        ]
    },
    "vertex_buffer" : 
    { 
        "members" : 
        [ 
            { "name" : "vertex_count",  "type" : "uint32" },
            { "name" : "stride", "type" : "uint32" },
            { "name" : "data", "type" : "array","subtype":"uint8" },
            { "name" : "elements", "type" : "array", "subtype": "vertex_element_desc" }
        ] 
    },
    "index_buffer":
    {
        "members" : 
        [
            { "name" : "index_count", "type" : "uint32" },
            { "name" : "data", "type" : "array","subtype":"uint16" }
        ]
    },
    "sub_mesh_desc":
    {
        "members" : 
        [
            { "name" : "index_buffer_offset", "type" : "uint32" },
            { "name" : "primitive_count", "type" : "uint32" },
            { "name" : "vertex_buffer_offset", "type" : "int32" },
            { "name" : "verex_count", "type" : "uint32" }
        ]
    }
}
}

Malformatted JSON results in unpretty error message

Extra , or missing , for example.

{
    "module" : "example",
    "types" : {
        "vec3" : {
            "members" : [
                { "name" : "x", "type" : "fp32" },
            ]
        }
    }
}

Yields

Traceback (most recent call last):
  File "src/external/dl/tool/dl_tlc/dl_tlc.py", line 91, in <module>
    tl = dl.typelibrary.TypeLibrary( options.input )
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 313, in __init__
    self.read( open(lib, 'r').read() )
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 324, in read
    self.from_text( lib )
  File "/home/kma/projects/demo13/src/external/dl/tool/dl_tlc/dl/typelibrary.py", line 398, in from_text
    data = json.loads( re.sub(pattern, replacer, lib ) )
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Enum feature: shifted values

Enum feature: shifted values (1ULL << 34ULL) etc.

Empty dl-definition fails with bad warning

Warning is:
50> expected '{' but got 'P' at 114
50> at line 0, col 0:
50> P��
50> ^

Should either not allow definition and print a more appropriate error or produce an empty typelibrary.

DL Generates none-C-headers.

Enums and structs are not on-par with the C-standard for how to define / use types. Example:

struct myStruct {};

struct myStruct2 {
myStruct m_a;
}

Is incorrect. The two solutions are: The struct "myStruct" must be typedef'ed to myStruct, or the m_a variable must be typed as struct myStruct. The same goes for enums.

Storing DL_TYPE_ATOM_INLINE_ARRAY with DL_TYPE_STORAGE_STRUCT's doesn't correct pointers

So if you write an atom inline array of structs with members that have pointers (such as strings), the storage fails to redirect the pointers to offsets.

References to other values in enums

References to other values in enums, such as:

enum A {
B
};

enum C {
D = B,
};

Running dl_tlc.exe with a library consisting of an enum and an empty type crashes

This test case crashes the dl_tlc when running it as:
dl_tlc.exe -c -o output.h input.json

{
    "module" : "component_resources",
    "enums" : {
        "damage_types" : {
            "DAMAGE_SLASHING" : 0,
            "DAMAGE_CRUSHING" : 1,
            "DAMAGE_PIERCING" : 2,
            "DAMAGE_HEAT" : 3,
            "DAMAGE_COLD" : 4,
            "DAMAGE_ELECTRIC" : 5,
            "DAMAGE_CORROSIVE" : 6,
            "DAMAGE_MENTAL" : 7,
            "DAMAGE_POISON" : 8,
            "DAMAGE_RADIATION" : 9,
            "DAMAGE_NUM_TYPES" : 10
        }
    },
    "types" : {
        "damage_component_resource" : {
            "members" : [
            ]
        }
    }
}

Rework pointer-patching on 64-bit.

Currently pointers are stored in packed data as the offset from instance. This could be reworked to make patching a lot faster and simpler.
Currently patching is done by traversing the struct with stored type-info.

If the storeage instead was stored as x bits offset and y bits offset-to-next-pointer a simple loop as this could be used to patch pointers:

uint64 iter = instance->first_ptr;
while( iter & NEXT_MASK )
{
patch_pointer( iter, iter & OFFSET_MASK );
iter = iter + ( iter & NEXT_MASK );
}

this will however force instances to be less than 4GB on 64bit and less than 65kb on 32bit.

To begin with I'll save the old parse-code, set a flag in instance how the pointers is patched, and dispatch depending on that flag.

enums not supported in python-bindings

As said above enums are not yet supported in python-bindings.

Setup autobuilders to test on MSVC 2015

dl_tlc reacts poorly to arguments

dl_tlc.exe -c test.h -o test.bin test.json

With the test json from the documentation fails to read test.h

dl_tlc.exe -c test.h test.json

Spews the generated header to stdout.

Expected behavior for case 1 would be that it creates the file if it doesn't exist, and also outputs the generated header there. Creating the file test.h yields only a test.bin generated.

Expected behavior for case 2 would be that it creates the file if it doesn't exists and overwrites any data inside it with the generated header.

Unable to compile tests on Linux (Ubuntu 16.10)

Could be something wrong with my setup, but this is what I get from a clean directory:

workdir$ git clone https://github.com/wc-duck/datalibrary
...

workdir$ git clone https://github.com/matricks/bam.git
...

workdir$ cd bam/

workdir/bam$ ./make_unix.sh
compiling using gcc...
/tmp/ccHhCuEn.o: In function `os_tmpname':
loslib.c:(.text+0x29d): warning: the use of `tmpnam' is dangerous, better use `mkstemp'

workdir/bam$ cd ../datalibrary/

workdir/datalibrary$ gcc --version
gcc (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO                                            
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

workdir/datalibrary$ ../bam/bam compiler=gcc platform=linux_x86_64 config=debug -r sc
compiler used "gcc"
[ 1/65] [1] c tool/dl_pack/getopt/getopt.c
[ 2/65] [2] c++ src/dl_alloc.cpp
[ 3/65] [3] c++ src/dl_reflect.cpp
[ 4/65] [4] c++ src/dl_typelib_write_c_header.cpp
[ 5/65] [5] c++ src/dl_typelib_read_bin.cpp
[ 6/65] [6] c++ src/dl_txt_pack.cpp
[ 7/65] [7] c++ src/dl_convert.cpp
[ 8/65] [8] c++ src/dl.cpp
[ 9/65] [2] c++ src/dl_txt_unpack.cpp
[10/65] [1] c++ src/dl_util.cpp
[11/65] [3] c++ src/dl_typelib_write_txt.cpp
[12/65] [4] c++ src/dl_patch_ptr.cpp
[13/65] [1] c++ src/dl_typelib_write_bin.cpp
[14/65] [8] c++ src/dl_typelib_read_txt.cpp
[15/65] [5] c++ tool/dl_tlc/dltlc.cpp
[16/65] [3] c++ external/gtest/src/gtest-all.cc
[17/65] [1] c++ src/dl_alloc.cpp
[18/65] [2] c++ src/dl_reflect.cpp
[19/65] [4] c++ src/dl_typelib_write_c_header.cpp
[20/65] [6] c++ src/dl_typelib_read_bin.cpp
[21/65] [1] c++ src/dl_txt_pack.cpp
[22/65] [2] c++ src/dl_convert.cpp
[23/65] [6] c++ src/dl.cpp
[24/65] [4] c++ src/dl_txt_unpack.cpp
[25/65] [8] c++ src/dl_util.cpp
[26/65] [5] c++ src/dl_typelib_write_txt.cpp
[27/65] [7] lib local/linux_x86_64/gcc/debug/libdl.a
ar: `u' modifier ignored since `D' is the default (see `U')
[28/65] [7] link local/linux_x86_64/gcc/debug/dltlc
[29/65] [8] c++ src/dl_patch_ptr.cpp
[30/65] [4] c++ src/dl_typelib_write_bin.cpp
[31/65] [1] c++ src/dl_typelib_read_txt.cpp
[32/65] [6] c++ tool/dl_pack/dl_pack.cpp
[33/65] [7] tlc local/generated/unittest2.h
[34/65] [7] tlc local/generated/unittest.h
[35/65] [7] tlc local/generated/dlbench.bin
[36/65] [7] tlc local/generated/small.bin
[37/65] [7] tlc local/generated/unittest2.bin
[38/65] [7] tlc local/generated/unittest2.bin.h
[39/65] [7] tlc local/generated/unittest.bin
[40/65] [4] tlc local/generated/dlbench.bin.h
[41/65] [8] tlc local/generated/dlbench.h
[42/65] [7] tlc local/generated/unittest.bin.h
[43/65] [5] tlc local/generated/unittest.bin.h
[44/65] [4] tlc local/generated/small.bin.h
[45/65] [4] c tests/dl_test_valid_c.c
[46/65] [6] link local/linux_x86_64/gcc/debug/dl_pack
[47/65] [7] c++ tests/dl_tests_string.cpp
[48/65] [5] c++ tests/dl_tests_typelib.cpp
In file included from tests/dl_test_valid_c.c:1:0:
local/generated/unittest2.h:66:26: error: ‘vec3_test’ undeclared here (not in a function)
 DL_STATIC_ASSERT( sizeof(vec3_test) == 12, "size of external type vec3_test do not match what was specified in tld." );
                          ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:66:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( sizeof(vec3_test) == 12, "size of external type vec3_test do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:67:30: error: expected specifier-qualifier-list before ‘vec3_test’
 DL_STATIC_ASSERT( DL_ALIGNOF(vec3_test) == 4, "alignment of external type vec3_test do not match what was specified in tld." );
                              ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:67:19: note: in expansion of macro ‘DL_ALIGNOF’
 DL_STATIC_ASSERT( DL_ALIGNOF(vec3_test) == 4, "alignment of external type vec3_test do not match what was specified in tld." );
                   ^~~~~~~~~~
local/generated/unittest.h:21:51: error: ‘struct <anonymous>’ has no member named ‘x’
 #    define DL_ALIGNOF(type) ((sizeof(type) > 1)? offsetof(struct { char c; type x; }, x) : 1)
                                                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:67:19: note: in expansion of macro ‘DL_ALIGNOF’
 DL_STATIC_ASSERT( DL_ALIGNOF(vec3_test) == 4, "alignment of external type vec3_test do not match what was specified in tld." );
                   ^~~~~~~~~~
local/generated/unittest.h:21:30: error: expression in static assertion is not an integer
 #    define DL_ALIGNOF(type) ((sizeof(type) > 1)? offsetof(struct { char c; type x; }, x) : 1)
                              ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:67:19: note: in expansion of macro ‘DL_ALIGNOF’
 DL_STATIC_ASSERT( DL_ALIGNOF(vec3_test) == 4, "alignment of external type vec3_test do not match what was specified in tld." );
                   ^~~~~~~~~~
[49/65] [8] c++ benchmark/dlbench.cpp
local/generated/unittest2.h:68:38: error: expected expression before ‘)’ token
 DL_STATIC_ASSERT( sizeof(((vec3_test*) 0)->x) == 4, "sizeof of member vec3_test::x in external type do not match what was specified in tld." );
                                      ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:68:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( sizeof(((vec3_test*) 0)->x) == 4, "sizeof of member vec3_test::x in external type do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:69:28: error: expected specifier-qualifier-list before ‘vec3_test’
 DL_STATIC_ASSERT( offsetof(vec3_test, x) == 0, "offset of member vec3_test::x in external type do not match what was specified in tld." );
                            ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:69:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( offsetof(vec3_test, x) == 0, "offset of member vec3_test::x in external type do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:70:38: error: expected expression before ‘)’ token
 DL_STATIC_ASSERT( sizeof(((vec3_test*) 0)->y) == 4, "sizeof of member vec3_test::y in external type do not match what was specified in tld." );
                                      ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:70:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( sizeof(((vec3_test*) 0)->y) == 4, "sizeof of member vec3_test::y in external type do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:71:28: error: expected specifier-qualifier-list before ‘vec3_test’
 DL_STATIC_ASSERT( offsetof(vec3_test, y) == 4, "offset of member vec3_test::y in external type do not match what was specified in tld." );
                            ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:71:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( offsetof(vec3_test, y) == 4, "offset of member vec3_test::y in external type do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:72:38: error: expected expression before ‘)’ token
 DL_STATIC_ASSERT( sizeof(((vec3_test*) 0)->z) == 4, "sizeof of member vec3_test::z in external type do not match what was specified in tld." );
                                      ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:72:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( sizeof(((vec3_test*) 0)->z) == 4, "sizeof of member vec3_test::z in external type do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:73:28: error: expected specifier-qualifier-list before ‘vec3_test’
 DL_STATIC_ASSERT( offsetof(vec3_test, z) == 8, "offset of member vec3_test::z in external type do not match what was specified in tld." );
                            ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
local/generated/unittest2.h:73:19: error: expression in static assertion is not an integer
 DL_STATIC_ASSERT( offsetof(vec3_test, z) == 8, "offset of member vec3_test::z in external type do not match what was specified in tld." );
                   ^
local/generated/unittest.h:41:63: note: in definition of macro ‘DL_STATIC_ASSERT’
 #        define DL_STATIC_ASSERT( cond, msg ) _Static_assert( cond, msg )
                                                               ^~~~
bam: 'c tests/dl_test_valid_c.c' error 256
[50/65] [4] c++ tests/dl_tests_union.cpp
[51/65] [2] c++ tests/dl_tests_struct.cpp
[52/65] [1] c++ tests/dl_tests_dl.cpp
[53/65] [6] c++ tests/dl_tests_txt.cpp
[54/65] [8] c++ tests/dl_tests_error.cpp
[55/65] [5] c++ tests/dl_tests_ptr.cpp
[56/65] [8] c++ tests/dl_tests_reflect.cpp
[57/65] [6] c++ tests/dl_tests_bitfield.cpp
[58/65] [7] c++ tests/dl_tests_inline_array.cpp
[59/65] [8] c++ tests/dl_tests_base.cpp
[60/65] [3] lib local/linux_x86_64/gcc/debug/libgtest.a
ar: `u' modifier ignored since `D' is the default (see `U')
[61/65] [3] c++ tests/dl_tests_util.cpp
[62/65] [2] c++ tests/dl_tests_array.cpp
[63/65] [4] link local/linux_x86_64/gcc/debug/dlbench
[64/65] [1] dll local/linux_x86_64/gcc/debug/dlsh.so
[65/65] [2] link local/linux_x86_64/gcc/debug/dl_tests
bam: error: a build step failed

Replace yajl with custom json parser.

Why?

One dependency less ( the last one! )
Bit most of all I think that it would open up for lots of optimizations a general json parser can't do that dl could since we have the type info.

However first some benchmarks need to be written, code can never be called optimized without numbers :)