jorgen/json_struct

json_struct is a single header only C++ library for parsing JSON directly to C++ structs and vice versa

json_struct's Issues

GNUInstallDirs is required for CMAKE_INSTALL_DATAROOTDIR

You have commented out #include(GNUInstallDirs) in the root CMakeLists.txt, but it actually is required for CMAKE_INSTALL_DATAROOTDIR, otherwise it defaults to /, at least it does so in my builds (on Windows, GNU/Linux and Mac OS):

$ mkdir build && cd $_
$ cmake -DCMAKE_INSTALL_PREFIX="../install" -DJSON_STRUCT_OPT_BUILD_BENCHMARKS=0 -DJSON_STRUCT_OPT_BUILD_EXAMPLES=0 -DJSON_STRUCT_OPT_BUILD_TESTS=0 ..
$ cmake --build . --target install --config Release

-- Installing: D:/code/json-struct/install/lib/cmake/json_struct/json_structConfigVersion.cmake
-- Installing: D:/code/json-struct/install/lib/cmake/json_struct/json_structConfig.cmake
-- Installing: D:/code/json-struct/install/./include
-- Installing: D:/code/json-struct/install/./include/json_struct.h
-- Installing: D:/code/json-struct/install/./include/json_struct_diff.h
-- Installing: /json_struct/package.xml

And if I uncomment include(GNUInstallDirs), then CMAKE_INSTALL_DATAROOTDIR gets a proper value and installation goes correctly:

...
-- Installing: D:/code/json-struct/install/share/json_struct/package.xml

So, I am not sure why you commented it out, maybe there was a good reason, but I decided to let you know about this, just in case.

While we are at it, perhaps you'd want to do something with . destination for installing include. It certainly works as it is, but doesn't look entirely correct (D:/code/json-struct/install/./include/json_struct.h).

It's not a bug. How to count elements of nested array?

Hi,
on this array
"vec" : [
{ "key" : 4, "value": 1.0 },
{ "key" : 5, "value": 2.0 },
{ "key" : 6, "value": 3.0 }
]
how to know the number of elements ?
On this sample, 3. Thank's

nothing

float deserial error

struct Json1
{
float num1;
double num2;

JS_OBJECT(JS_MEMBER(num1), JS_MEMBER(num2));
};
const char json_data1[] = R"json(
{
"num1": 32587.403333333333333333,
"num2": 32587.403333333333333333
}
)json";

how to serialize std::map and std::set ?

can I use this to serialize std::map and std::set ?
for example i define a set A,
I use JS::serializeStruct(A) give me a error.
expect the output is ["1" , "2", "3" ]...

Reading JSON which contains certain extraneous values breaks parsing

I discovered this issue when I was trying to parse a small subset of data from a large JSON message. If I created a set of nested structs with members corresponding to only the data I needed, the parse would fail with an ExpectedObjectStart error at an arbitrary location. However, if I added in all of the values present in the message, it would parse correctly.

I spent some time investigating this and have managed to come up with a minimal reproducible example. In the example the parsing doesn't explicitly fail, but rather the parser seems to get out of sync and places incorrect values into structs. I believe that the parser getting out of sync was the cause of my original issue.

The fundamental issue seems to be surrounding nested objects. Take the following JSON:

{
    "object1":
    {
        "value": 1,
        "nested_object":
        {
            "nested_1":
            {
                    "some_value": "foo"
            },
            "nested_2":
            {
                    "some_value": "bar"
            }
        }
    },
    "object2":
    {
            "value": 2
    }
}

If this message is parsed using structs, where every key-value pair is catered for, then parsing succeeds as expected. However, if nested_object is omitted from the struct definition, this triggers the parsing issue. Note that the issue does not occur unless the nested_object has at least two other nested JSON objects within it.

The attached C++ project encapsulates the minimal reproducible example. In the case where the nested object fails to parse correctly. the struct representing object2 contains a value of 0 when it should contain a value of 2.

json_struct MRE.zip

Adding js member listeners

Hi, I really like your library. I think it provides a really elegant interface for serializing and deserializing json.

I would like to propose adding a feature that I think could be useful to the users of json_struct. The feature I would like to propose is having the ability to register listeners to struct members that will be called during deserialization.
For example, deserializing a Person object with the fields name and age could trigger methods registered as property listeners like onNameChanged(std::string) and onAgeChanged(int).
The syntax for adding these property listeners could be something similar to the existing workflow. Something along the lines of:

struct Person
{
    std::string name;
    unsigned age;
    void onNameChanged(std::string);
    void onAgeChanged(unsigned);

    JS_OBJECT(JS_MEMBER_WITH_LISTENER(name, &Person::onNameChanged), 
              JS_MEMBER_WITH_LISTENER(age, &Person::onAgeChanged));
};

The outcome of this would be that upon deserializing, the registered listeners would be triggered, which could enable some additional custom logic to run for every deserialized field.

I understand this feature might not exactly be part of the original scope for json_struct, but I think it might add real value to the library and increase its utility by adding something more than serializing and deserializing.

I would like to volunteer to write the implementation of this. Even if you think this feature doesn't belong in json_struct, I would still appreciate some pointers on how you would recommend integrating this in json_stuct, as I would like to give it a whirl anyway.

Brilliant!

Can we now get rid of std:: lib, please?

serial a class' private member

i try to serial a class like this,

class a{
public:
  std::string b;
}

JS_OBJ_EXT(a,b);

a my_a;
auto json_string_a = JS::serializeStruct(my_a);

it works fine.

but when i add a private member c

class a{
public:
  std::string b;
private:
  std::string c;
}

JS_OBJ_EXT(a,b,c);

the complier says it can NOT access private member.
so, how can i make it work again ?

std::optional<std::vector<my_struct>> will cause program exit

change from:
#ifdef JS_STD_OPTIONAL
/// \private
template
struct TypeHandler<std::optional>
{
public:
static inline Error to(std::optional &to_type, ParseContext &context)
{
to_type = {};
return TypeHandler::to(to_type.value(), context);
}

static inline void from(const std::optional &opt, Token &token, Serializer &serializer)
{
if (opt.has_value())
TypeHandler::from(opt.value(), token, serializer);
}
};
#endif

to:
#ifdef JS_STD_OPTIONAL
/// \private
template
struct TypeHandler<std::optional>
{
public:
static inline Error to(std::optional &to_type, ParseContext &context)
{
to_type = T();
return TypeHandler::to(to_type.value(), context);
}

static inline void from(const std::optional &opt, Token &token, Serializer &serializer)
{
if (opt.has_value())
TypeHandler::from(opt.value(), token, serializer);
}
};
#endif

will fix the bug

Compilation fails if std::tuple member in struct

I'm running on Linux Ubuntu 22.04. I ported the TEST_CASE("test_serialize_simple", "[json_struct][serialize]") in tests/json-struct-serialize-test.cpp to google test, and added members to the Simple struct. If I include the std::tuple member in the JS_OBJ list, the compiler generates reams of failure messages.

struct Simple
{
    std::string A;
    bool b;
    int some_longer_name;
    std::vector<int> vector;
    std::string string;
    std::tuple<std::string, float, int> tup;
    std::unordered_map<std::string, double> uo_map;
    std::optional<std::string> optional;

    //JS_OBJECT(JS_MEMBER(A), JS_MEMBER(b), JS_MEMBER(some_longer_name));
    //JS_OBJ(A, b, some_longer_name, vector, string, tup, uo_map, optional); /// <<<< this fails
    JS_OBJ(A, b, some_longer_name, vector, string, uo_map, optional);        /// <<<< this succeeds

};

TEST_F(EsiJsonSerdesGTest, GenerateSimple)
{
    Simple simple;
    simple.A = "TestString";
    simple.b = false;
    simple.some_longer_name = 456;
    simple.vector = {1,2,3,4,5,6,7,8,9};
    simple.unordered_map = { {"foo", 100.0}, {"bar", 101.1}};
    simple.optional = "hello";

    std::string output = JS::serializeStruct(simple, JS::SerializerOptions(JS::SerializerOptions::Compact));
    EXPECT_STREQ(output.c_str(), expected1_compact);
}

• git commit hash

$ git log -n 1
commit 9b719771470536763430703b7f04acc558dafe56 (HEAD -> master, origin/master, origin/HEAD)
Author: Jørgen Lind <[email protected]>
Date:   Thu Jan 19 12:12:19 2023 +0100

    Fix some warnings
$

• OS

$ uname -a
Linux demo 5.15.0-70-generic #77-Ubuntu SMP Tue Mar 21 14:02:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy
$

• compiler

$ g++ --version
g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

min and max ?

#if defined(min) || defined(max)

#error min or max macro is defined. Make sure these are not defined before including json_struct.h.\
 Use "#define NOMINMAX 1" before including Windows.h
#endif

I have #define NOMINMAX 1,
but I can NOT compile this either,

using vs 2022 and qt5.
How can I find where are minx and max macro is defined and disable it ?

JS_ENUM Defaults

how can I setenum defaults like this
JS_ENUM(Color, Red = 0x01, Green = 0x02, Blue = 0x03, Yellow4 = 0x04, Purple = 0x05)
struct ColorData {
Color color;

JS_OBJ(color);

};
JS_ENUM_DECLARE_STRING_PARSER(Color)

Compilation can pass, but running or have assert in source code 4438 lines.

I looked at the code and probably understood that this is looking for the name of the enumeration member, but there is no related '=' processing。

Does this library support enumerations with initial values?

Parsing std::vector<MyStruct*>

How do I setup a member std::vector<MyStruct*> for serialization and deserialization? If I make it a vector of values rather than pointers it works fine.

does parseTo check struct members ?

for example

struct a{
  int a;
  JS_OBJ(a);
}

struct b{
  int b;
  JS_OBJ(b);
}

a s_a;
s_a.a = 19;
b s_b;
s_b.b = 0;

auto json = JS::serializeStruct(s_a);
JS::ParseContext context(json);
auto error = context.parseTo(s_b);

the code give me no error,
but s_b.b is still zero.

Not compile at armv7

Hello! I try compile your library at orange pi (arm v7). Use gcc/g++ version 7 and 8. But this error on compile:

 error: static assertion failed: Missing JS_OBJECT JS_OBJECT_EXTERNAL or TypeHandler specialisation

   static_assert(sizeof(HasJsonStructBase<JS_OBJECT_T>::template test_in_base<JS_OBJECT_T>(nullptr)) ==
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
                   sizeof(typename HasJsonStructBase<JS_OBJECT_T>::yes),
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                               
/home/user/socket/json_struct.h:2778:98: error: ‘long int’ is not a class, struct, or union type
   using TT = decltype(JS_OBJECT_T::template JsonStructBase<JS_OBJECT_T>::js_static_meta_data_info());
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/home/user/socket/json_struct.h:2779:99: error: ‘long int’ is not a class, struct, or union type
   using ST = decltype(JS_OBJECT_T::template JsonStructBase<JS_OBJECT_T>::js_static_meta_super_info());

error: ‘js_static_meta_data_info’ is not a member of ‘JS::Internal::JsonStructBaseDummy<long int, long int>’
   auto members = Internal::JsonStructBaseDummy<T, T>::js_static_meta_data_info();
error: ‘js_static_meta_data_info’ is not a member of ‘JS::Internal::JsonStructBaseDummy<long int, long int>’
   using Members = decltype(Internal::template JsonStructBaseDummy<T, T>::js_static_meta_data_info());
                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/home/user/socket/json_struct.h:3108:101: error: ‘js_static_meta_super_info’ is not a member of ‘JS::Internal::JsonStructBaseDummy<long int, long int>’
   using SuperMeta = decltype(Internal::template JsonStructBaseDummy<T, T>::js_static_meta_super_info());
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/home/user/socket/json_struct.h:3107:98: error: ‘js_static_meta_data_info’ is not a member of ‘JS::Internal::JsonStructBaseDummy<long int, long int>’
   using Members = decltype(Internal::template JsonStructBaseDummy<T, T>::js_static_meta_data_info());

Cannot parse std::tuple<bool>

On MSVC, I'm unable parse fields like "member": [true] to a type std::tuple<bool> member. A minimal example is below

#include <iostream>
#include "json_struct.h"

struct TestInt {
  std::tuple<int32_t> member;
  JS_OBJ(member);
};

struct TestBool {
  std::tuple<bool> member;
  JS_OBJ(member);
};

int main(int argc, char** argv) {
  TestInt tiStruct;
  JS::ParseContext intContext(R"({ "member": [5] })");
  if (intContext.parseTo(tiStruct) == JS::Error::NoError) {
    std::cout << "Success, member is: " << std::get<0>(tiStruct.member) << std::endl;
  }
  else {
    std::cout << intContext.makeErrorString();
  }

  TestBool tbStruct;
  JS::ParseContext boolContext(R"({ "member": [true] })");
  if (boolContext.parseTo(tbStruct) == JS::Error::NoError) {
    std::cout << "Success, member is: " << std::get<0>(tbStruct.member) << std::endl;
  }
  else {
    std::cout << boolContext.makeErrorString();
  }
}

Output:

Success, member is: 5
Error ExpectedDelimiter:
{ "member": [true] }
                  ^

I'm not sure what the cause here is, all I can say from current investigation is that single item tuples of other basic types (number/string) still work, and tuples involving bools + other types work, e.g. std::tuple<bool, int> will parse fine if given [true, 2]

Could be useful to consider the On Demand API

This is maybe of interest to you...
https://github.com/simdjson/simdjson/blob/master/doc/ondemand.md

Enhancement in EnumHandler

Hello.

First of all, thanks you your library, which saved pretty much time for me!

While using Enum serializer, I have noticed that it is only possible to serialize string bash to enum:
https://github.com/jorgen/json_struct/blob/master/include/json_struct.h#L4261

static inline Error to(T &to_type, ParseContext &context)
  {
    if (context.token.value_type == Type::String)
    {
      auto &strings = F::strings();
      for (size_t i = 0; i < strings.size(); i++)
      {
        const DataRef &ref = strings[i];
        if (ref.size == context.token.value.size)
        {
          if (memcmp(ref.data, context.token.value.data, ref.size) == 0)
          {
            to_type = static_cast<T>(i);
            return Error::NoError;
          }
        }
      }
    }
    return Error::IllegalDataValue;
  }

It would be very helpful if it become possible to serialize enum from int value, which is the enum value actually.

EnumHandler::to can be adjusted with another clause to parse number, something like this:

     else if (context.token.value_type == Type::Number)
     {
       int32_t tmp;
       const char *pointer;
       auto parse_error = Internal::ft::integer::to_integer(context.token.value.data, context.token.value.size, tmp, pointer);
       if (parse_error != Internal::ft::parse_string_error::ok || context.token.value.data == pointer)
         return Error::FailedToParseInt;

       to_type = static_cast<T>(tmp);
       return Error::NoError;
     }

How to Install this?

Hello, I am trying to use this in a C++ dll project I have. I am just not sure how to install this?

seems like I just copy the json_tools.h into my project, but this is causing the following error on line 4088:
C2589 '(': illegal token on right side of '::'

Which makes no since considering the code does not have that at all. What am I missing?

Thanks,
Bob

GCC11 compatibility

Hello.

I'm now trying to migrate to GCC11 and see those errors:

/lt2http/src/external/json_struct/include/json_struct.h: In function ‘T JS::Internal::ft::iabs(typename std::enable_if<std::is_signed<_Tp>::value, T>::type)’:
/lt2http/src/external/json_struct/include/json_struct.h:5095:17: error: ‘numeric_limits’ is not a member of ‘std’
 5095 |   if (a == std::numeric_limits<T>::min())
      |                 ^~~~~~~~~~~~~~
/lt2http/src/external/json_struct/include/json_struct.h:5095:33: error: expected primary-expression before ‘>’ token
 5095 |   if (a == std::numeric_limits<T>::min())
      |                                 ^
/lt2http/src/external/json_struct/include/json_struct.h:5095:36: error: ‘::min’ has not been declared
 5095 |   if (a == std::numeric_limits<T>::min())
      |                                    ^~~
/lt2http/src/external/json_struct/include/json_struct.h:5095:36: note: suggested alternatives:
In file included from /usr/x86_64-linux-musl-cross/x86_64-linux-musl/include/c++/11.2.1/algorithm:62,
                 from /lt2http/src/external/json_struct/include/json_struct.h:115,
                 from /lt2http/src/bittorrent/meta_info.h:6,
                 from /lt2http/src/bittorrent/meta_info.cpp:1:
/usr/x86_64-linux-musl-cross/x86_64-linux-musl/include/c++/11.2.1/bits/stl_algo.h:3455:5: note:   ‘std::min’
 3455 |     min(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
In file included from /lt2http/src/bittorrent/meta_info.h:6,
                 from /lt2http/src/bittorrent/meta_info.cpp:1:

Google bring me to this page - https://www.gnu.org/software/gcc/gcc-11/porting_to.html
I think they now require explicit use of specific header files, and that is also a change in Clang 12 as well.

JS_OBJ_EXT triggers "ISO C++11 requires at least one argument for the "..." in a variadic macro" warning

I get this warning when compiling with GCC 10.3.0, which is a bit of an issue for me since my repo is configured to treat warnings as errors, and these warnings are difficult to turn off because they don't seem to have a -Wno-... switch. As the related code is generated inline from the macros, I'd have to switch either the pedantic or warnings-as-errors settings off for my whole repo. Is there a way to resolve this properly?

Serialize runtime exceptions when debug enabled and struct contains vector

serializeStruct throws various errors (sometimes a "bad alloc" exception, sometimes segv, sometimes invalid pointer) if a struct contains a vector and the compiler debug options are enabled.

Compiler:

$ g++ --version
g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0

Compiler command line options:

-g -std=c++20     <<<<<< with -DCMAKE_BUILD_TYPE=Debug (this fails at runtime)

-O3 -DNDEBUG -std=c++20   <<<<<  with -DCMAKE_BUILD_TYPE=Release (this succeeds at runtime)

Google Test version of test code:

struct Simple
{
    std::vector<int> v;
    JS_OBJECT(JS_MEMBER(v));
};

const char expected1_compact[] = R"json({"v":[1,2,3,4,5,6,7,8,9]})json";

TEST_F(EsiJsonSerdesGTest, Struct2Json)
{
    std::vector<int> temp_vect{1,2,3,4,5,6,7,8,9};

    Simple simple;
    simple.v = temp_vect;

    std::string output = JS::serializeStruct(simple, JS::SerializerOptions(JS::SerializerOptions::Compact));
    EXPECT_STREQ(output.c_str(), expected1_compact);
}

Build/run commands:

 rm -rf build
 mkdir build
 cd build
 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install_dir ..
 ninja
./gtest_sim --gtest_filter=EsiJsonSerdesGTest.Struct2Json
Running main() from /home/sscott/eg_link/eg_esi/apps/build/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = EsiJsonSerdesGTest.Struct2Json
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EsiJsonSerdesGTest
[ RUN      ] EsiJsonSerdesGTest.Struct2Json
[       OK ] EsiJsonSerdesGTest.Struct2Json (0 ms)
[----------] 1 test from EsiJsonSerdesGTest (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 1 test.

cd ..
rm -rf build
mkdir build
cd build
cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=./install_dir ..
ninja
./gtest_sim --gtest_filter=EsiJsonSerdesGTest.Struct2Json
Running main() from /home/sscott/eg_link/eg_esi/apps/build/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = EsiJsonSerdesGTest.Struct2Json
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EsiJsonSerdesGTest
[ RUN      ] EsiJsonSerdesGTest.Struct2Json
free(): invalid pointer
Aborted (core dumped)

Errors 'js_static_meta_data_info' and 'js_static_meta_super_info'

Hi,
I have this struct into a variable cfg_

   struct Config
    {
        std::string webInterface = "0.0.0.0";
        int webPort = 9898;
        std::string apiKey = "xxxyyyzzz";
        std::string productId = "xxxxxxxxxx";
        std::string productVersion = "0.1";
        std::string licFile = "licfiles\\licfile_9001494765.lic";
        bool useSwagger = false;
        std::string revalidate = "0 */30 * * * ?";
        int thresholdOfflineHours = 48;
        int thresholdValidatedHours = 720;
        int cicoModule = -1;
        std::vector<Module> moduleList = fillDefaultModuleList();
        Config() {}
        Config(int argc, char* argv[]);
        JS_OBJECT(
            JS_MEMBER(webInterface),
            JS_MEMBER(webPort),
            JS_MEMBER(apiKey),
            JS_MEMBER(productId),
            JS_MEMBER(productVersion),
            JS_MEMBER(licFile),

            JS_MEMBER(useSwagger),
            JS_MEMBER(revalidate),
            JS_MEMBER(thresholdOfflineHours),
            JS_MEMBER(thresholdValidatedHours),
            JS_MEMBER(cicoModule),
            JS_MEMBER(moduleList)
        );
    };

The JS::ParseContext parseContext(jsonData) is ok, but I try to serialize it to obtain a JSON string:

std::string prettyJson = JS::serializeStruct(cfg_);

I have these errors on build:

C2039	'js_static_meta_data_info': is not a member of 'std::shared_ptr<licmgrapi::Config>'	
C2039	'js_static_meta_super_info': is not a member of 'std::shared_ptr<licmgrapi::Config>'

What I'm missing ?

Thanks in advance

this doest NOT support serialize a std::map with an int as its key

Hi,
I have been putting off making these TypeHandlers since then we have to include <map>, <set>, <unordered_set> in json_struct.h and suddenly everyone has has these includes just by including json_struct. I regret most of the includes in there already, but changing this can break user code, (I still have some members with spelling mistakes because chaning it will break code).

I have come up with this scheme where the TypeHandlers are guarded by their own include guards. So if you want the std::map TypeHandler then you will have to define JS_STL_MAP before including json_struct.h. The unit test should give you the idea: https://github.com/jorgen/json_struct/blob/master/tests/json-unordered-map.cpp#L4.

This is available in 3ddf92e.

Originally posted by @jorgen in #27 (comment)

for example;

std::map<int,std::map<int,std::string>> a;

JS::serializeStruct(a);

will give me an errer

invalid conversion from 'int' to 'const char*'

it seems that

/*!
 *  \brief Pointer to data
 *
 *  DataRef is used to refere to some data inside a json string. It holds the
 *  start posisition of the data, and its size.
 */
struct DataRef
{
  /*!
   * Constructs a null Dataref pointing to "" with size 0.
   */
  constexpr explicit DataRef()
    : data("")
    , size(0)
  {
  }

  /*!
   * Constructs a DataRef pointing to data and size.
   * \param data points to start of data.
   * \param size size of data.
   */
  constexpr explicit DataRef(const char *data, size_t size)
    : data(data)
    , size(size)
  {
  }

  /*!  Cobstructs a DataRef pointing to an array. This will \b NOT look for
   * the null terminator, but just initialize the DataRef to the size of the
   * array - 1. This function is intended to be used with string literals.
   * \param data  start of the data.
   */
  template <size_t N>
  constexpr explicit DataRef(const char (&data)[N])
    : data(data)
    , size(N - 1)
  {
  }

  explicit DataRef(const std::string &str)
    : data(&str[0])
    , size(str.size())
  {
  }

  explicit DataRef(const char *data)
    : data(data)
    , size(strlen(data))
  {
  }

  const char *data;
  size_t size;
};

does NOT privide an constructor which treat int as an input.

This line can overflow by one and underflow by alot!

json_struct/include/json_struct.h

Line 3877 in 0c68678

auto diff = &ref.data[ref.size - 1] - start;

build.sh script fails on Ubuntu 22.04

Hi - I'm getting compiler errors running the build.sh script on Linux Ubuntu 22.04. Are these expected?

Thanks.

• git commit hash

$ git log -n 1
commit 9b719771470536763430703b7f04acc558dafe56 (HEAD -> master, origin/master, origin/HEAD)
Author: Jørgen Lind <[email protected]>
Date:   Thu Jan 19 12:12:19 2023 +0100

    Fix some warnings
$

• OS

$ uname -a
Linux demo 5.15.0-70-generic #77-Ubuntu SMP Tue Mar 21 14:02:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy
$

• compiler

$ g++ --version
g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Command sequence:

git clone [email protected]:jorgen/json_struct.git
cd json_struct/
./build.sh

Shell output:

.
.
.
[ 73%] Building CXX object tests/CMakeFiles/unit-tests-cxx17.dir/json-tokenizer-fail-test.cpp.o
[ 73%] Building CXX object tests/CMakeFiles/unit-tests-cxx17.dir/json-tokenizer-partial-test.cpp.o
[ 74%] Building CXX object tests/CMakeFiles/unit-tests-cxx17.dir/json-tokenizer-test.cpp.o
[ 75%] Building CXX object tests/CMakeFiles/unit-tests-cxx17.dir/json-function-test.cpp.o
In file included from /home/sscott/json_struct/tests/json-function-test.cpp:23:
/home/sscott/json_struct/include/json_struct/json_struct.h: In function ‘void {anonymous}::____C_A_T_C_H____T_E_S_T____10()’:
/home/sscott/json_struct/include/json_struct/json_struct.h:4113:39: error: array subscript ‘int (**)(...)[0]’ is partly outside array bounds of ‘{anonymous}::CallErrorCheck [1]’ [-Werror=array-bounds]
 4113 |     (container.*functionInfo.function)();
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/home/sscott/json_struct/tests/json-function-test.cpp:372:18: note: while referencing ‘errorCheck’
  372 |   CallErrorCheck errorCheck;
      |                  ^~~~~~~~~~
In file included from /home/sscott/json_struct/tests/json-function-test.cpp:23:
/home/sscott/json_struct/include/json_struct/json_struct.h:4014:39: error: array subscript ‘int (**)(...)[0]’ is partly outside array bounds of ‘{anonymous}::CallErrorCheck [1]’ [-Werror=array-bounds]
 4014 |     (container.*functionInfo.function)(arg, context.error_context);
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/sscott/json_struct/tests/json-function-test.cpp:372:18: note: while referencing ‘errorCheck’
  372 |   CallErrorCheck errorCheck;
      |                  ^~~~~~~~~~
In file included from /home/sscott/json_struct/tests/json-function-test.cpp:23:
/home/sscott/json_struct/include/json_struct/json_struct.h: In function ‘void {anonymous}::____C_A_T_C_H____T_E_S_T____2()’:
/home/sscott/json_struct/include/json_struct/json_struct.h:3935:62: error: array subscript ‘int (**)(...)[0]’ is partly outside array bounds of ‘{anonymous}::CallFunctionSub [1]’ [-Werror=array-bounds]
 3935 |     TypeHandler<Ret>::from((container.*functionInfo.function)(arg), token, context.return_serializer);
      |                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
/home/sscott/json_struct/tests/json-function-test.cpp:143:19: note: while referencing ‘cont’
  143 |   CallFunctionSub cont;
      |                   ^~~~
In file included from /home/sscott/json_struct/tests/json-function-test.cpp:23:
/home/sscott/json_struct/include/json_struct/json_struct.h: In function ‘void {anonymous}::____C_A_T_C_H____T_E_S_T____6()’:
/home/sscott/json_struct/include/json_struct/json_struct.h:3996:39: error: array subscript ‘int (**)(...)[0]’ is partly outside array bounds of ‘{anonymous}::SuperParamCallable [1]’ [-Werror=array-bounds]
 3996 |     (container.*functionInfo.function)(arg);
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
/home/sscott/json_struct/tests/json-function-test.cpp:232:22: note: while referencing ‘cont’
  232 |   SuperParamCallable cont;
      |                      ^~~~
cc1plus: all warnings being treated as errors
gmake[2]: *** [tests/CMakeFiles/unit-tests-cxx17.dir/build.make:328: tests/CMakeFiles/unit-tests-cxx17.dir/json-function-test.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:696: tests/CMakeFiles/unit-tests-cxx17.dir/all] Error 2
gmake: *** [Makefile:166: all] Error 2
$

Typo in json_struct.h

Hello,
I just wanted to point out a small typo at line 155 of file json_struct.h.
It's:
#if __cpp_if_consexpr

It probably should be:
#if __cpp_if_constexpr

problem on unordered_map<string,vector<T>>

seems that unordered_map<string,vector<T>> is not suupported
the key of unordered_map is empty string

test case

#include <string>
#include <vector>
#include <map>
#include <json_struct.h>
#include <cinttypes>
//these two have to be ifdef guarded becuase JS support compiler versions where
//they are not implemented
#ifdef JS_STD_UNORDERED_MAP
#include <unordered_map>
#endif
#ifdef JS_STD_OPTIONAL
#include <optional>
#endif


const char json[] = R"json(
{
    "unordered_map" : {
        "foo" : [1],
        "bar" : [2]
    }
}
)json";

struct JsonData
{
#ifdef JS_STD_UNORDERED_MAP
    std::unordered_map<std::string, std::vector<double>> unordered_map;
#else
    JS::JsonObject unordered_map;
#endif
  JS_OBJ(unordered_map);

};

int main()
{
    JsonData dataStruct;
    JS::ParseContext parseContext(json);
    if (parseContext.parseTo(dataStruct) != JS::Error::NoError)
    {
        std::string errorStr = parseContext.makeErrorString();
        fprintf(stderr, "Error parsing struct %s\n", errorStr.c_str());
        return -1;
    }

#ifdef JS_STD_UNORDERED_MAP
    fprintf(stderr, "unordered_map:\n");
    for (auto it : dataStruct.unordered_map)
        fprintf(stderr, "\t%s", it.first.c_str());  // empty string here
#endif
    return 0;
}

Enum class support

Is there a way to serialize enum class props without a custom TypeHandler?

Example:

enum class MyEnum : int {

}

struct MyObject {
   MyEnum myEnum;

   JS_OBJ(myEnum);
}

Invalid memory access when parsing message

This bug was discovered when we accidentally parsed a gateway error page as a JSON response from an API. I wasn't expecting the parsing to succeed, but the parsing process actually caused a crash.

I created a minimal reproducible example, below:

#include <iostream>
#include "json_struct.h"

static const char* const MESSAGE =
	"<html>\r\n"
	"<head><title>400 Bad Request</title></head>\r\n"
	"<body>\r\n"
	"<center><h1>400 Bad Request</h1></center>\r\n"
	"<hr><center>cloudflare</center>\r\n"
	"</body>\r\n"
	"</html>\r\n";

int main(int*, char**)
{
	std::cout << "Parsing message" << std::endl;

	JS::ParseContext context(MESSAGE);
	JS::Map outMap;
	JS::Error errror = context.parseTo(outMap);

	std::cout << "Error: " << context.makeErrorString() << std::endl;

	return 0;
}

When attempting to parse this, I get output that begins:

Parsing message
Error: Error EncounteredIllegalChar:
HHt�H�H�;r�HHtgH�H�#r�HHtOH�H��r�LHt7HD$HA...

After inspection in the debugger, the issue appears to be an unsigned underflow on line 1740:

for (cursor_back = real_cursor_index - 1; cursor_back > stop_back; cursor_back--)

Here, because the first character of the input was not valid, real_cursor_index is 0, so subtracting 1 underflows the index.

Error triggered but not surfaced when setting allow_missing_members or allow_unassigned_required_members

I believe I found a bug when setting allow_missing_members or allow_unassigned_required_members. In the example below, parseTo() does indeed return an error (in one of my tests, a JS::Error::MissingPropertyMember), but when makeErrorString() is called it returns "Error NoError:".

#include "json_struct.h"
#include <iostream>

struct MyStruct {
  int a;
  int b;

  JS_OBJ(a, b);
};

int main() {
  std::string json = "{\"a\":1, \"c\": 3}";

  MyStruct my_struct;

  JS::ParseContext parse_ctx(json);

  parse_ctx.allow_missing_members = false;
  parse_ctx.allow_unnasigned_required_members = false;

  if (parse_ctx.parseTo(my_struct) != JS::Error::NoError) {
    std::cout << "Error parsing struct: " << parse_ctx.makeErrorString() << "\n";
  }

}

Output: Error parsing struct: Error NoError:

Expected: either a MissingPropertyMember or UnassignedRequiredMember error string, depending on the circumstances.

Serializer::writeAsString will not write empty string

the Serializer::writeAsString will write nothing when given "" as data, most likely resulting in a bad JSON.

to reproduce give it something like ... {"key1": "value1", "key2" : "", "key3":"value3"}... the output will be:

 {
 	"key1": "value1",
 	"key2": ,
 	"key3": "value3"
 }

the problem might be in the tokenizer giving the token size = 0.

or is the code wrong?

std::string prettify(const std::string& json) const
{
	std::string prettyJson;

	JS::Tokenizer tokenizer;
	tokenizer.addData(json.c_str(), json.size());
	JS::Token token;
	JS::Error e = JS::Error::NoError;
	JS::JsonTokens jsonTokens;

	while (e == JS::Error::NoError)
	{
		e = tokenizer.nextToken(token);
		if (e == JS::Error::NoError)
			jsonTokens.data.emplace_back(token);
	}
	assert(e == JS::Error::NeedMoreData);

	JS::SerializerContext context(prettyJson);
	context.serialize(jsonTokens.data);

	return prettyJson;
}

makeErrorString() fails with exception in basic_string::_M_create

#include "json_struct.h"

struct AfterburnerConfig 
{
    double beam_hor_variance = 0;
    JS_OBJ(beam_hor_variance);
};

I put some invalid JSon here:

{
    /* dfd */
    "beam_hor_variance" : 6.0,
}

Readout code is nothing fancy:

// Read
{
    std::ifstream reader(file_name);
    std::stringstream buffer;
    buffer << reader.rdbuf();
    auto json = buffer.str();

    AfterburnerConfig config2;
    JS::ParseContext parseContext(json);
    if (parseContext.parseTo(config2) != JS::Error::NoError)
    {
        std::string errorStr = parseContext.makeErrorString();   // <=== fails here
        throw std::runtime_error(errorStr);
    }
    REQUIRE(config2.beam_hor_variance == Catch::Approx( 6 ));
}

Instead of getting an error with message about the wrong character it fails in makeErrorString():

due to unexpected exception with message:
  basic_string::_M_create

It fails exactly here:

inline std::string Tokenizer::makeErrorString() const
{
  static_assert(sizeof(Internal::error_strings) / sizeof *Internal::error_strings == size_t(Error::UserDefinedErrors),
                "Please add missing error message");

  std::string retString("Error");
  if (error_context.error < Error::UserDefinedErrors)
    retString += std::string(" ") + Internal::error_strings[int(error_context.error)];
  if (error_context.custom_message.size())
    retString += " " + error_context.custom_message;
  retString += std::string(":\n");
  for (size_t i = 0; i < error_context.lines.size(); i++)
  {
    retString += error_context.lines[i] + "\n";
    if (i == error_context.line)
    {
      std::string pointing(error_context.character + 1, ' ');
      pointing[error_context.character - 1] = '^';
      pointing[error_context.character] = '\n';
      retString += pointing;   // <========= HERE 
    }
  }
  return retString;
}