netmail-open/wjelement

advanced, flexible JSON manipulation in C

WJElement - JSON manipulation in C

WJElement is a very flexible JSON library developed by Messaging Architects. It was created for MA's "WARP" webserver ("Warp Json Elements"), and is built on top of the lower-level WJReader and WJWriter libraries (also included).

See the wiki for more information, example code and full API reference.

WJReader and WJWriter are optimized for speed and memory-efficiency. WJElement focuses on flexibility and handy features, allowing C code to manipulate JSON documents with as few statements (fewer, sometimes!) as JavaScript itself. WJElement is also capable of json-schema validation.

WJElement has grown into a generally-useful library, and is used across Messaging Architects' netmail and related projects. It is loved enough by MA's developers that we desire to use it elsewhere too, and we think others will enjoy it as well. So, here it is, ready to be consumed in any project, open or closed, as outlined by the GNU LGPL (any version). Include it as-is and link to it from your code, massage it into your own statically-linked package, or use it in ways we haven't thought of. Read the docs/headers, have fun, and if you use it for something awesome, let us know about it! :^)

• Owen Swerkstrom <[email protected]> - community/repo admin, WJESchema
• Micah N Gorrell <[email protected]> - primary author of WJElement

wjelement's Issues

Spurious validation errors when using combining keywords

When validating against a JSON schema with combining keywords, spurious errors might appear. In the pass case this is not a big problem since errors can be simply ignored. However, in case the validation fails, the library user can no longer tell which validation issues are actually relevant.

The attached example leads to this spurious error when using the validate example application:

...
schema: good
(root): required member 'name' not found.
validation: PASS

example.zip

Build warnings with -Wparentheses

wjelement/src/wjelement/schema.c: In function ‘SchemaValidate’:
wjelement/src/wjelement/schema.c:429:3: warning: suggest parentheses around assignment used as truth value [-Wparentheses]
if(str = WJEString(schema, "["$ref"]", WJE_GET, NULL)) {
^
wjelement/src/wjelement/schema.c:652:6: warning: suggest parentheses around assignment used as truth value [-Wparentheses]
if(sub = WJEObject(schema, "properties", WJE_GET)) {

JSON Schema validation - additionalItems

Hi!

First thanks a lot for providing this C library for working with JSON files.

I have a question about validating a JSON file against a schema using the additionalItems functionality. Given the following schema:

{
"items": [ ],
"additionalItems": {"type": "integer"}
}

And the JSON is:

[ 1, 2, 3, "foo" ]

This sample is taken from the JSON schema test suite also available here on github and shouldn't be valid. However on using WJESchemaValidate() the function returns true and the error callback is also not called. Other sample files and schemas are validating fine and errors are also generated for invalid JSON.

Am I missing something? Is the additionalItems keyword supported by the wjelement library?

Kind regards
Clemens

Graceful malloc failure handling and/or memory optimization for constrained environments

Thank you for the library. It may be just what we need for a project on an (embedded) AVR platform. We're specifically interested in the schema validation feature for validating some relatively simple JSON blobs.

I'm having trouble figuring out what I presume is a memory allocation problem.

We have a pretty capable build environment in which we can run unit tests on our production AVR code natively (OS X and Linux) and build some code libraries for all three platforms. I was able to strip down wjelement (removed 64 bit support, etc.) and add missing functionality (strndup()) such that the library would build both for AVR release and our native testing environments.

I have been successful in using the slimmed down library to validate JSON in our native test environment. Now I'm working to see if wjelement will run on our AVR chip. At first everything died when I put validation calls in our existing code. I soon figured out that it was because of memory allocation issues.

The default internal memory allocation is for 4k RAM in WJROpenDocument(), but we only have 8k RAM total!

I tweaked _WJEParse() to take a buffer allocation size value as a parameter. With that change and smaller memory allocation calls to _WJEParse() for a test JSON blob and a schema string both parsing calls now succeed. Unfortunately, the firmware crashes upon trying to validate the JSON blob against the schema. I assume this is more memory allocation issues.

1. What kind of memory optimization might I be able to do in the guts of WJESchemaValidate()? Is it even worth trying? I'm hoping there's more memory blocks whose default sizes I can cut down.
2. Could you direct me on how to change the code to more gracefully error out if memory allocation fails so I can track down the issue and robustify validation for our use case?

Anything you can say is very much appreciated.

reserved identifier violation

I would like to point out that identifiers like "__WJEBool" and "_WJROpenDocument" do not fit to the expected naming convention of the C language standard.
Would you like to adjust your selection for unique names?

min/max definitions in xpl.h

These definitions can cause a headache in some situations. In some build environments, I need to include standard library files before I (directly or indirectly) include xpl.h.

I suggest changing these macros to XPL_MIN and XPL_MAX to avoid conflicts.

Sort an array of object literals

In my application I have the need to sort an array of object literals by a key, and the key can be any type. For example:

this...

{
  "myArray": [
    {"key": "B", "value": "Buzz"},
    {"key": true, "value": "Baz"},
    {"key": "42", "value": "Flam"},
    {"key": 1, "value": "Bar"},
    {"key": "A", "value": "Foo"},
    {"key": false, "value": "Flub"},
    {"key": 2, "value": "Fizz"},
  ]
}

should sort to to perhaps something like this...

{
  "myArray": [
    {"key": true, "value": "Baz"},
    {"key": false, "value": "Flub"},
    {"key": 1, "value": "Bar"},
    {"key": 2, "value": "Fizz"},
    {"key": "42", "value": "Flam"},
    {"key": "A", "value": "Foo"},
    {"key": "B", "value": "Buzz"},
  ]
}

I am not aware of any way to do this within the library itself but I can think of how to do it outside the library (looping over the array, sorting the keys and attaching the literals into a new array). I am curious if this use case is common enough that it would make sense being in the library itself, or if there was some clever way to use selectors?

cmake -i is no longer supported

I was trying to install WJElement, following the instructions in the INSTALL file, and when I tried cmake -i, I got the following message:

The "cmake -i" wizard mode is no longer supported.
Use the -D option to set cache values on the command line.
Use cmake-gui or ccmake for an interactive dialog.

What should I do instead?

json schema draft 04 support

Hi

I'd love to use this library and I'm interested if there are concrete plans when draft 04 will be supported.
Otherwise I'll go on based on draft 03

Thanks in advance.
Steffen

using $ref doesn't call load callback?

Hi!

The sample JSON event schema uses $ref to reference an external "geo" schema for validating the geo object. I used this sample to take a first look how I could provide the load callback but my function was never called. The event schema file would be:

{
"description": "A representation of an event",
"type": "object",
"properties": {
"dtstart": {
"format": "date-time",
"type": "string",
"description": "Event starting time",
"required": true
},
"dtend": {
"format": "date-time",
"type": "string",
"description": "Event ending time"
},
"summary": { "type": "string", "required": true },
"location": { "type": "string" },
"url": { "type": "string", "format": "uri" },
"duration": {
"format": "time",
"type": "string",
"description": "Event duration"
},
"rdate": {
"format": "date-time",
"type": "string",
"description": "Recurrence date"
},
"rrule": {
"type": "string",
"description": "Recurrence rule"
},
"category": { "type": "string" },
"description": { "type": "string" },
"geo": { "$ref": "geo.json" }
}
}

I replaced the value of the $ref keyword/property here with a simplified relative path... On validating a simple example instance like (invalid by intention):

{
"dtstart": "",
"dtend": "",
"location": "",
"summary": "",
"url": "",
"rdate": "",
"rrule": "",
"category": "",
"description": "",
"geo": { "a": false, "b": 123 }
}

the loading callback is never called. The code in SchemaValidate() is looking for a $ref with

    /* swap in any $ref'erenced schema */
    if((str = WJEString(schema, "$ref", WJE_GET, NULL))) {
        if(schema && freecb) {
            freecb(schema, client);
            schema = NULL;
        }
        schema = loadcb(str, client, __FILE__, __LINE__);
    }

where schema also points to the "geo" object/definition and everything looks fine but inside WJEString it returns without a match on this condition (cb is set, name is zero and depth is 1):

    if (!cb || !name || depth) {
        /* This portion of the path is invalid.  Bail. */
        return(NULL);
    }

Is there something I need to change?
Thanks in advance for any feedback.

Clemens

test memory ,found Coredump

Segmentation fault

Whe using the following code to test the lib,I found "Segmentation fault":
for(int i =0; i<10000;i++){
char json[] = "{"name":"Serenity","class":"firefly","crew":[{"name":"Malcolm Reynolds","job":"captain","born":2486},{"name":"Kaywinnet Lee Fry","job":"mechanic","born":2494},{"name":"Jayne Cobb","job":"public relations","born":2485}]}";
WJReader jsonReader =WJROpenMemDocument(json,NULL,0);
WJElement jsonDoc = WJEOpenDocument(jsonReader, NULL, NULL, NULL);
WJEDump(jsonDoc);
WJECloseDocument(jsonDoc);
}

when i >6000

pkg-config support?

Does this library have pkg-config support? If not, what would it take to convince you guys to include it?

Schema validation is incomplete

• "patternProperties" validation is unimplemented; JSON with invalid data in such a defined property will pass.
• "format" validation on strings is unimplemented. The json-schema draft specification allows this to be missing, and so far, so do we.

json object comparision in c using CompareJson API

Hi,
I want to compare to json objects , could you suggest any API.
Can I use Comparejson API by passing two json object for comparision in c language.

Thanks,
Mohan

maximum rules wrong when validate a negative integer

I want to deine an int type, it's schema is the following
{
"type": "integer",
"required": true,
"minimum": -2147483648,
"maximum": 2147483647
}

When i validate the value -1, the WJESchemaValidate is not passed the test.

I look thought the code in schema.c ,and find out that the function SchemaValidate call __WJEDouble to get the double value.

When the number is negative integer,WJEDouble return the wrong double value.
it use "return((double) -e->value.number.i);"．

for example
uint64 i64=1
double v = (double)-i64;

the v is negative value,it is a very large position value.
i think we can simply use
return((double)-((int64)e->value.number.i);

maybe it's not true for a very small negative integer.

JSON errors

I'm a little puzzled that wjelement seemingly misses a very fundamental feature, namely handling erroneous JSON streams.

WJEOpenDocument seems to swallow anything it doesn't recognise. So if I parse this non JSON string "{this 1 is truely not working}" it returns an empty object. I tried to use the callback I can provide to WJEOpenDocument, but in this example it is only called once with all parameters NULL.

Is there any way of handling errors?

Large json item seems to confuse the parser

The following code demonstrates the issue:

void test_large_json_pasring_wjelements() { char large_json[] = "{\"BeforeLargeList\":\"123\", \"LargeList\":\"939885|1157278|1296629|1178708|97069|1043125|1300908|1270232|1155448|1263160|871681|1163361|1301383|1283502|42588|68554|234022|759212|428474|1301969|1297102|1138831|1298159|89342|1191373|1151256|1139376|1301359|517236|1128823|1158484|1299600|1256903|303736|949111|472184|1301914|1212022|1227407|716390|1182637|1104734|1295203|679697|1058074|1012246|1290820|755601|254675|824505|697729|1301953|1211766|998718|1248163|460060|486542|405581|1301875|1302042|951938|770999|356894|90697|1295782|1058621|1297104|920706|858095|407433|126772|1255711|1014614|147012|1248711|796429|1301696|1259857|1301983|1291187|1195768|993064|1192559|1128938|1107914|1295671|1138650|1294147|1293446|1281496|1301915|954017|857049|550576|1199137|1227166|1196950|757883|1285141|771441|1212613|1271424|1055467|1124004|1053132|1053509|1062099|1273181|1301946|100936|1189434|1248264|1295205|688908|1070338|1301633|1289025|1249098|1280954|1301492|1268513|891174|1279140|467334|1105736|1294228|1301831|1062832|1258266|1270010|1207428|853700|1286559|1221342|872693|1160797|1292312|1243237|103934|1204446|1297895|83320|1000901|914019|1117550|1301993|1298674|1247437|983634|982737|1223072|1007442|968395|1268662|1276828|1277655|1302026|1103679|1284225|1205753|1295204|889705|1191401|1200436|1199151|851478|569613|888619|994579|107994|748779|1274214|1250704|382492|1166497|1300251|1299832|1283577|1191900|1292225|1182725|1294212|93915|1297778|1272804|1227712|1270251|124036|170874|1278829|1301337|45458|666641|1089462|2774|1281480|900584|1301986|1290139|1298560|516504|1157984|47996|653628|1289526|1029671|447092|1301241|1300273|1198688|652708|1230190|1171973|1124707|1299169|944119|1143102|738436|895649|1029247|1183460|878057|1293381|927095|1290755|1128419|1045068|766570|1131367|884502|1182827|1266603|1185678|786960|1014134|1132067|1246678|1244519|898058|475315|933531|1111878|873132|1301916|941712|1112534|648744|1300149|1225532|64190|1243988|1260643|1301083|829198|1012704|1278945|13022|1301877|874988|94814|1064634|1286722|1026002|11162|985256|1224769|681182|1073851|23144|1214138|869983|1186184|777764|1228525|1300412|1239022|1201294|905664|1292478|1258014|1291461|1222665|280490|1217944|1249031|1061970|1263093|1002415|844710|159469|1266086|760579|666299|88413|1219961|678144|1091530|1301511|977280|1238291|865355|1273635|1248176|994667|878576|74322|1026860|1251550|1177740|835657|987437|706274|1047876|1171782|1110476|1190173|1231247|1301306|1067388|1011268|1254957|1259972|1214183|1300419|1161915|1049067|1249404|1283178|882186|948621|642094|438277|771080|888703|800562|843845|1124144|1263997|894683|782875|1126517|1292677|885563|1260017|680070|993634|1290679|1238284|1226516|1153525|1306|1157554|1247728|1264683|1297786|1222034|1301949|976469|22270|413359|1294413|743329|778693|1145931|1288336|735721|709|1288453|1133211|1202357|1033620|1237308|1302037|1301932|661880|641296|1286053|553460|1263584|1301341|1276087|1299682|1291194|498297|730982|25906|1299586|1106427|706368|461045|1273147|381981|1260095|1300916|1086765|1267992|1277897|812543|1286975|1238357|1204406|880679|1001438|965377|956813|438781|1302028|50728|972672|815070|1300527|1292255|1271376|646023|1228195|1285397|142029|966992|1237953|995160|1083974|1060892|1300272|1300592|851195|1300502|1301145|1168853|1182560|1294384|1198950|1301931|1162528|931001|1002073|257267|1019483|1184937|554449|812077|1089410|1179217|1301984|47549|1189522|1268145|1283703|142431|1270168|1301535|967331|1189170|1266211|1269691|946793|38535|1274639|496514|752318|917110|1289751|1301985|781616|1297762|95564|1090453|545824|1274119|1100730|1281867|1302036|234215|15392|1302027|358039|980864|1294056|1165935|1301334|353797|1086682|907592|748960|1279279|1171479|1175742|985576|923611|1077471|679179|16612|1203504|1299259|787142|1295509|355630|949797|934049|1301307|542315|1248976|1028796|1282218|1017112|1288154|1085017|652160|1301374|831188|1077510|368165|1223767|1294472|1282320|1112567|1223567|1300789|1241268|1228776|1254890|794826|1287023|1301542|1299432|964535|1154135|726281|1254392|1250848|566568|43679|1189396|1246914|863151|1289220|1172752|1176838|974471|1132350|813875|1279593|1243604|1023135|845481|1301217|715692|1053117|1301836|1000260|1098758|381508|1282187|971331|1104076|1065206|1301936|1245020|1229744|69662|1301441|1161702|1301689|1267138|1021636|760954|839168|774842|550203|1299583|1301435|1054732|1214497|356509|1256484|146715|1094301|1277652|897270|1292339|1297526|1227432|1186202|1237960|299|42413|1279069|1289920|982910|1040630|1193330|886451|1179713|1280727|986326|1189997|1298694|148194|982277|1279722|1224395|908051|1277664|1109447|739573|984905|1301833|1299064|1297133|811208|1258547|1218229|921727|878763|1298619|1297819|1272705|1287710|994988|950673|1298156|1272835|1243326|974134|866853|1301782|1109487|1297991|723389|924189|1301863|1210319|521129|730877|754398|788466|459637|41952|18849|1190992|1283999|1033789|979521|1299801|1301765|1230972|1294289|1297808|1036279|1034656|1259099|1222209|627381|125488|545109|1292694|1301834|983935|550949|1204409|802377|1249602|1113624|1259776|1138958|1215939|1129089|1302020|1286224|846605|615995|134144|1012622|1299767|410946|992715|546479|1168395|1257030|1277459|1292785|1281177|1124239|873381|1256945|1024918|1247617|1298414|769811|684404|997533|695285|1299591|306608|1290586|423818|1156022|1049365|1201718\",\"Merge\":\"Evaluate\"}"; WJReader r = WJROpenMemDocument(large_json, NULL, 0); WJElement handle = WJEOpenDocument(r, NULL, NULL, NULL); char *Before = WJEString(handle, "BeforeLargeList", WJE_GET | WJE_IGNORE_CASE, ""); if (strcmp(Before, "123")) { printf("e: BeforeLargeList doesn't match expected value ('%s' != '%s') \n", Before, "123"); } char *Merge = WJEString(handle, "Merge", WJE_GET | WJE_IGNORE_CASE, ""); if (strcmp(Merge, "Evaluate")) { printf("e: Merge doesn't match expected value ('%s' != '%s') \n", Merge, "Evaluate"); } WJECloseDocument(handle); WJRCloseDocument(r); }

Examining a key before the long item seems to be working correctly, whereas the items that follows the large key seems to be incorrectly parsed.

Json key can't start from $

I'm working with mongdb. The Date object data of mongodb is mapped to json data
{ "$date": 3333333.2}．
I tested that the current version of wjelement can't validate '$date' field.
If I change the '$date' to 'date', it will work well.
I know some names start from '$' are the reserved keys, but i think orther names start from '$' should be treated as normal name if they aren't the reserved keys.

v4 "required" assertion applied for instance that is not object

Section 3.2.1 of JSON Schema Validation specification states:

Most validation assertions only constrain values within a certain
primitive type. When the type of the instance is not of the type
targeted by the keyword, the instance is considered to conform to the
assertion.

If the required keyword is used with an array value, per v4 syntax, it is asserted even if the instance being tested is not an object.

For example:
{"required":["name1"]} - Schema
{"name1":null} - Instance conforms
{} - Instance does not conform
null - Instance conforms but WJESchemaValidate() claims it does not

Below is code for an example console application, expected output and actual output.

#include <stdarg.h>
#include "stdafx.h"
#include "wjelement.h"

void ErrCB(void *client, const char *format, ...)
{
	va_list args;
	va_start(args, format);
	vprintf(format, args);
	va_end(args);
	printf("\n");
}

void RunTest(int index, WJElement schema, char* json)
{
	if (WJElement document = WJEParse(json)) {
		printf("Test %d\n", index);
		if (WJESchemaValidate(schema, document, &ErrCB, NULL, NULL, NULL)) {
			printf("OK\n");
		}
		WJECloseDocument(document);
		printf("\n");
	}
}

int main()
{
	XplBool valid;
	if (WJElement schema = WJEParse("{\"required\":[\"name1\"]}")) {
		RunTest(1, schema, "{\"name1\":null}");
		RunTest(2, schema, "{}");
		RunTest(3, schema, "null");
		WJECloseDocument(schema);
	}
	return 0;
}

The expected output is as follows:

Test 1
OK

Test 2
(root): required member 'name1' not found.

Test 3
OK

The actual output is as follows:

Test 1
OK

Test 2
(root): required member 'name1' not found.

Test 3
(root): required member 'name1' not found.

Direct access to array by index

I'd like to be able to get an array element by index. I don't see a way to do it. The best I've found is to create a selector from a template eg sprintf(selector, "[%d]", index).

Change licence to something more liberating

This licence does not allow of inclusion of this library into iOS applications. Any chance you can fork the project with MIT, BSD or some similar licence?

wjelement.h is missing function definition for WJEWriteMEM

The title says everything WJEWriteMEM is missing from wjelement.h which seems like an oversight.

No uninstall option in generated makefile

Is this intentional? If so, why? It seems like a strange thing to not give users the ability to automatically uninstall your library if they wish to.

schema validation: "format"

another documented quirk. should be pretty easy now that we optionally consume a regex lib.

Memleak while validating array elements

It seems that there is a leak in schema.c while processing array elements. You are allocating memory in line 739 (MemAsprintf) without freeing it. I added MemFree(str); at the end of the while-loop and that fixed it for me (tested with valgrind).

or am I missing something completely?

Greetings

Error use Raspberry pi 3

I did the installation, and compiled an example. But at the time of using the error appears.

./json: error while loading shared libraries: libwjelement.so.1: cannot open shared object file: No such file or directory

Help

Completion of error handling

I have looked at a few source files for your current software. I have noticed that some checks for return codes are missing.

Would you like to add more error handling for return values from functions like the following?

• MemStrdup ⇒ _WJECopy
• printf ⇒ usage

Inconsistency: on a mismatch in type array error cb may be called or not

If I validate

1

against

{
    "type": [
        {
            "type": "string"
        },
        "integer"
    ]
}

My error callback gets called upon the string mismatch.
However if I validate against

{
    "type": [
        "string",
        "integer"
    ]
}

it is not called.

[Question] Validate against just a portion of a schema file (an all in one validation file)

I've tried a lot of things stepping thru code and such, but how do I do schema validation against just a portion of a schema file. I tried using the "where" parameter of the WJEOpenDocument, but wasn't quite sure if that was right or what to pass in.

Sample schema:
{
"properties":{},
"validateAgainstThis":
{
"type": "object",
"properties":
{
}
}
}

When will release a new version?

Hey guys:

 It is almost one year since version 1.1 release.
 Could you release a new version, thanks.

Exponential memory growth in schema validation

Consider a simple schema like:
{
"$schema": "http://json-schema.org/draft-03/schema#",
"type": "array",
"items": {
"type": "object",
"properties": {
"headers": {
"type": "object",
"required": true,
"properties": {
"messageId": {
"type": "string",
"required": true
}
}
},
"properties": {
"type": "object",
"required": false
},
"body": {
"type": "string",
"required": true
}
}
}
}

If the "body" element is ~600 byte string, it takes 12K RAM to validate.
If the "body" element is ~308K byte string, it takes 17.6M RAM to validate.
If the "body" element is ~9.1M byte string, it takes 16.9G (yes, giga bytes) RAM to validate.

Schema Validation fails when missing required property in child but child not present

The object:

{
  "name": "myName"
}

The schema:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "myKey": {
           "type": "object",
           "properties": {
              "A": {
                 "type": "array",
                 "description": "Array of A object IDs",
                 "items": { "type": "integer" }
               },
              "B": {
                 "type": "array",
                 "description": "Array of B object IDs",
                 "items": {"type": "integer" }
               }
           },
          "required": ["A", "B"],
          "additionalProperties": false
        }
    },
    "additionalProperties": false
}

It shouldn't matter that A and B are missing because myKey wasn't required in its parent, right?

WJESchemaValidate returns false, with the error callback saying

myKey: required member '["A"]' not found.
myKey: required member '["B"]' not found.

This same object is marked as valid by this website: http://json-schema-validator.herokuapp.com/ (try it out)

C++ code used:

         char* testSchema = "{\n    \"$schema\": \"http://json-schema.org/draft-04/schema#\",\n    \"type\": \"object\",\n    \"properties\": {\n        \"name\": {\"type\": \"string\"},\n        \"myKey\": {\n           \"type\": \"object\",\n           \"properties\": {\n              \"A\": {\n                 \"type\": \"array\",\n                 \"description\": \"Array of stream object IDs\",\n                 \"items\": { \"type\": \"integer\" }\n               },\n              \"B\": {\n                 \"type\": \"array\",\n                 \"description\": \"Array of stream object IDs\",\n                 \"items\": {\"type\": \"integer\" }\n               }\n           },\n          \"required\": [\"A\", \"B\"],\n          \"additionalProperties\": false\n        }\n    },\n    \"additionalProperties\": false\n}\n\n";
         char* testString = "{\n  \"name\": \"myOtherName\"\n}";
         WJReader reader = WJROpenMemDocument((void*)testString, NULL, 0);
         WJElement doc = WJEOpenDocument(reader, NULL, NULL, NULL);
         WJReader testSchemaReader = WJROpenMemDocument((void*)testSchema, NULL, 0);
         WJElement testSchemaDoc = WJEOpenDocument(testSchemaReader, NULL, NULL, NULL);

         if(WJESchemaValidate(
               testSchemaDoc, doc,
               NULL, NULL, NULL, NULL))
         {
            printf("YES\n");
         }
         else
         {
            printf("NO\n");
         }

Add an inital git tag to the repo.

Would it be possible to create an initial git tag (and therefore a version) for WJElement? This simplifies distribution packaging a bit.

Why both the text of the GPL and the LGPL?

This makes it unclear what license this is being offered under. By the text of your README, I assume that you mean the LGPL, but it'd be nice to make it clear.

Schema case sensitivity inconsistent

The schema logic correctly requires the input property name to match case with the schema property name for validation against the schema object to occur, but the required and additionalProperties json-schema keyword validation is case insensitive. This allows for a situation where an input property name with the incorrect case passes the required validation, but is never matched against the expected schema object. Or in the case of additionalProperties, it fails to detect that a property with a differently cased name is included. See below for examples that demonstrate this behavior.

Case 1: required

schema:

  { 
    "type": "object", 
    "properties": { "test": { "type": "integer" } }, 
    "required": [ "test" ] 
  }

input1: { "test": "str" }
result1: FALSE, test is of incorrect type

input2: { "TEST": "str" }
result2: TRUE

Result1 is correct; result2 is incorrect. The second input passes the required validation, but is not validated against the lowercase "test" property, creating a false positive for the user.

Case 2: additionalProperties

schema:

  { 
    "type": "object", 
    "properties": { "test": { "type": "integer" } },
    "additionalProperties": false 
  }

input: { "TEST": "str" }
result: TRUE

The result is incorrect. The incorrectly cased property passes the additionalProperties validation, but is not validated against the lowercase "test" schema property, creating a false positive for the user.

Documents with large strings contain \u0000 in JSON Output

WJElement seems to be adding \u0000 to large strings. I observed this behavior while using the library within an application and found that it can be easily reproduced this way:

$ wje
wje> load test.json
wje> save test2.json
Wrote JSON document to test2.json
wje> exit

files:
test.json: http://pastebin.com/EZyU6Ex2
test2.json: http://pastebin.com/vVV8A1Ey

The test2.json file seems to be padded with many instances of \u0000. This only seems to happen with strings larger than about 2.5k. For example the JSON document below with a slightly smaller data string does not suffer from this issue: http://pastebin.com/tERWyze4

Optimize nodes access

Hi,
on long arrays adding new items is really slow. I presume it is because of chained-list-like implementation. Therefore I would like to optimize the library.
So far I came up with two things:

• add "last" pointer which would point to the last child, therefore improving adding new items to constant time access
• add some form of index-pointer table for constant time access using concrete index

Please, state your ideas on this, state possible pitfalls. Thanks.

Non-linear parsing performance

When using the following code:
WJReader r = WJROpenMemDocument(buffer, NULL, 0);
handle = WJEOpenDocument(r,NULL,NULL,NULL);

I've noted that the performance degrade as a function of n.

Unfortunately it's degrading in a non-linear way:
json parsing of 100000 items took : 0.204 per-element time : 0.000002
json parsing of 200000 items took : 0.697 per-element time : 0.000003
json parsing of 300000 items took : 1.465 per-element time : 0.000005
json parsing of 400000 items took : 2.639 per-element time : 0.000007
json parsing of 500000 items took : 4.190 per-element time : 0.000008
json parsing of 600000 items took : 6.559 per-element time : 0.000011
json parsing of 700000 items took : 9.934 per-element time : 0.000014
json parsing of 800000 items took : 14.148 per-element time : 0.000018
json parsing of 900000 items took : 18.535 per-element time : 0.000021

Any suggestions would be appreciated.

Corrupted string when reading then writing large json file

When reading a fairly large JSON file to a WJElement document, then immediately writing it back, I see several differences in the resulting file. I've included a picture of a diff tool I used, and the files (input and output) are here: https://gist.github.com/mehagar/4b7f1399568b2e0f8456

Here's the program I used:

// WJelementTest.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <fstream>
#include <iostream>
#include <string>
#include "wjelement.h"
#undef max
#undef min

size_t writeToStringCB(char* data, size_t size, void *writedata)
{
   std::string &s = *static_cast<std::string*>(writedata);
   s += data;
   return size;
}

int _tmain(int argc, _TCHAR* argv[])
{
   std::ifstream sFile;
   std::string line;
   std::string jsonInput;

   if (argc < 3) {
      std::cout << "USAGE: WJElementTest inFile outFile\n";
      exit(-1);
   }

   sFile.open (argv[1]);
   while (std::getline (sFile, line)) 
      jsonInput += line + "\n";
   sFile.close();

   // I get same results passing NULL and zero for buf, bufSize below.
   int bufSize = jsonInput.size() * 2;
   char* buf = new char[bufSize];

   WJReader reader = WJROpenMemDocument(static_cast<void*>(const_cast<char*>(jsonInput.c_str())), buf, bufSize);
   WJElement doc = WJEOpenDocument(reader, nullptr, nullptr, nullptr);

   std::string target;
   WJWriter w = WJWOpenDocument(TRUE, &writeToStringCB, &target);
   WJEWriteDocument(doc, w, NULL);
   WJWCloseDocument(w);

   std::ofstream oFile;
   oFile.open (argv[2]);
   oFile << target;
   oFile.close();

    return 0;
}

(root) is of incorrect type when parsing a string thorugh WJEParse and validating

I get this error everytime when i try to validate a JSON string with an schema. This JSON string is parsed through WJEParse like in the example _WJEParse("{ 'foo': true, 'bar': 'yup' }", ''');
Any idea?

Large Integer support

The largest integer WJElement can parse depends on the C integer limitation, but according to JSON spec there is no limitation about the Integer. Only when the content has decimal point even the decimal is the last char, WJElement will treat the content to be Number (double type).
Can WJElement support large integer? Always store any integer/number to double type.

JOSE support

We should look into implementing support for JOSE (JSON Object Signing and Encryption) in WJElement.

Schema pattern matching always comes back valid

Schema:

{
    "$schema": "http://json-schema.org/draft-03/schema#",
    "type": "object",
    "properties": {
      "key": {
        "type": "string",
        "pattern": "ABC"
      }
    }
}

JSON String to validate

{
  "key": "123"
}

Example

   std::string jsonString = "{\n  \"key\": \"123\"\n}";
   WJReader reader = WJROpenMemDocument((void*)jsonString.c_str(), NULL, 0);
   WJElement doc = WJEOpenDocument(reader, NULL, NULL, NULL);

   std::string schemaString = "{\n    \"$schema\": \"http://json-schema.org/draft-03/schema#\",\n    \"type\": \"object\",\n    \"properties\": {\n      \"key\": {\n        \"type\": \"string\",\n        \"pattern\": \"ABC\"\n      }\n    }\n}";
   WJReader SchemaReader = WJROpenMemDocument((void*)schemaString.c_str(), NULL, 0);
   WJElement SchemaDoc = WJEOpenDocument(SchemaReader, NULL, NULL, NULL);

   if(WJESchemaValidate(
      SchemaDoc, doc,
      NULL, NULL, NULL, NULL))
   {
      std::cout << "schema matches\n";
   }
   else
   {
      std::cout << "schema does not match\n";
   }

This prints "schema matches\n", as does any other setting of the pattern.

Is there a way to delete a property?

Is there a way to delete a property on a WJElement object?

For example an equivalent to the following in javascript:

delete object.property;

I was searching for something like:

WJEDelete(object, "property");

Support for floating-point numbers has not been implemented

Support for floating-point numbers was not needed in netmail, so they have not yet been implemented.

_WJEGet takes path as char*

Since the path is char* and not const char*, uses of constant strings cause compiler warnings.
Yet all _WJEGet does is pass the parameter to _WJESearch which takes a const pointer anyway.

Schema validation unexpectedly succeeds when key doesn't match patternProperties and additionalProperties=false

With this schema…

{
	"$schema": "http://json-schema.org/draft-04/schema#",
	"type": "object",
	"patternProperties": {
		"^string[0-9]+$": {
			"type": "string"
		}
	},
	"additionalProperties": false
}

and this document…

{"foo":"bar"}

…since the key doesn't match patternProperties, and additionalProperties is false, I'd expect the document to fail schema validation. But, with wjelement-1.2, it passes:

echo 'validate patternProperties-noAdditional.jsonschema' | ./wjecli test.json
Schema validation: PASS

How do I add a header to the JSON frame?

I want to add something that is not an object,it's a header. I wrote mi code in c with WJEObject and WJEString functions. I made this frame: "{"chargePointVendor":"Upna","chargePointModel":"Grid"}"

But I need to add this: [2, "19389", "{"chargePointVendor":"Upna","chargePointModel":"Grid"}"]

Any idea? Thanks

Valgrind complains about uninitialized memory access

Got the following valgrind trace:

==26413== Conditional jump or move depends on uninitialised value(s)
==26413==    at 0x4A0B0A9: strlen (vg_replace_strmem.c:458)
==26413==    by 0x8FD3E29: WJRFillBuffer (wjreader.c:234)
==26413==    by 0x8FD41A0: WJRDown (wjreader.c:316)
==26413==    by 0x8FD4C42: WJRNext (wjreader.c:822)
==26413==    by 0x6CF9016: _WJELoad (element.c:254)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF8FF2: _WJELoad (element.c:255)
==26413==    by 0x6CF920A: _WJEOpenDocument (element.c:328)
==26413==    by 0x6CF93EB: __WJEFromString (element.c:413)
==26413==    by 0x138AFB: validate_against_schema (protocol_schema_validator.c:58)
==26413==    by 0x137B2B: test_Agent_PerformTask_DownloadPackage (utest_agent_tasks.c:232)
==26413==    by 0x6AF0B98: cmocka_run_one_test_or_fixture (in /usr/lib64/libcmocka.so.0.4.1)
==26413==    by 0x6AF1480: _cmocka_run_group_tests (in /usr/lib64/libcmocka.so.0.4.1)
==26413==    by 0x1386AD: __wrap_main (utest_agent_tasks.c:313)
==26413==    by 0x6F29AE6: (below main) (libc-start.c:308)
==26413==  Uninitialised value was created by a heap allocation
==26413==    at 0x4A07EAF: malloc (vg_replace_malloc.c:299)
==26413==    by 0x93DD562: MemMallocEx (xpl.c:327)
==26413==    by 0x8FD43A8: _WJROpenDocument (wjreader.c:433)
==26413==    by 0x6CF93B8: __WJEFromString (element.c:412)
==26413==    by 0x138AFB: validate_against_schema (protocol_schema_validator.c:58)
==26413==    by 0x137B2B: test_Agent_PerformTask_DownloadPackage (utest_agent_tasks.c:232)
==26413==    by 0x6AF0B98: cmocka_run_one_test_or_fixture (in /usr/lib64/libcmocka.so.0.4.1)
==26413==    by 0x6AF1480: _cmocka_run_group_tests (in /usr/lib64/libcmocka.so.0.4.1)
==26413==    by 0x1386AD: __wrap_main (utest_agent_tasks.c:313)
==26413==    by 0x6F29AE6: (below main) (libc-start.c:308)
==26413== 

I traced it down to access to be before doc->read.
I extended the memset in _WJROpenDocument() from sizeof(WJIReader) to sizeof(WJIReader) + maxdepth and it seems to work now.

Can WJElement provide a way to retrieve any number value as same format as their original format?

I know WJElement has parsed the number value and stored it into number type (double / integer), so when trying to get the value, we lose the original format of the value. Can WJElement provide a way to return more information of the original value? Maybe when calling WJEString() on a number value can return the original string type of the value?