cesanta/mjson

C/C++ JSON parser, emitter, JSON-RPC engine for embedded systems

mjson - a JSON parser + emitter + JSON-RPC engine

Features

• Small, ~1k lines of code, embedded-friendly
• No dependencies
• State machine parser, no allocations, no recursion
• High level API - fetch from JSON directly into C/C++ by jsonpath
• Low level SAX API
• Flexible JSON generation API - print to buffer, file, socket, etc
• JSON-RPC client/server. Connects any microcontroller online via https://vcon.io

Parsing example

const char *s = "{\"a\":1,\"b\":[2,false]}";  // {"a":1,"b":[2,false]}

double val;                                       // Get `a` attribute
if (mjson_get_number(s, strlen(s), "$.a", &val))  // into C variable `val`
  printf("a: %g\n", val);                         // a: 1

const char *buf;  // Get `b` sub-object
int len;          // into C variables `buf,len`
if (mjson_find(s, strlen(s), "$.b", &buf, &len))  // And print it
  printf("%.*s\n", len, buf);                     // [2,false]

int v;                                           // Extract `false`
if (mjson_get_bool(s, strlen(s), "$.b[1]", &v))  // into C variable `v`
  printf("boolean: %d\n", v);                    // boolean: 0

Printing example

// Print into a statically allocated buffer
char buf[100];
mjson_snprintf(buf, sizeof(buf), "{%Q:%d}", "a", 123);
printf("%s\n", buf);  // {"a":123}

// Print into a dynamically allocated string
char *s = mjson_aprintf("{%Q:%g}", "a", 3.1415);
printf("%s\n", s);  // {"a":3.1415}
free(s);            // Don't forget to free an allocated string

JSON-RPC example

In the following example, we initialize JSON-RPC context, and call a couple of JSON-RPC methods: a built-in rpc.list method which lists all registered methods, and our own foo method.

The sender() implementation just prints the reply to the standard output, but in real life it should send a reply to the real remote peer - UART, socket, or whatever else.

#include "mjson.h"

// A custom RPC handler. Many handlers can be registered.
static void foo(struct jsonrpc_request *r) {
  double x;
  mjson_get_number(r->params, r->params_len, "$[1]", &x);
  jsonrpc_return_success(r, "{%Q:%g,%Q:%Q}", "x", x, "ud", r->userdata);
}

// Sender function receives a reply frame and must forward it to the peer.
static int sender(char *frame, int frame_len, void *privdata) {
  printf("%.*s\n", frame_len, frame); // Print the JSON-RPC reply to stdout
  return frame_len;
}

int main(void) {
  jsonrpc_init(NULL, NULL);

  // Call rpc.list
  char request1[] = "{\"id\": 1, \"method\": \"rpc.list\"}";
  jsonrpc_process(request1, strlen(request1), sender, NULL, NULL);

  // Call non-existent method
  char request2[] = "{\"id\": 1, \"method\": \"foo\"}";
  jsonrpc_process(request2, strlen(request2), sender, NULL, NULL);

  // Register our own function
  char request3[] = "{\"id\": 2, \"method\": \"foo\",\"params\":[0,1.23]}";
  jsonrpc_export("foo", foo);
  jsonrpc_process(request3, strlen(request3), sender, NULL, (void *) "hi!");

  return 0;
}

Build options

• -D MJSON_ENABLE_PRINT=0 disable emitting functionality, default: enabled
• -D MJSON_MAX_DEPTH=30 define max object depth, default: 20
• -D MJSON_ENABLE_BASE64=0 disable base64 parsing/printing, default: enabled
• -D MJSON_ENABLE_RPC=0 disable RPC functionality, default: enabled
• -D MJSON_DYNBUF_CHUNK=256 sets the allocation granularity of mjson_print_dynamic_buf
• -D MJSON_ENABLE_PRETTY=0 disable mjson_pretty(), default: enabled
• -D MJSON_ENABLE_MERGE=0 disable mjson_merge(), default: enabled
• -D MJSON_ENABLE_NEXT=0 disable mjson_next(), default: enabled
• -D MJSON_REALLOC=my_realloc redefine realloc() used by mjson_print_dynamic_buf(), default: realloc

Parsing API

mjson_find()

int mjson_find(const char *s, int len, const char *path, const char **tokptr, int *toklen);

In a JSON string s, len, find an element by its JSONPATH path. Save found element in tokptr, toklen. If not found, return JSON_TOK_INVALID. If found, return one of: MJSON_TOK_STRING, MJSON_TOK_NUMBER, MJSON_TOK_TRUE, MJSON_TOK_FALSE, MJSON_TOK_NULL, MJSON_TOK_ARRAY, MJSON_TOK_OBJECT. If a searched key contains ., [ or ] characters, they should be escaped by a backslash.

Example:

// s, len is a JSON string: {"foo": { "bar": [ 1, 2, 3] }, "b.az": true} 
char *p;
int n;
assert(mjson_find(s, len, "$.foo.bar[1]", &p, &n) == MJSON_TOK_NUMBER);
assert(mjson_find(s, len, "$.b\\.az", &p, &n) == MJSON_TOK_TRUE);
assert(mjson_find(s, len, "$", &p, &n) == MJSON_TOK_OBJECT);

mjson_get_number()

int mjson_get_number(const char *s, int len, const char *path, double *v);

In a JSON string s, len, find a number value by its JSONPATH path and store into v. Return 0 if the value was not found, non-0 if found and stored. Example:

// s, len is a JSON string: {"foo": { "bar": [ 1, 2, 3] }, "baz": true} 
double v = 0;
mjson_get_number(s, len, "$.foo.bar[1]", &v);  // v now holds 2

mjson_get_bool()

int mjson_get_bool(const char *s, int len, const char *path, int *v);

In a JSON string s, len, store value of a boolean by its JSONPATH path into a variable v. Return 0 if not found, non-0 otherwise. Example:

// s, len is a JSON string: {"foo": { "bar": [ 1, 2, 3] }, "baz": true} 
bool v = mjson_get_bool(s, len, "$.baz", false);   // Assigns to true

mjson_get_string()

int mjson_get_string(const char *s, int len, const char *path, char *to, int sz);

In a JSON string s, len, find a string by its JSONPATH path and unescape it into a buffer to, sz with terminating \0. If a string is not found, return -1. If a string is found, return the length of unescaped string. Example:

// s, len is a JSON string [ "abc", "de\r\n" ]
char buf[100];
int n = mjson_get_string(s, len, "$[1]", buf, sizeof(buf));  // Assigns to 4

mjson_get_hex()

int mjson_get_hex(const char *s, int len, const char *path, char *to, int sz);

In a JSON string s, len, find a string by its JSONPATH path and hex decode it into a buffer to, sz with terminating \0. If a string is not found, return -1. If a string is found, return the length of decoded string. The hex string should be lowercase, e.g. string Hello is hex-encoded as "48656c6c6f". Example:

// s, len is a JSON string [ "48656c6c6f" ]
char buf[100];
int n = mjson_get_hex(s, len, "$[0]", buf, sizeof(buf));  // Assigns to 5

mjson_get_base64()

int mjson_get_base64(const char *s, int len, const char *path, char *to, int sz);

In a JSON string s, len, find a string by its JSONPATH path and base64 decode it into a buffer to, sz with terminating \0. If a string is not found, return 0. If a string is found, return the length of decoded string. Example:

// s, len is a JSON string [ "MA==" ]
char buf[100];
int n = mjson_get_base64(s, len, "$[0]", buf, sizeof(buf));  // Assigns to 1

mjson()

int mjson(const char *s, int len, mjson_cb_t cb, void *cbdata);

Parse JSON string s, len, calling callback cb for each token. This is a low-level SAX API, intended for fancy stuff like pretty printing, etc.

mjson_next()

int mjson_next(const char *s, int n, int off, int *koff, int *klen, int *voff,
               int *vlen, int *vtype);

Assuming that JSON string s, n contains JSON object or JSON array, return the next key/value pair starting from offset off. key is returned as koff (key offset), klen (key length), value is returned as voff (value offset), vlen (value length), vtype (value type). Pointers could be NULL. Return next offset. When iterating over the array, koff will hold value index inside an array, and klen will be 0. Therefore, if klen holds 0, we're iterating over an array, otherwise over an object. Note: initial offset should be 0.

Usage example:

const char *s = "{\"a\":123,\"b\":[1,2,3,{\"c\":1}],\"d\":null}";
int koff, klen, voff, vlen, vtype, off;

for (off = 0; (off = mjson_next(s, strlen(s), off, &koff, &klen,
																&voff, &vlen, &vtype)) != 0; ) {
	printf("key: %.*s, value: %.*s\n", klen, s + koff, vlen, s + voff);
}

Emitting API

The emitting API is flexible and can print to anything: fixed buffer, dynamic growing buffer, FILE *, network socket, etc etc. The printer function gets the pointer to the buffer to print, and a user-specified data:

typedef int (*mjson_print_fn_t)(const char *buf, int len, void *userdata);

mjson library defines the following built-in printer functions:

struct mjson_fixedbuf {
  char *ptr;
  int size, len;
};
int mjson_print_fixed_buf(const char *ptr, int len, void *userdata);

int mjson_print_file(const char *ptr, int len, void *userdata);
int mjson_print_dynamic_buf(const char *ptr, int len, void *userdata);

If you want to print to something else, for example to a network socket, define your own printing function. If you want to see usage examples for the built-in printing functions, see unit_test.c file.

mjson_printf()

int mjson_vprintf(mjson_print_fn_t, void *, const char *fmt, va_list ap);
int mjson_printf(mjson_print_fn_t, void *, const char *fmt, ...);

Print using printf()-like format string. Supported specifiers are:

• %Q print quoted escaped string. Expect NUL-terminated char *
• %.*Q print quoted escaped string. Expect int, char *
• %s print string as is. Expect NUL-terminated char *
• %.*s print string as is. Expect int, char *
• %g, print floating point number, precision is set to 6. Expect double
• %.*g, print floating point number with given precision. Expect int, double
• %d, %u print signed/unsigned integer. Expect int
• %ld, %lu print signed/unsigned long integer. Expect long
• %B print true or false. Expect int
• %V print quoted base64-encoded string. Expect int, char *
• %H print quoted hex-encoded string. Expect int, char *
• %M print using custom print function. Expect int (*)(mjson_print_fn_t, void *, va_list *)

The following example produces {"a":1, "b":[1234]} into the dynamically-growing string s. Note that the array is printed using a custom printer function:

static int m_printer(mjson_print_fn_t fn, void *fndata, va_list *ap) {
  int value = va_arg(*ap, int);
  return mjson_printf(fn, fndata, "[%d]", value);
}

...
char *s = NULL;
mjson_printf(&mjson_print_dynamic_buf, &s, "{%Q:%d, %Q:%M}", "a", 1, "b", m_printer, 1234);
/* At this point `s` contains: {"a":1, "b":[1234]}  */
free(s);

mjson_snprintf()

int mjson_snprintf(char *buf, size_t len, const char *fmt, ...);

A convenience function that prints into a given string.

mjson_aprintf()

char *mjson_aprintf(const char *fmt, ...);

A convenience function that prints into an allocated string. A returned pointer must be free()-ed by a caller.

mjson_pretty()

int mjson_pretty(const char *s, int n, const char *pad,
                 mjson_print_fn_t fn, void *userdata);

Pretty-print JSON string s, n using padding pad. If pad is "", then a resulting string is terse one-line. Return length of the printed string.

mjson_merge()

int mjson_merge(const char *s, int n, const char *s2, int n2,
                mjson_print_fn_t fn, void *fndata);

Merge JSON string s2,n2 into the original string s,n. Both strings are assumed to hold objects. The result is printed using fn,fndata. Return value: number of bytes printed.

In order to delete the key in the original string, set that key to null in the s2,n2. NOTE: both strings must not contain arrays, as merging arrays is not supported.

JSON-RPC API

For the example, see unit_test.c :: test_rpc() function.

jsonrpc_init

void jsonrpc_init(void (*response_cb)(const char *, int, void *),
                  void *response_cb_data);

Initialize JSON-RPC context. The sender() function must be provided by the caller, and it is responsible to send the prepared JSON-RPC reply to the remote side - to the UART, or socket, or whatever. The sender() function receives the full frame to send, and the privdata poitner.

The response_cb() function could be left NULL. If it is non-NULL, it will be called for all received responses generated by the jsonrpc_call(). The response_cb() function receives full response frame, and the privdata pointer.

jsonrpc_process

jsonrpc_process(const char *frame, int frame_len, jsonrpc_sender_t fn, void *fdata, void *userdata);

Parse JSON-RPC frame contained in frame, and invoke a registered handler. The userdata pointer gets passed as r->userdata to the RPC handler.

jsonrpc_export

#define jsonrpc_export(const char *name,
                       void (*handler)(struct jsonrpc_request *));

Export JSON-RPC function. A function gets called by jsonrpc_process(), which parses an incoming frame and calls a registered handler. A handler() receives struct jsonrpc_request *. It could use jsonrpc_return_error() or jsonrpc_return_success() for returning the result.

NOTE: a name is a glob pattern that follows these rules:

• * matches 0 or more characters, excluding /
• ? matches any character
• # matches 0 or more characters
• any other character matches itself

For example, after jsonrpc_export("Foo.*", my_func);, the server triggers my_func on Foo.Bar, Foo.Baz, etc.

struct jsonrpc_request

struct jsonrpc_request {
  struct jsonrpc_ctx *ctx;
  const char *params;     // Points to the "params" in the request frame
  int params_len;         // Length of the "params"
  const char *id;         // Points to the "id" in the request frame
  int id_len;             // Length of the "id"
  mjson_print_fn_t fn;    // Printer function
  void *fndata;           // Printer function data
  void *userdata;         // userdata pointer passed to jsonrpc_process()
};

This structure gets passed to the method callback.

jsonrpc_return_success

void jsonrpc_return_success(struct jsonrpc_request *r, const char *result_fmt, ...);

Return result from the method handler. NOTE: if the request frame ID is not specified, this function does nothing.

jsonrpc_return_error

void jsonrpc_return_error(struct jsonrpc_request *r, int code, const char *message, const char *data_fmt, ...);

Return error from the method handler. JSON-RPC error frame looks like this:

{"id":1, "error": {"code": -32602, "message": "Invalid params", "data": {"foo": "bar"}}}

The frame contains a error object with numeric code and string message keys, and an optional data which can be arbitrary - a simple JSON type, or an array/object. In the optional data, you can pass some extra information about the error, for example a faulty request.

NOTE: if the request frame ID is not specified, this function does nothing.

JSON-RPC Arduino example

#include "mjson.h"

// Gets called by the RPC engine to send a reply frame
static int sender(const char *frame, int frame_len, void *privdata) {
  return Serial.write(frame, frame_len);
}

// RPC handler for "Sum". Expect an array of two integers in "params"
static void sum(struct jsonrpc_request *r) {
  int a = mjson_get_number(r->params, r->params_len, "$[0]", 0);
  int b = mjson_get_number(r->params, r->params_len, "$[1]", 0);
  jsonrpc_return_success(r, "%d", a + b);
}

void setup() {
  jsonrpc_init(NULL, NULL);     // Initialise the library
  jsonrpc_export("Sum", sum);   // Export "Sum" function
  Serial.begin(115200);         // Setup serial port
}

static void handle_serial_input(unsigned char ch) {
  static char buf[256];  // Buffer that holds incoming frame
  static size_t len;     // Current frame length

  if (len >= sizeof(buf)) len = 0;  // Handle overflow - just reset
  buf[len++] = ch;                  // Append to the buffer
  if (ch == '\n') {                 // On new line, parse frame
    jsonrpc_process(buf, len, sender, NULL, NULL);
    len = 0;
  }
}

void loop() {
  char buf[800];
  if (Serial.available() > 0) {
    int len = Serial.readBytes(buf, sizeof(buf));
    jsonrpc_process(buf, len, sender, NULL, NULL);
  }
}

When this sketch is compiled and flashed on an Arduino board, start Arduino Serial Monitor, type {"id": 1, "method": "Sum", "params": [2,3]} and hit enter. You should see an answer frame:

Example - connect Arduino Uno to AWS IoT device shadow

See https://vcon.io for more information.

Contact

Please visit https://vcon.io/contact.html

mjson's Issues

[bug]accept invalid numbers

The parser accepts invalid numbers (i.e., leading zeros).
This is not allowed according to the specification in section 2.4.

It can be reproduced with the input:
{"a":00000000}

Errors printing floating point numbers with trailing zeroes in the integer part

Some numbers get incorrectly formatted. Example: -600.nnnn. While -599.nnnn and -601.nnnn format correctly (with width=7).

The common factor to the mis-formatted numbers seems to be the trailing zeroes. I do have a potential fix for this; I'll open a PR shortly with that proposal.

Just wondering how to use the escape sequences.

#include "mjson.h"
#include <stdio.h>

int main(){

  const char *s = "{\"a\\u0061\": true}";
  int v;
  if (mjson_get_bool(s, strlen(s), "$.aa", &v))
    printf("boolean: %d\n", v);

  return 0;
}

Here I create an object with the key "aa", one "a" is directly written down, the other represented by an escape sequence.
Then I try to get the boolean value from that key, however it doesn't work. What's up with that?

JSONRPC request with no 'method' but with 'id'

A request like this, {"jsonrpc": "2.0", "params": [42, 23], "id": 1}.
Should we send back a error response with id?

mjson_printf incorrect

code:

char *s = nullptr;
mjson_printf(&mjson_print_dynamic_buf, &s, "{%Q:%g}", "value", 10.5454);
std::cout << s << std::endl;
free(s);

output:

{"value":1.54545}

expression must have constant value line 875

Hi there,

I recently downloaded the the code and implement it in my project. When I go to compile it I get a "expression must have a constant value on line 875. the code is "char path[klen + 1]. klen is declared as an int on line 870

I have not modified the code in anyway

Any suggestions?

[bug] very very slow in parsing specific numbers

the trigger input: 8891110122900e913013935755114

va_copy without va_end in mjson_vprintf

All calls to va_start and va_copy should have a corresponding va_end.
There should be a va_end in the 'M' formatter of mjson_vprintf.
Initially I thought it was as simple as adding the missing call, va_end(tmp).

But, I don't use MSC, and don't know about this:

#if !defined(_MSC_VER) || _MSC_VER >= 1700
#else
#define va_copy(x, y) (x) = (y)
#define snprintf _snprintf
#endif

Which looks like the va_end should not be there if using MSC, but should be there otherwise.
Thoughts?

confusing parse return value

mjson_get_XXX(...) return 0 or -1 for "not found", I suggest all use -1.
or better, return true on "found" and false on "not found", so these parser function can be used in if(...)

jsonrpc 2.0 member missing

According to the JSON-RPC 2.0 specification, a response object must contain a member specifying the protocol version as follows: "jsonrpc":"2.0"

Currently that member is missing from any response. However, I am not sure if mjson is supposed to adhere to the JSON-RPC 2.0 standard or not, as I could not find any mention of it in the documentation.

mjson_print_fixed_buf writes out of bounds on zero-length output buffer

When passing a zero length output buffer, mjson_print_fixed_buf (specifically, this line https://github.com/cesanta/mjson/blob/master/src/mjson.c#L479) writes out of bounds fb->len is -1.

I would expect to not write anything, but writing before the beginning of the output buffer is extra bad.

I happened to exercise this with something like mjson_snprintf(NULL, 0, "foo");.

See below patch for a unit test that exercises the issue.

diff --git a/test/unit_test.c b/test/unit_test.c
index 93bdd36..14579ed 100644
--- a/test/unit_test.c
+++ b/test/unit_test.c
@@ -412,6 +412,19 @@ static void test_print(void) {
     ASSERT(memcmp(tmp, str, 15) == 0);
     ASSERT(fb.len < fb.size);
   }
+
+    {
+    struct mjson_fixedbuf fb = {&tmp[1], 0, 0};
+    tmp[0] = (char)0xA5; // arbitrary non-ascii value
+    const char *s = "a\b\n\f\r\t\"";
+    // Unsure about what I would expect the return value to be
+    ASSERT(mjson_print_str(&mjson_print_fixed_buf, &fb, s, 7) == -1);
+    // wrote a null to buf[-1]
+    ASSERT(tmp[0] == '\0');
+    // This is what we should actually expect if it didn't write out of bounds.  This fails
+    ASSERT(tmp[0] == (char)0xA5);
+    ASSERT(fb.len < fb.size);
+  }
 }
 
 static int f1(mjson_print_fn_t fn, void *fndata, va_list *ap) {

Insufficient support for JSON path query.

The very nice mjson library really needs to support json fields with spaces in their names. For example, consider the following valid json:

{
  "translations": {
    "expression with spaces": {
      "fr": "Translation of Expression1 in French",
      "es": "Translation of Expression1 in Spanish",
      "de": "Translation of Expression1 in German"
    }
  }
}

The following JSONpath statements are valid to perform a lookup:

$.["translations"]["expression with spaces"]["fr"]
$.translations."expression with spaces".fr

But none will work in mjson:

    const char *s = \
    "{"
      "\"translations\": {"
        "\"expression with spaces\": {"
          "\"fr\": \"Translation of Expression1 in French\","
          "\"es\": \"Translation of Expression1 in Spanish\","
          "\"de\": \"Translation of Expression1 in German\""
        "}"
      "}"
    "}\"";
    
    const char *xp = "$.translations.\"expression with spaces\".fr";
    
    char *p;
    int m;
    int n = mjson_find(s, strlen(s), xp, &p, &m);

mjson_find will always return zero.

As such, there is no way to construct a JSONpath statement to query JSON objects that contain spaces in their names using mjson. This is a trivial JSONpath statement, no fancy functions here.

Multiple definitions when including mjson headers in several source files

I separately compile all my source files before linking as a final step. This causes mjson functions to be compiled multiple times, causing multiple definition errors during linking.

What are your thoughts on reverting to the previous .c + .h implementation? It's much more versatile, and as this library is now an arduino one, multiple source files shouldn't be a large burden on novice users.

Wrong values retrieved when parsing list of objects with different keys

---

name: bug report 🐞
title: parsing list of objects with different keys

Describe the bug

Wrong values are retrieved when parsing a list with objects with different keys

To Reproduce

It can be reproduced with a simple test where the json string has a list of objects where each object doesn't have the same keys of the others.

The mjson version used is 1.2.4 but the bug can be reproduced even with earlier versions.

static const char *json_string = "{\"request\":{\"foo\":\"bar\",\"users\":["
    "{\"name\":\"richard\",\"surname\":\"miller\"},"
    "{\"name\":\"john\",\"surname\":\"doe\",\"address\":\"abbey road\"},"
    "{\"name\":\"mike\",\"surname\":\"wazowski\",\"address\":\"trafalgar square\"}"
  "]}}";

int main(int argc, char **argv)
{
    char *token;
    int tok_len;
    int ret;

    int len = strlen(json_string);
    char *search_path = "$.request.users[0].address";

    ret = mjson_find(json_string, len, search_path, &token, &tok_len);
    printf("{%d} %s: %.*s (%d)\n", ret, search_path, tok_len, token, tok_len);
}

output:

$> ./mjson
{11} $.request.users[0].address: "abbey road" (12)

Expected behavior

The address key is not present in the first object (index 0) of the users list.
I would expect a MJSON_TOK_INVALID return code when parsing a key which is not present

Json parsing issue

Hi,

I have a json file (attached) that I am trying to parse. I am trying to parse components from the first array using a logic like below:
<<<
i = 0
snprintf(path, MAX_JSON_SIZE, "$.Info1[%d].Number", i);
while (mjson_get_string(buffer, strlen(buffer), path, tmp, sizeof(tmp)) != 0)
{
info_num = strtol(tmp, NULL, 16);
printf ("value read: %x ", info_num);
memset(tmp, 0, 64);
i++;
snprintf(path, MAX_JSON_SIZE, "$.Info1[%d].Number", i);
}

When I read back, I find that it is returning the 3 elements (index 0-2) of the first array. It is also returning elements from the second array (index 3-4).
Is this a bug in the library or am I not iterating correctly?

I also tried using mjson_find, but that is also returning the whole json back (with both arrays)

    bool isarray = mjson_find(buffer, strlen(buffer), "$.Info1", &Info, &len) == MJSON_TOK_ARRAY;

    printf("isArray temp: %d %d %s\n", isarray, len, Info);

Please advise.

Thanks!

when get json array index out of bounds.

/*
{
  "a": ["a1", "a2"],
  "b": ["b1", "b2", "b3", "b4", "b5", "b6"]
}
*/
char buf[] = "{\"a\":[\"a1\", \"a2\"],\"b\":[\"b1\",\"b2\",\"b3\",\"b4\",\"b5\",\"b6\"]}";
int len = strlen(buf);
char val[32];
mjson_get_string(buf, len, "$.a[1]", val, sizeof(val));  // ok
printf("val: %s\n", val);  // a2
mjson_get_string(buf, len, "$.a[2]", val, sizeof(val)); // fail
printf("val: %s\n", val); // b3

mjson_get_string returning 0 for any path

I do not believe my mjson_get_string function is working properly. In the documentation, it states it will return -1 if string is not found, or the length of the found string. However for any test case, it returns 0. Both paths that exist in my JSON and paths that are made up (i.e. "$.foobar" return 0. Shouldn't it return a -1 if it is not a valid path? Thanks

Here is my code...

int buffersize;
/* Function I wrote to read json, works properly and validated/tested */
char *buffer = <some_read_json_file_function>(<some path here>, buffersize)

char buf[50];
int test = mjson_get_string(buffer, buffersize, "$.flag", buf, sizeof(buf));

if(strcmp(cmdbuf,"x") == 0){
    <some code>
}
else{
    ERROR("value not found");
}
free(buffer);

JSON file

{"flag":"x"}

Based on the json file, length of path "$.flag" is 0 but clearly exists. If I do a random path, .foobar. It still returns 0.

README examples won't compile

I was trying mjson by reading the README, and I've noticed that the examples on the README won't compile. The parsing example uses a sub variable that is not declared, the emitting example uses mjson_snprintf instead of mjson_aprintf in the dynamic allocated string example.

The RPC example is also calling jsonrpc_export with wrong number of arguments.

JSON key with '/' in key name

I try to parse some jwt json string which looks like this.

{
    "iss":"kubernetes/serviceaccount",
    "kubernetes.io/serviceaccount/namespace":"openshift-logging",
    "kubernetes.io/serviceaccount/secret.name":"deployer-token-m98xh",
    "kubernetes.io/serviceaccount/service-account.name":"deployer",
    "kubernetes.io/serviceaccount/service-account.uid":"35dddefd-3b5a-11e9-947c-fa163e480910",
    "sub":"system:serviceaccount:openshift-logging:deployer"
}

That's my quick and dirty code

#include <stdio.h>  // For printf
#include <string.h> // For strlen

#include "../../src/mjson.h"

int main(){
  const char *jwt ="{\"iss\":\"kubernetes/serviceaccount\",\"kubernetes.io/serviceaccount/namespace\":\"openshift-logging\",\"kubernetes.io/serviceaccount/secret.name\":\"deployer-token-m98xh\",\"kubernetes.io/serviceaccount/service-account.name\":\"deployer\",\"kubernetes.io/serviceaccount/service-account.uid\":\"35dddefd-3b5a-11e9-947c-fa163e480910\",\"sub\":\"system:serviceaccount:openshift-logging:deployer\"}";
  size_t jwt_len = strlen(jwt);

  char p[100];
  
  // enum mjson_tok rc =  mjson_find(jwt, jwt_len, "$.iss", &p, &n);
  int rc = mjson_get_string(jwt, jwt_len, "$.\"kubernetes.io/serviceaccount/namespace\"", p, sizeof(p));
  
  printf("Info string length :%d:\n",rc);
  printf("Info :%s: :%d:\n",p,rc);
}

When I now run the program I expect the output openshift-logging bu I get this.

alex:~/Downloads/mjson-1.2.2/examples/jwt$ cc ../../src/mjson.c jwt.c -I../../src -o jwt_test
alex:~/Downloads/mjson-1.2.2/examples/jwt$ ./jwt_test 
Info string length :-1:
Info :: :-1:

Please can anyone point me to the right direction how to fix this, thank you.

Floating point precision

Hi all,

Is it planned to support emitting floats with variable precision?

Printing double with custom precision outputs wrong format

The precision argument while printing a double with given precision seems to be treated as width instead.

Documentation

%.*g, print floating point number with given precision. Expect int, double

Test 1

int num_precision = 10;
double num = 123.1234567891;
char *s = mjson_aprintf("{%Q:%.*g}", "num", num_precision, num);
printf("%s\n", s);
free(s);
// output: {"num":123.1234568}

Test 2

int num_precision = 13;
double num = 123.1234567891;
char *s = mjson_aprintf("{%Q:%.*g}", "num", num_precision, num);
printf("%s\n", s);
free(s);
// output: {"num":123.1234567891}

Question: Can mjson+jsonpath be done streaming?

Does mjson allow streaming jsonpath queries? I.e., can it use constant memory even for very large json documents?

Floating point precision

@boraozgen previously opened this issue but closed it because he no longer needed this functionality. However, the issue still needs to be fixed.

Using the mjson_printf() function with "%f", the variable with the maximum number of digits after the point is output, for example, in my example

{
  "telemetry":{
    "phaseA":{
      "Urms":"220.123000",
      "Irms":"0.100000",
      "p":"0.000000", ...
    }
  ...
  }
}

The standard way of limiting "%.(num)f" is not implemented. With large telemetry, like mine, the message becomes very long.

Now the only way to display floating numbers adequately is to use "%g":

{
  "telemetry":{
    "phaseA":{
      "Urms":"220.123",
      "Irms":"0.1",
      "p":"0", ...
    }
  ...
  }
}

but this way is more expensive in terms of performance than "%f"

I will ask the question again: Is it planned to support emitting floats with variable precision?

Additional documentation for parsing data from object arrays

Hello Cesanta Software,

I can improve the documentation by adding a simple code snippet for parsing an object from an object array. I used the mjson library for my own application mkhtml and used the string parsing system to parse data from the json that comes as a curl response.

/*  parsing json text data */
mjson_get_string(jsonData, strlen(jsonData), "$.choices[].text", htmlData, sizeof(htmlData));
fprintf(stdout, "%s\n", htmlData);

The JSON I parsed ($.choices[].text):

{"id":"cmpl-6dHFcMmoxmlVinYbEtRtmz3SOnRlu","object":"text_completion","created":1674819124,"model":"text-davinci-003","choices":[{"text":"\n\nYes, this is a test.","index":0,"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":9,"total_tokens":14}}

And I think adding a code snippet about this might be a good idea for developers. Because it is a bit hard to understand this concept from the documentation.

Thanks,
Mehmet Mert

Suggest using bool type instead of int in function mjson_get_bool()

because sizeof(bool) is not the same as sizeof(int), If someone put a bool var as the last param into funciton int mjson_get_bool(const char *s, int len, const char *path, int *v), adjacent memory with bool var may be overwritten.

So I suggest change the function to signature to int mjson_get_bool(const char *s, int len, const char *path, bool* b) since the name says "get bool"

Request for project (C)MakeFile

Hello,
I've been eyeing this project which looks very helpful for embedded C Json processing. I wonder if the project will consider adopting a top-level CmakeLists.txt file with instructions on how to configure/build the project.

I notice a Makefile exists within the test sub-directory, and found it interesting no similar such thing existed at the top-level for easy workflow.

For me adopting a CMakeLists.txt file would be helpful as I am evaluating using mjson as a library in a larger CMake based project. I think Cmake is a good option here in this case as it allows cross platform builds. In the context of mjson I notice there are alternative compile paths based on MSVC or non MSVC compiles. It seems like a fit use case for a cross platform build tool here.

I could open a pull request with a very basic CmakeLists.txt to expose the projects configure options and provide a basic build target. Thoughts?

Thanks!

how to mjson_set_string efficiently？

I analyze by path arr = split ( 'path' , '. ' )) then use mjson_merge() to diff merge. but i think is not good idea.

'jsonrpc_call' was not declared in this scope in ArduinoMqttExample

ArduinoMqttExample:22:66: error: 'jsonrpc_call' was not declared in this scope
ArduinoMqttExample:49:32: error: 'jsonrpc_call' was not declared in this scope

jsonrpc_call API has been removed. Maybe replace with mjson_printf?

mjson_find return wrong result.

    const char* j3 = "{\"a\":[],\"ab\":[{\"c1\":1,\"c2\":2},{\"c1\":3,\"c2\":4}]}";
    ASSERT(mjson_find(j3, strlen(j3), "$.a[0]", &p, &n) == MJSON_TOK_INVALID);  // this line FAIL!

    printf("mjson got: %d\n", mjson_find(j3, strlen(j3), "$.a[0]", &p, &n));  
    printf("got: %.*s\n", n, p); 

output:

mjson got 123
got: {"c1":1,"c2":2}

src/mjson.c: unused variable suspected by clang

Clang creates a warning about unused variables which was discovered by HAPoxy community haproxy/haproxy#1868

One suggestion is to create a pragma for clang haproxy/haproxy#1868 (comment)
Another option is to add a new option like -D MJSON_DEBUG=1 and change the code accordingly.

Something like the code below instead of this block https://github.com/cesanta/mjson/blob/master/src/mjson.c#L186-L192.

static int plen1(const char *s) {
  int i = 0;
#if MJSON_DEBUG
  int n = 0;
#endif

  while (s[i] != '\0' && s[i] != '.' && s[i] != '[') {
    i += s[i] == '\\' ? 2 : 1;
#if MJSON_DEBUG
    n++;
#endif
  }

#if MJSON_DEBUG
  printf("PLEN1: s: [%s], [%.*s] => %d\n", s, i, s, n);
#endif

  return n;
}

As I would like to create a Pull Request to mjson and then add the new version to HAProxy please let me know what's your preferred way is to solve the clang warning.

[Question] How to parse and get array from JSON string?

First of all, thank you for great work. I used it and works fine so far.

By the way, I can't seem to fine the way to get an entire array back instead of always know the exact location/index to get value back. Is there a way to do this?

json printf give wrong result for -10.f

json print -10.f gives -0.x error encoding.
I fixed this by
replace this line
https://github.com/cesanta/mjson/blob/HEAD/src/mjson.c#L604

while (t >= 1.0 && n + s < (int) sizeof(buf)) buf[n++] = '0', t /= 10.0;

with

while (t >= 1.0 && n + s < (int)sizeof(buf)) buf[s+n++] = '0', t /= 10.0;

suggestion: handling the BOM in json text

Hi, I find a rare case when utf-8 byte order mark (BOM) \xef\xbb\xbf involved.

Specifically, it will reject the json text when handling such case:

const char *s = "\xef\xbb\xbf{\"a\": [null, 1]}";
char *p;
int n;
int len = strlen(s);
int parsed_res = mjson_find(s, len, "$", &p, &n);

when we print the parsed_result, the value is 0, indicating the input json text is invalid.

JSON message exchanged in different platforms (e.g., Desktop and Embedded device) may contain the BOM prefix,
and the specification suggests we can (not a must) ignore that prefix.
Meanwhile, I checked that multiple JSON implementations in other platforms such as https://github.com/nlohmann/json, https://github.com/alibaba/AliOS-Things/tree/master/components/cjson really ignored the prefix, and they correctly parsed the json text.
Therefore, it would be better if this implementation also ignore the BOM prefix, thus can provide better interoperability.

RFE: mjson_get_int_number

The mjson_get_number returns a double as shown in the examples.

const char *s = "{\"a\":1,\"b\":[2,false]}";  // {"a":1,"b":[2,false]}

// Extract value of `a` into a variable `val` and print its value
double val;
if (mjson_get_number(s, strlen(s), "$.a", &val)) printf("a: %g\n", val);

But from my point of view should it be a int.

How about to keep the mjson_get_number for double values and add a function which returns the int value?

README.md typing errors

Some headings like:
msjon_printf -> mjson_printf
msjon_pretty -> mjson_pretty
msjon_merge -> mjson_merge

64-bit integer support

Hi all,

First of all thanks for publishing this great library.

I noticed that mjson_printf does not support 64 bit integers (llu and lld identifiers). Is there a particular reason for this (for example C89 compatibility)? The addition seems pretty simple and I could open a PR for it.

How to get all elements of a JSONPath array?

Pretty simple, passing something like $.cursus[*].grade gives back an array in Insomnia. How can I get the array elements using mjson?

Documentation error for mjson_tok_find return value

In the readme mjson_tok_find is documented with a return value of enum mjson_tok but in the header file it is defined as int.

jsonrpc_process calls mjson_print_fn_t char by char instead of full frame

Source code:

#include "mjson.h"
// Sender function receives a reply frame and must forward it to the peer.
static int sender(const char *frame, int frame_len, void *privdata) {
  // Print the JSON-RPC reply to stdout
  printf("%s - [%.*s]\n", __FUNCTION__, frame_len, frame);
  (void) privdata;
  return frame_len;
}

int main(void) {
  jsonrpc_init(NULL, NULL);

  // Call rpc.list
  const char request1[] = "{\"id\": 1, \"method\": \"rpc.list\"}";
  printf("request1 - %s\n", request1);
  jsonrpc_process(request1, strlen(request1), sender, NULL);

  return 0;
}

Output:

request1 - {"id": 1, "method": "rpc.list"}
sender - [{]
sender - ["]
sender - [i]
sender - [d]
sender - ["]
sender - [:]
sender - [1]
sender - [,]
sender - ["]
sender - [r]
sender - [e]
sender - [s]
sender - [u]
sender - [l]
sender - [t]
sender - ["]
sender - [:]
sender - [[]
sender - ["]
sender - [r]
sender - [p]
sender - [c]
sender - [.]
sender - [l]
sender - [i]
sender - [s]
sender - [t]
sender - ["]
sender - []]
sender - [}]
sender - [
]

unit_test failed.

I run the unit_test.c in arm clang. test_printf fail because "%M" point to jsonrpc_print_methods.

invalid conversion from void to mjson_print_fn_t

I'm getting an invalid conversion error trying to use jsonrpc from a class. Any idea ?

src/jsonrpc.cpp: In member function 'void JSONRPC::process(const char*)':
.pio/libdeps/esp32/mjson/src/mjson.h:193:80: error: invalid conversion from 'void (*)(const char*, int, void*)' to 'mjson_print_fn_t {aka int (*)(const char*, int, void*)}' [-fpermissive]
   jsonrpc_ctx_process(&jsonrpc_default_context, (buf), (len), (fn), (fnd), (ud))
                                                                                ^
src/jsonrpc.cpp:21:5: note: in expansion of macro 'jsonrpc_process'
     jsonrpc_process(jsonRequest, strlen(jsonRequest), JSONRPC::jsonReplier, NULL, (void *) "hi!");
     ^
.pio/libdeps/esp32/mjson/src/mjson.h:183:6: note:   initializing argument 4 of 'void jsonrpc_ctx_process(jsonrpc_ctx*, const char*, int, mjson_print_fn_t, void*, void*)'
 void jsonrpc_ctx_process(struct jsonrpc_ctx *ctx, const char *req, int req_sz,
      ^

// jsonrpc.h
#ifndef JSONRPC_h
#define JSONRPC_h

#pragma once

#include "mjson.h"
class JSONRPC
{
    public:
        JSONRPC();

        void init();
        void process(const char* jsonRequest);
        
        static void myCommand(struct jsonrpc_request *r);
        static void jsonReplier(const char *frame, int frame_len, void *privdata);

    private:
};

extern JSONRPC jsonrpc;
#endif








// jsonrpc.cpp
#include "jsonrpc.h"
#include "mjson.h"

JSONRPC::JSONRPC() {};

void JSONRPC::init()
{
    jsonrpc_init(NULL, NULL);
    jsonrpc_export("myCommand", JSONRPC::myCommand);
}

void JSONRPC::process(const char* jsonRequest) {
    jsonrpc_process(jsonRequest, strlen(jsonRequest), JSONRPC::jsonReplier, NULL, NULL);
}

void JSONRPC::myCommand(struct jsonrpc_request *r)
{
    jsonrpc_return_success(r, "%d", 12345);
}

void JSONRPC::jsonReplier(const char *frame, int frame_len, void *privdata) {
    Serial.println("here i have to reply to the sender");
}

Functions in parsing API don't stop once element is found

The entirety of the json data is parsed, even after the element you are trying to find/get is found/got.

For example, if your json data contains 1000 elements and you want to get one that happens to be the 1st token, you would expect it to stop there as it is unnecessary to continue (waste of resources, especially on embedded systems).

Path to JSON array returns MJSON_TOK_INVALID

for a give JSON string, "$.cars" returns MJSON_TOK_INVALID.

Commit information,
SHA-1: f995fe6
commit message: Fix multiple context handling.

{
    "cars": [
        {
            "name": "Volksvagen Vento",
            "model": 2015,
            "color": "NightBlue"
        },
        {
            "name": "Maruti Swift Desire",
            "model": 2010,
            "color": "SilverGery"
        },
        {
            "name": "TATA Tiago",
            "model": 2019,
            "color": "Pearl White"
        },
        {
            "name": "Mahindra Thar",
            "model": 2018,
            "color": "Tangy Yellow"
        }
    ]
}

mjson_get_number retrun invaild value

I tried testing the following code, the former always returns as expected but the latter always returns the wrong value

 const char *s = "{\"a\":1.2388,\"b\":[2,false]}";  // {"a":1,"b":[2,false]}

double val;                                       // Get `a` attribute
if (mjson_get_number(s, strlen(s), "$.a", &val))  // into C variable `val`
  rt_kprintf("a: %g\n", val);                         // a: 1


msh >mjson_test
**a: 1.2388**

static void foo(struct jsonrpc_request *r) {
  double x;
  if(mjson_get_number(r->params, r->params_len, "$[0]", &x)){
      rt_kprintf("x: %g\n", x);
  }


char request3[] = "{\"id\": 2, \"method\": \"foo\",\"params\":[1.2,1,1.3]}";
jsonrpc_export("foo", foo);
jsonrpc_process(request3, rt_strlen(request3), sender, RT_NULL, (void *)"hi!");

msh >mjson_test
**x: 4.66726e-62**
**{"id":2,"result":{"can_id":4.66726e-62,"":"ud"}}**

Rare problem of outputting negative float numbers

About a year ago I used in my project an old version of mjson which consisted of one header file and a slightly different API. In this version Sergey Lyubka was specified in copyright notice. Everything seemed to work correctly in it.

The project consists in the fact that the board sends data in json format to the server, and I can see the data for the last day in the server logs. I do not know which logger is used on the server (I did not write applications for this server), but I assume that it outputs data in the form in which it came from the board

A few months ago I updated the mjson in the project to version 1.2.5. After that, I noticed that sometimes strange telemetry comes from the board. Forming a json string, I am using the API mjson_printf as follows:

  float v = &data.p;
  ...
  mjson_printf(&mjson_print_fixed_buf, &out, "%Q:\"%g\",", "p", *v++);
  ...

I noticed that very rarely and only for negative real numbers that I print using the %g specifier, something like this is output:

  \"p\":\"-80\u0002.01\",

\uXXXX can be \u0002, \u0003 or \u0005. This insertion always happens for negative numbers in the integer part and can happen in different variables (not only for "p"). To understand how often this happens, I attach a graph based on telemetry values for a day. Data comes to the server every 5 seconds, I replaced these strange values with -6666 so that you can see how often they appear.

Boards with the old version of mjson do not send such values, but I did not find a place in the mjson library code where such an escape sequence is written to the output json string.

Is it possible for such a sequence to occur when using mjson_printf()?

Bug in mjson_find(..) function

The JSON string is,

{
"cars":[
{
"name":"car1",
"model":1965,
},
{
"name":"car2",
"model":1978,

    },
    {
        "name":"car3",
        "model":1984
    },
    {
        "name":"car4",
        "model":1999
        "color":"nightblue"
    }
]

}

error: "$.cars[0].color" returns "nightblue"
error: "$.cars[1].color" returns "nightblue"
error: "$.cars[2].color" returns "nightblue"