rxi/json.lua

A lightweight JSON library for Lua

A lightweight JSON library for Lua

Features

• Implemented in pure Lua: works with 5.1, 5.2, 5.3 and JIT
• Fast: generally outperforms other pure Lua JSON implementations (benchmark scripts)
• Tiny: around 280sloc, 9kb
• Proper error messages, eg: expected '}' or ',' at line 203 col 30

Usage

The json.lua file should be dropped into an existing project and required by it:

json = require "json"

The library provides the following functions:

json.encode(value)

Returns a string representing value encoded in JSON.

json.encode({ 1, 2, 3, { x = 10 } }) -- Returns '[1,2,3,{"x":10}]'

json.decode(str)

Returns a value representing the decoded JSON string.

json.decode('[1,2,3,{"x":10}]') -- Returns { 1, 2, 3, { x = 10 } }

Notes

• Trying to encode values which are unrepresentable in JSON will never result in type conversion or other magic: sparse arrays, tables with mixed key types or invalid numbers (NaN, -inf, inf) will raise an error
• null values contained within an array or object are converted to nil and are therefore lost upon decoding
• Pretty encoding is not supported, json.encode() only encodes to a compact format

License

This library is free software; you can redistribute it and/or modify it under the terms of the MIT license. See LICENSE for details.

json.lua's Issues

How to install it for Tengine

I'm using Tengine. How to add this lib for lua?

Large numbers serialized incorrectly

I have a lua object containing a large number:

local obj = {prop=1925036013156063406}

When the object is serialized, the JSON looks like:

{"prop":1.9250360131561e+18}

and can not be parsed by other JSON libraries on other programming languages.

The correct serializaion output would be:

{"prop":1925036013156063406}

Unable to encode 'sparse array's

What is the problem with parsing it like this?

local function encode_table(val, stack)
  local res = {}
  stack = stack or {}

  -- Circular reference?
  if stack[val] then error("circular reference") end

  stack[val] = true

  local isArray = true
  for k in pairs(val) do
    if type(k) ~= "number" then
      isArray = false
      break
    end
  end
  if isArray then
    -- Encode
    for i=1, #val do
      table.insert(res, encode(val[i], stack))
    end
    stack[val] = nil
    return "[" .. table.concat(res, ",") .. "]"
  else
    -- Treat as an object
    for k, v in pairs(val) do
      if type(k) ~= "string" then
        error("invalid table: mixed or invalid key types")
      end
      table.insert(res, encode(k, stack) .. ":" .. encode(v, stack))
    end
    stack[val] = nil
    return "{" .. table.concat(res, ",") .. "}"
  end
end```

Floating point numbers with dot causing error

Parsing such document

{ "dt": 1548784800, "main": { "temp": -2.05, "temp_min": -2.18, "temp_max": -2.05, "pressure": 998.5, "sea_level": 1019.79, "grnd_level": 998.5, "humidity": 89, "temp_kf": 0.12 }, "weather": [ { "id": 800, "main": "Clear", "description": "clear sky", "icon": "01n" } ], "clouds": { "all": 36 }, "wind": { "speed": 1.56, "deg": 233.002 }, "rain": {}, "snow": { "3h": 0.004 }, "sys": { "pod": "n" }, "dt_txt": "2019-01-29 18:00:00" }

gives an error:
invalid number '-2.05' at line 4 col 17

Please publish json.lua on LuaRocks

Please publish json.lua on LuaRocks, the de facto standard Lua packages repository. This issue has been already raised in #3, but the submitter closed it without any reason.

Decoding fails for JSON with array of strings within an object.

Hi,

This example seems to cause errors:
{"this":["is", "a", "minimal", "example"]}

It gives an error at symbol 10; the opening square bracket.

一个json字符串，decode成table,之后再把这个table encode成json，俩个json不一样！

local json_before = {"key":[{},{},{},{},{},{},{},{},{},{}],"roleAttr":{"3":0,"4":0,"5":0,"6":0,"7":0,"8":0,"9":0,"0":0,"1":0,"2":0}}
local list = json.decode(json_before );
local json_after = json.encode(list);
此时的json_afre为：{"key":[[],[],[],[],[],[],[],[],[],[]],"roleAttr":{"1":0,"0":0,"3":0,"2":0,"5":0,"4":0,"7":0,"6":0,"9":0,"8":0}}

为何不一样？

decoding of object with 'duplicate' key

in lua:

> dump( json.decode("{1: 1, '1':2}"))
{ [1] = 1,["1"] = 2,} 

whereas in javacript we have:

> Object.entries({1: 1, '1':2}).length
1

so in lua we should probably have:

> dump( json.decode("{1: 1, '1':2}"))
{["1"] = 2,} 

Empty object encodes as array

% lua
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio
> json=require'json'
> json.encode(json.decode('{}'))
[]

Why doesn't return '{}'? (Maybe related to #19)

Thanks,

json.lua:170: unexpected character 's' at line 1 col 1

If I comment the line 170 it works, if not :

json.lua:170: unexpected character 's' at line 1 col 1
stack traceback:
	[C]: in function 'error'
	/home/.../json.lua:170: in function 'decode_error'
	/home/.../json.lua:368: in function 'parse'
	/home/.../json.lua:376: in function 'decode'
	const.lua:566: in function 'json_test'
	const.lua:572: in main chunk
	[C]: in function 'dofile'
	...urin/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: in ?

Also, how to read a json file in lua and write the result of encode back to file?

Why not treat as object when mixing key types?

And then allow numbers for keys too, is there a technical reason why it shouldn't be done?

json encode error

Large numbers lose precision when decoding

A web service I need to use is returning the following in its JSON:
"Id":1618065507111835497

But when using this Lua library to decode it, the result is losing precision as "Id" is returned as:
1.6080655071118e+18

In this case I can't change what the webservice is providing. I'm quite happy referring the data as a string - it's just an id value.
I am limited to only using Lua 5.1.4.

Any help appreciated (e.g. pass back numbers that can't be correctly converted as strings?).

Thanks,
Rob

bad argument #1 to 'rawget' (table expected, got string)

I am new to lua.
A case, i was built a nginx + luajit +redis2.
I was used json.lua and got follow errer message:

2021/02/07 08:42:02 [error] 37#37: *11 failed to run body_filter_by_lua*: /etc/nginx/redis.lua:238: bad argument #1 to 'rawget' (table expected, got string)
stack traceback:
	[C]: in function 'rawget'
	/etc/nginx/redis.lua:238: in function 'decode'
	/etc/nginx/ueditor/ueditor_upload_file_rewrite_response.lua:22: in function </etc/nginx/ueditor/ueditor_upload_file_rewrite_response.lua:1> 

code fragments

if eof then
    local whole = table.concat(ngx.ctx.buffered)
    local ret_body = whole
    if whole then
        ngx.ctx.buffered = nil
        local body_json = json.decode(whole)     #################### line number 22 ####################
        local ret_table = {}
        if body_json then
            ret_table['original'] = body_json['originalFilename']
            ret_table['size'] = body_json['size']
            ret_table['state'] = 'SUCCESS'
            ret_table['title'] = body_json['originalFilename']:match("(.+)%.%w+$")
            ret_table['type'] = body_json['file']:match(".+(%.%w+)$")
            local imageUrl = body_json['thumbnail']
            if imageUrl and imageUrl ~= "" then
                ret_table['url'] = imageUrl
            else
                ret_table['url'] = body_json["file"]
            end
        else
            ret_table['state'] = 'Error: upload file failed!'
        end
        ret_body = json.encode(ret_table)
    end
    ngx.arg[1] = ret_body
end

Any help appreciated .
Thanks,

Annotated Tags Please

Suggestion / Request to use annotated tags for version tag numbers.

This is considered "best practice" by a number of open source projects and annotated tags have a number of advantages for official releases.

For our purposes, it will simplify the way that we use 'git describe' to collect information about packages that are a part of our build. By default, git describe ignores "lightweight tags"

Many thanks!

--

Please see a related request here:
inotify-tools/inotify-tools#124

Output is all on one line

Output is all on one line with \n instead of a return
and the return statement is still there.

	local treeFile = io.open("TreeData/"..treeVersion.."/tree.lua", "r")
	if treeFile then
		treeText = treeFile:read("*a")
		treeFile:close()
		f = io.open("TreeData/"..treeVersion.."/tree.json", "w")
		f:write(json.encode(treeText))
		f:close()
        end

decode simple table error

local z = {
    [2] = 3
}
local testVar = {
    [1] = {"2", "3"},
    [31] = z
}

local str = json.encode(testVar)
print(str)

Please add recursion / unlimited depth support

Currently the lib does not seem to support nested tables a.k.a. unlimited depth, e.g.

luaTable = { innerTable = { value1 = "value", value2 = "value" }, value1 = "value }

won't be supported currently. So I would like to request to support any kind of nested structure by using recursion.

empty arrays not handled correctly

Parsing JSON with empty arrays does not work.
Example:
local json = loadfile("json.lua")()

d = json.decode("{"tags":[]}")
local fmt = string.format

print( fmt("[test] %s ", d.tags) )

Error

expected ':' after key at line 1 col 5

No functions support

If table contains function, this happens: " json.lua:130: unexpected type 'function' "

Luarock for json.lua

Are you going to distribute json.lua as a Luarock module? It seems to me, it would be very convenient - just add json.lua as a luarock dependency.

Invalid number like '0123' or '1.' should not be decoded without raising errors

Strings like '0123' or '1.' are invalid as described in ECMA-404 and can not be parsed to numbers. However, the function tonumber() regards such strings convertible to numbers.

So the following code:

json.decode('1.')

will return the value 1 without any errors, which go against the JSON standard.

Missing decode bench for json4lua

For some reason json4lua's decode is commented out on bench_decode:

https://github.com/rxi/json.lua/blob/master/bench/bench_decode.lua#L8

When I uncomment it out, it locks up.

What's going on?

Not working with NGNIX

I installed nginx-full and trying to use json.lua with NGINX:

Example:

nginx.conf:

location /test { default_type 'text/plain'; content_by_lua_block { local json = require("json") ngx.say(json.decode('[1,2,3,{"x":10}]')) }

then curl http://localhost:8080/test and got 500 error. But everything is ok in lua console:

`> json = require("json")

json.encode({ 1, 2, 3, { x = 10 } })
[1,2,3,{"x":10}]`

What I am doing wrong?

Encoding runs in worst-case quadratic time

Consider the following benchmark which encodes the deeply nested structure [1000, [999, [998, [...]]]]:

local json = require"json"
for _ = 1, 1e3 do
	local t = {}
	for i = 1, 1e3 do
		t = {i, t}
	end
	json.encode(t)
end

this should run in a fraction of a second. Instead, it takes ~2 seconds on my device. This is because string concatenations / recursive table.concat are internally used to encode the JSON:

return "[" .. table.concat(res, ",") .. "]"

This means that the outer loop will have to add the [ and ] brackets in time linear in table.concat(res, ","), which in our example will be just marginally shorter, which in turn will only differ in length by a constant. Thus you do a linear time operation linearly often, resulting in worst-case quadratic encoding runtime in the depth of the nested structure.

The proper fix is to use a shared rope and have all recursive calls write to the same rope. You then only concatenate strings once at the end. Building the string then runs in linear time. You can find an example of this approach in my JSON implementation.

Uh how do i use this in lua... the readme.md was very vague

pcall json.decode failing, "attempt to call a table value"

I've tried to wrap json.decode within a pcall yet it is not succeeding as expected. Below is the minimum code that can produce this issue; using jit (lua 5.1) built within love2d 11.3

local json = require("json")
local str = [[{"foo":"bar"}]]
local contents = json.decode(str) -- works as expected
local success, contents = pcall(json.decode(str)) --  fails, success returns false
if not success then
  error(contents) -- Throws "attempt to call a table value"
end

The error message attempt to call a table value. This might be just my poor understanding of pcall as it's one of the first times I've started to use it.

Benchmark results

Any chance you could provide the benchmark results in a section of the README? It helps to evaluate the options out there. I appreciate how concise and portable your module is.

There is an idea to optimize the code for json.lua

When used .. in loops, the lua will create a lot of garbage and thus affect performance. I changed your script (json-fix.lua)

c:\test>busybox diff -U 0 json.lua json-fix.lua
--- json.lua
+++ json-fix.lua
@@ -41 +41 @@
-local json = { _version = "0.1.2" }
+local json = { _version = "0.1.2fix" }
@@ -235 +235 @@
-  local res = ""
+  local res = {}
@@ -246 +246 @@
-      res = res .. str:sub(k, j - 1)
+      table.insert(res, str:sub(k, j - 1))
@@ -253 +253 @@
-        res = res .. parse_unicode_escape(hex)
+        table.insert(res, parse_unicode_escape(hex))
@@ -259 +259 @@
-        res = res .. escape_char_map_inv[c]
+        table.insert(res, escape_char_map_inv[c])
@@ -264,2 +264,2 @@
-      res = res .. str:sub(k, j - 1)
-      return res, j + 1
+      table.insert(res, str:sub(k, j - 1))
+      return table.concat(res), j + 1

I created a .HAR file (size 313971312 bytes) using firefox and tested it with a simple Lua script.

local function read(file)
	local f, err = io.open(file, "r");
	if not f then
		return;
	else
		local data = f:read("a");
		f:close();
		return (data);
	end
end

local function test(obj)
	print (obj._version)
	local last = os.time();
	local har = obj.decode(read(file));
	print ('decode - ' .. os.time() - last .. " sec")
end

json    = require "json";
jsonfix = require "json-fix";
file    = 'Archive [21-05-25 11-43-43].har';

test(json)
test(jsonfix)

I got results like this on my computer (Intel(R) Core(TM) i3-9100 CPU @ 3.60GHz)

c:\test>lua read_har.lua
0.1.2
decode - 145 sec
0.1.2fix
decode - 23 sec

c:\test>lua read_har.lua
0.1.2
decode - 144 sec
0.1.2fix
decode - 24 sec

c:\test>lua read_har.lua
0.1.2
decode - 147 sec
0.1.2fix
decode - 22 sec