hydrobyte/mcjson

A Delphi / Lazarus / C++Builder simple and small class for fast JSON parsing.

mcjson's Issues

When value is a json string, tostring does not handle escaping

When value is a json string, tostring does not handle escaping

HasKey exception

Hi!

If I want to check whether a particular key exists in a json file (eg config file), I can check it with the HasKey function. The only problem is that it gives an exception (object reference is nil), even though the boolean return value would be enough.
During development, it is very unfortunate if unnecessary exceptions interrupt the program execution.

Thank you!

  JsonConfig := TMcJsonItem.Create;
  JsonConfig.LoadFromFile(cConfigFile);

  if not(JsonConfig.HasKey(cOptions)) then;

ToString using string buffer instead concatenation

Hello,

Would a String Buffer, String Stream, Binary Stream... be used instead String Concatenation when invoking ToString?

Delphi has TStringBuilder Class.

Best regards.

Unknown escape at pos (escpaing support)

This simple code cause errors (for example in config file):

N.AsJSON := '{"files": {"f1":"c:\a\a.json", "kf":"d:\a\b.json"}}';

It wouldn't be bad if this json library supported json special characters, see:

https://www.tutorialspoint.com/json_simple/json_simple_escape_characters.htm

Check function speed problem

Hi!

There is an approx. My 11 MB json file, in the following structure (~20k block):

"<number>": {
       "key1": "value1",
       "key2": "value2",
       "key3": "value3",
       "key4": "value4",
     },
"<number+1>": {
       "key1": "value1",
       "key2": "value2",
       "key3": "value3",
       "key4": "value4",
     },

The Check function takes seconds. The interesting thing is that if I slightly put a main key (object) in front of it, the Check function will be much faster, which I find completely pointless.

So the structure will be like this:

"JsonData": {
"<number>": {
       "key1": "value1",
       "key2": "value2",
       "key3": "value3",
       "key4": "value4",
     },
"<number+1>": {
       "key1": "value1",
       "key2": "value2",
       "key3": "value3",
       "key4": "value4",
     },

If I mess up the structure somewhere, it finds the error in the same way, so I think checking the json structure of the first version can take so long because of some error.

Range check error

Hi!

This simple code cause range check error (in debug mode) at line 1155
(Lazarus 2.2.4 official, x64 on windows)

procedure TForm1.Button1Click(Sender: TObject);
var
N: TMcJsonItem;
begin
N := TMcJsonItem.Create;
try
N.AsJSON := '{"o": {"k1":"v1", "k2":"v2"}}';
ShowMessage(N.Path('o.k2').AsString);
finally
N.Free;
end;

end;

MCJsonUnEscapeString "/" and "\" missing after unescape

Hi!

Very thank you fro the new valuable functions, but I found a little error:

procedure TForm1.Button1Click(Sender: TObject);
var
  s: string;
begin
  s:='A / B';

  Memo1.Lines.Add(s);
  s:= MCJsonEscapeString(s);
  Memo1.Lines.Add(s);
  s:= MCJsonUnEscapeString(s);
  Memo1.Lines.Add(s);

  s:='A \ B';

  Memo1.Lines.Add(s);
  s:= MCJsonEscapeString(s);
  Memo1.Lines.Add(s);
  s:= MCJsonUnEscapeString(s);
  Memo1.Lines.Add(s);
end;                

result:

Memo1
A / B
A \/ B
A  B
A \ B
A \\ B
A  B

"/" and "" is missing after unescape.

New empty key destroys the json structure

In practice, I also solve the configuration with MCJson. Therefore, due to new situations, unexpected errors also occur. If I add a key to an existing structure (asJson, LoadFromFile) (main key, without value, because it won't have any), it destroys the json, and next time it won't even be loaded.
It's possible that an empty key (category) shouldn't be added this way, but I don't think it should spoil the json structure.

procedure TForm1.Button1Click(Sender: TObject);
const
  fn = 'test.json';
var
  N: TMcJsonItem;
begin
  DeleteFile(fn);

  N := TMcJsonItem.Create;
  try
    N.AsJSON := '{"i": 123, "f": 123.456, "s": "abc", "b": true, "n": null}';

    //N.SaveToFile(fn);
    //N.LoadFromFile(fn);
    //Memo1.Lines.LoadFromFile(fn);

    N.Add('EmptyNewKey'); // <- this destroy json structure

    N.SaveToFile(fn);
    N.LoadFromFile(fn);
    Memo1.Lines.LoadFromFile(fn);

  except
    N.Free;
  end;
end; 

Invalid char at pos 1 (UTF8 BOM support)

Hi!

Unfortunately, many utf8 json files contain a BOM, which McJSON cannot read. I solved this in Lazarus, of course, but I don't know how the code would work under Delphi (UTF16?), so I don't offer a pull request.
So my request/recommendation would be that you also support UTF8 BOM encoded files, at least for reading (I did it for writing as well, but it's not that important). The code itself is not complicated, because if a BOM is detected in LoadFromStream, you only need to position it after the BOM (but the reading moved it anyway).
UTF8 BOM: $EF $BB $BF
See: https://learn.microsoft.com/en-us/globalization/encoding/byte-order-mark
Since you also support Delphi, as far as I know, UTF16 encoding is the basis there, so then UTF16LE/BE BOMs could also be supported.

PrjTestMcJSON - many exception

Hi!

With release mode, the result is: "6 test failed"

With debug mode, I got many exceptions...

Lazarus 2.2.4 x64 official on Windows x64

Test #13 failed in Main branch (trunk rev. #94)

My windows codepage:

** Command line:
CHCP:
Active code page: 852

** Power shell:
Get-WinSystemLocale:
LCID Name DisplayName
1038 hu-HU magyar (magyarországi)

** Wiki:
https://en.wikipedia.org/wiki/Windows-1250
My country is Hungary.

Shorteners self-test result

Hi!

It's not a bugreport, only help for you (with "," separator):

[PASS] Test 01: parse simple object
[PASS] Test 02: parse simple array
[PASS] Test 03: parse simple sub object array
[PASS] Test 04: simple object value change
[PASS] Test 05: Add, Insert, Delete functions
[PASS] Test 06: object is nil
Error: Invalid index: get item by index 3
[PASS] Test 07: getters and setters
[FAIL] Test 08: numbers: scientific notation
Error: Can't convert item "number" with value "-1.23456789E-10" to "double"
[PASS] Test 09: escapes
[PASS] Test 10: invalid JSON
[PASS] Test 11: valid or unusual JSON
[FAIL] Test 12: type transformations
[FAIL] Test 13: Save and Load using files
[PASS] Test 14: constructors
[PASS] Test 15: Copy, Clone, IsEqual, Remove functions
[PASS] Test 16: exceptions
Error: Object reference is nil: get item by key "not"
Error: Object reference is nil: get item by key "not"
Error: Object reference is nil: get item by index 1
Error: Invalid item type: expected "object" got "value"
Error: Can't convert item "string" with value "123a" to "integer"
Error: Can't convert item "null" to "integer"
Error: Error while parsing text: "duplicated key k" at pos "14"
[PASS] Test 17: enumerators
[FAIL] Test 18: example like JsonDataObjects
[PASS] Test 19: At() shortener for array item access
[PASS] Test 20: key paths
[FAIL] Test: Github readme.md content
Error: Error while parsing text: "expected : got 3" at pos "48"

5 tests FAILED

With "." separator:

[PASS] Test 01: parse simple object
[PASS] Test 02: parse simple array
[PASS] Test 03: parse simple sub object array
[PASS] Test 04: simple object value change
[PASS] Test 05: Add, Insert, Delete functions
[PASS] Test 06: object is nil
Error: Invalid index: get item by index 3
[PASS] Test 07: getters and setters
[PASS] Test 08: numbers: scientific notation
[PASS] Test 09: escapes
[PASS] Test 10: invalid JSON
[PASS] Test 11: valid or unusual JSON
[PASS] Test 12: type transformations
[FAIL] Test 13: Save and Load using files
[PASS] Test 14: constructors
[PASS] Test 15: Copy, Clone, IsEqual, Remove functions
[PASS] Test 16: exceptions
Error: Object reference is nil: get item by key "not"
Error: Object reference is nil: get item by key "not"
Error: Object reference is nil: get item by index 1
Error: Invalid item type: expected "object" got "value"
Error: Can't convert item "string" with value "123a" to "integer"
Error: Can't convert item "null" to "integer"
Error: Error while parsing text: "duplicated key k" at pos "14"
[PASS] Test 17: enumerators
[PASS] Test 18: example like JsonDataObjects
[PASS] Test 19: At() shortener for array item access
[PASS] Test 20: key paths
[PASS] Test: Github readme.md content

1 tests FAILED

Lazarus 2.2.4 x64 (official installer) on Win10 x64

Excepted got "/" (Comment support)

In principle, the JSON5 standard:
https://spec.json5.org/#comments
Of course, one could argue about how standard it is, but for example Json.NET supports it: https://www.newtonsoft.com/json
I don't think it's too complicated to implement, you just have to ignore it when reading:

// This is a single line comment.

/* This is a multi-
line comment. */

McJsonEscapeString (overzealous)

Hi!

Very-very thanks for this feature, but it's very overzealous as it also converts all accented characters, which isn't necessarily lucky since json is a human-readable/modifiable format (It is used by translators, for example).

So, I think this code uneccesary:

if ((Integer(c) <  32) or (Integer(c) > 126)) then
   Result := Result + ESCAPE + 'u' + IntToHex(Integer(c), 4)

But if you need this, I suggest more parameters, for example:

function McJsonEscapeString(const aStr: string; const aEscapeNonEnglishChars: boolean = False): string;
...
      begin
        if  (integer(c) < 32) or ((integer(c) > 126) and aEscapeNonEnglishChars) then
          Result := Result + ESCAPE + 'u' + IntToHex(integer(c), 4)
        else
          Result := Result + c;
      end;
...

Check function does not return the wrong position

Hi!

I would like to use the check function, but the problem is that it hides the error position (without debugger).

function TMcJsonItem.Check(const aStr: string; aSpeedUp: Boolean): Boolean;
var
  aItem: TMcJsonItem;
begin
  aItem := TMcJsonItem.Create;
  try
    aItem.fSpeedUp := aSpeedUp;
    aItem.AsJSON   := aStr;
//    Result := True;
    Result := (aItem.AsJSON = trimWS(aStr));
  except
    Result := False;
  end;
  aItem.Free;
end;   

Idea, integer result:

function TMcJsonItem.Check(const aStr: string; aSpeedUp: Boolean): integer // return value is the wrong position...

and then the original function stay, and use it:

function TMcJsonItem.Check(const aStr: string; aSpeedUp: Boolean): booelan;
begin
  if Check(aStr, aSpeedup)<0 then Result:= true else Result:=false;
end;

... or other way, if you have time.

Improve test case

Just looking at the test function Test99 and I note that it performs a save followed by a load, however, this doesn't prove if the load actually worked. An improvement would be to perform a change between the save and load, but not a complete clear. A possible change could be the addition of an extra item before the load, an item that then should not be present in the resulting JSON for the test to pass.

Slow json generation speed

Presumably due to my not quite correct code. I create a json structure this way:

Json.Add(IndexStr);
for x in Tx do
  Json.Path(IndexStr).Add(x.key).AsString := McJsonEscapeString(x.value);   

This about look this way (about 150k line):

 "200": {
    "Key1": "Value1",
    "Key2": "Value2",
    "Key3": "Value3"
  },

 "201": {
    "Key1": "Value1",
    "Key2": "Value2",
    "Key3": "Value3"
  },

I think the low generation speed is because of the path usage, can this be fixed somehow? For example
If I always write to the last one, the library doesn't need to search through the entire structure.

How check path exist by not throw error

How to check nill of specific paht which throw to show error?
such I want to check
N.Path('o.k2')
or
N['o']['k2']
= nil ?

but now if path not exist McJSON show error automatically, but I don't want that.

I just want to be able to check for myself whether it's there or not. Even if there is, I will proceed with one thing. But if you don't have it, don't do another thing, for example.

if N['o']['k2']  = nil then
 // do something
else
 // do something else ;

or

if N['o']['k2'].ItemType  = jitNone then
 // do something
else
 // do something else ;

---

Note : for JsonDataObjects can do this

if N['o']['k2'].Typ = jdtNone then
 // do something
else
 // do something else ;

Please guide me for this
Thank you

Slower shortener operation (but I realized this is normal because of the AddOrSet search)

Hi

I started to standardize the code from the traditional code to a shortener version (which had not yet been rewritten). The code below in the shortened version is approx. 20 times slower. It is possible that this is not how the use of the shortener code was invented (create object), but it makes sense to me and it works: ( "->" just an indicator in the code)

var
 Json: TMcJsonItem = nil;
 JsonDataObject: TMcJsonItem = nil;
 JsonIDObject: TMcJsonItem = nil;
 
begin
...
Json := TMcJsonItem.Create;
...
      JsonDataObject := Json.O[cJsonDataObject]; // Create object... ('JsonData')

      for i := 0 to Self.Count - 1 do
      begin
        IndexStr := IntToStr(Self.Data[i].Index);
        -> JsonIDObject := JsonDataObject.O[IndexStr]; // Create object... this 20 times slower than the next line:
        -> JsonIDObject := JsonDataObject.Add(IndexStr); // Create object... (I know it isn't a correct code, but see ***

        for lt in TLanguageTypes do
          JsonIDObject.Add(lt.KeyName).AsString := Self.Data[i].GetItem(lt);
      end;

What I also don't understand is that earlier you showed a code for creating an empty key (object) #17 (comment):

So the correct code for to create object, this (non shortener variant): ***

->JsonIDObject := JsonDataObject.Add(IndexStr, jitObject);

But no, because the compiler gives an error:

Error: Incompatible type for arg no. 2: Got "TJSONInstanceType", expected "TJItemType"

but the "jitObject" is TJItemType:

TJItemType  = (jitUnset, jitValue, jitObject, jitArray);

Thank you for your patience! I tried to describe everything as accurately as possible.

how to get int64 value ?

how to get int64 value ?

How fix warning compiler about string

Hello,

I gave the variable a JSON string value before adding the value to the JSON object with the AsJSON method, which caused a warning.
TestMcJSON_.zip [This if sample project]
But I get 33 warning messages after compilation.
about :

• W1050 WideChar reduced to byte char in set expressions. Consider using 'CharInSet' function in 'SysUtils' unit
• W1057 Implicit string cast from 'AnsiString' to 'string'
• W1058 Implicit string cast with potential data loss from 'string' to 'AnsiString'

Plese advice me to fix these

Thank you

McJSONEscapeString TAB problem

Hi!

I'v found small error in the new function:

procedure TForm1.Button1Click(Sender: TObject);
const
  cSample = 'Fisch'+#9;
var
  s: string;
begin
  s := McJSONEscapeString(cSample, jetNormal);
  Memo1.Lines.Add('MC: "'+s+'"');
  s := McJSONUnEscapeString(s);
  if s <> cSample then Memo1.Lines.Add('MC error!')
  else Memo1.Lines.Add('MC OK!');

  s := StringToJSONString(cSample);
  Memo1.Lines.Add('FP: "'+s+'"');
  s := JSONStringToString(s);
  if s <> cSample then Memo1.Lines.Add('FP error!')
  else Memo1.Lines.Add('FP OK!');
end;      

Result:

Memo1
MC: "Fisch\	"
MC error!
FP: "Fisch\t"
FP OK!