Comments (4)
Hi, thanks for participating.
As you said, it is not possible to parse nested items in objects. JSON Pointer is too simple a language for that. We can't just start supporting "-" as a wildcard for object keys because what if a key in an object is "-"?
Parser has to always read everything to get to the object keys. The main use case for the parser is to sequentially read all the items in a specified subtree. That's why it stores every item in memory. It's usually what a programmer wants.
Once I have list of categories, I can parse the objects in second round, using pointers for each category.
This will only complicate things for you. If you're already there, decode it and use it. The second round will do exactly the same work as the first. If you expect it to be somehow more efficient or faster the second round, keep in mind that using json pointers will not affect parsing time in any way, only memory usage. The parser always has to read everything to get to the desired key. No direct access as in hashmaps.
PassThruDecoder
is there for such situations. If a single item is too big, do the top-level parsing using it and then parse the produced string via ExtJsonDecoder
as shown in README.
If you're really low on memory try #36. The prototype should work. It should be installable via
composer require halaxa/json-machine:dev-recursive
It might nudge me to finish it :)
If any of this is of no use to you, try for example salsify/jsonstreamingparser.
Does this answer your questions?
from json-machine.
We can't just start supporting "-" as a wildcard for object keys because what if a key in an object is "-"?
Yep, true. But I am sure this problem would be solvable with some kind of escaping. And maybe use something else than hyphen...
Parser has to always read everything to get to the object keys.
I perfectly understand this. I just though that the value does not need to be hold in memory. If the /results/fruits
has few thousands of records, few MB each, than even use of PassThruDecoder
means holding some GBs in memory. However if the parser would not hold that text in $jsonBuffer
variable, just iterate over keys and throw those data away, I would be able to iterate over keys with use of very few memory only, would not I? And once I have the keys, I can construct pointers like /results/fruits
, /results/vegetable
, etc. and iterate over the file once again and hold only single record in the memory each time. That was just my dumb idea...
The #36 looks promising and much more elegant solution of course. I will try it.
Thanks for pointing to salsify parser, I will also have a look to it.
from json-machine.
Yep, true. But I am sure this problem would be solvable with some kind of escaping. And maybe use something else than hyphen...
The only option via escaping would be adding another escape sequence, for example ~2
which could mean asterisk (maybe .*
regex equivalent), and thus stop being compatible with the json pointer spec. I'm not sure if I incline to this.
https://datatracker.ietf.org/doc/html/rfc6901#section-3
from json-machine.
@halaxa, hi! Thanks for the awesome lib! I really hope it will become a fully-comprehensive solution one day, because as of now it seems to be at least the best 🙏
To the point, IMHO: if the spec doesn't mention that you must not introduce any other escape sequences and everything besides existing defined 2 sequences must be treated literally, I believe you would technically still be compatible with their spec; you'd simply supersede it.
from json-machine.
Related Issues (20)
- Why only red is output HOT 4
- Iterating items from specific position HOT 1
- Document-dependent path evaluation HOT 11
- Problem With `getCurrentJsonPointer()` HOT 4
- Non-Intersecting Pointers Considered Intersecting HOT 2
- Unicode Escape Sequences Support HOT 4
- Wrong key when combining list and scalar value pointers
- GeoJSON properties only
- Thank you for this awesome library. HOT 3
- Deprecated warning HOT 2
- Getting un catchable errors on non JSON files HOT 9
- Option or method for preserving the precision of numeric literals HOT 6
- CLI script for streaming HOT 8
- passing headers HOT 1
- UnexpectedEndSyntaxErrorException HOT 19
- taking time while parsing HOT 8
- Basic Usage Using PHP and Brew HOT 7
- File parsing takes too long. HOT 6
- Unexpected symbol with Items::fromStream HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from json-machine.