maxakawizard/jsoncollectionparser

Streaming parser for JSON collections

Max Grigorian

Passionate developer, paranoid in self-improvement.

More than 15 years of programming experience.

Symfony framework fan and SymfonyCon attendee.

Skills

• PHP 5/7/8, Symfony 2-7, Silex, Doctrine2, OOP, DI, Patterns
• HTML 4/5, CSS 2/3, LESS, SASS
• JavaScript, Vue 2, NUXT, Angular, jQuery, JSON, Google Maps API
• MODx, Joomla, Opencart, Wordpress
• MySQL, MongoDB, ValentinaDB
• Memcached
• Sphinx
• Git, Subversion
• Jenkins
• Grunt, Webpack, Phing, Capistrano
• PHPUnit, Codeception, Karma
• Composer, npm, yarn, bower
• Apache, Nginx
• Vagrant, Docker
• JIRA, Mantis, Redmine
• Mac OS X, Linux, Unix, Windows

Github Stats

show

Contacts:

jsoncollectionparser's Issues

Support for parsing progress

JsonStreamingParser is able to report progress of file parsing. It will be really handful to have this progress, because processing of large file can take a lot of time.

The only what you need is to add function "filePosition" to your Listener and add propagate progress values to callback (or add another callback).

Support for document streams, objects, objects in subarrays

Currently the \JsonCollectionParser\Listener supports only:

• objects inside array: [ { } , { } , { } , ... ]

It seems reasonable to add support for more forms of input data, at least object-oriented:

• object: { }
• stream of objects: { } { } { } ...
• objects in subarrays: [ [ { } , { } , ... ] ]
• and combination of above, e.g: [ { } , { } ] { }

Except for the first case, the data is actually a concatenation of json documents, a frequent case when working with stream data.

The underlying library unfortunately didn't support for multiple documents, hence this PR:
salsify/jsonstreamingparser#60

After and if it's accepted, please check out and evaluate this branch: https://github.com/OnkelTem/JsonCollectionParser/tree/documents-stream-support
It implements the mentioned cases.

Input data format descend

Hello,

Is it possible to descend in JSON structure? Suppose I have data format like:

         {
            "objects": [
                {
                    "uid": 1,
                    "name": "Name 1"
                },
                {
                    "uid": 2,
                    "name": "Name 2"
                }
            ]
        }

So I can walk through the objects items
Thank you

Support for list of objects {}, {},...

I have some json data that is presented in the format of object lists e.g.:
{object},
{object},

So in order to process htme i have to sed add [ at beginning and end ]

It seems very trivial, so it would be great if the parse could support files that are not wrapped in an array but otherwise perfectly represent the format.

Thanks

500mb json file timeout error

I have a 500mb json file, which is also available here (https://archive.scryfall.com/json/scryfall-all-cards.json) When I'm trying to process this file I need to raise max execution time to something like 1000 seconds as otherwise I receive timeout in the salsify parser.

Is there any way to solve this? I mean it's great, that the files is not loaded into memory, but many servers don't allow for raising execution time to insane values neither?

The error hits at Parser.php Line 152, 197, or 201.

Thanks

PHP8 support

Hi 👋

I've noticed that support for PHP8 is implemented, but not released even though CHANGELOG mentions v1.8.0.

Just wanted to see if there is a plan to release a new version with PHP8 support soon?

Thanks for your work on this great package! Cheers!

please consider support psr/http-message 2

my project runs psr/http-message 2.0, but this project require ~1.0, can you consider compatible it ?