ploubser/json-grep

Command line tool and API for parsing JSON documents

json-grep's Issues

Valid JSON, can't figure out how to jgrep

I've tried a lot, but cannot figure out how to "show the tags field for those matching state=running". If anyone has any suggestions, I would love to hear them. I could not find any other way to contact someone, so am creating an issue. FWIW, the following passes arc90's JSON linter and is the output from "puppet node_aws list --render-as json"

[
 {
    "i-09387672": {
      "id": "i-09387672",
      "tags": "{\"Name\"=>nil, \"node-type\"=>\"bastion\"}",
      "dns_name": "ec2-23-21-146-98.compute-1.amazonaws.com",
      "state": "running",
      "created_at": "Sat Aug 18 18:31:48 UTC 2012",
      "keyname": "bld_admin"
    },
    "i-c8d3d5b2": {
      "id": "i-c8d3d5b2",
      "tags": "{\"Name\"=>\"graph-inserter-1\"}",
      "dns_name": null,
      "state": "stopped",
      "created_at": "Thu Sep 20 05:36:41 UTC 2012",
      "keyname": "jlekey"
    }
  }
]

remove json dependency for ruby > 1.8

Ruby comes with json now. Do we still need the json dependency? Can we replace with s.add_runtime_dependency 'json' if RUBY_VERSION =~ /^1\.8/. This is preventing me from bundling this gem into a package since it requires compilation

Ruby 2.7 support

When running jgrep under Ruby 2.7, the following warning can be seen:

/home/ekohl/.gem/ruby/gems/jgrep-1.5.2/lib/parser/parser.rb:84: warning: deprecated Object#=~ is called on FalseClass; it always returns nil

There may be more

Support matching on null

We should be able to match docs like :

[
 {"foo":null}
]

Support YAML (e.g. a -y/--yaml boolean flag)

This may exceed the purview of this utility (after all it's jgrep, not ygrep), but it would be handy if you could parse both YAML and JSON files. If it's something you'd be willing to consider, I'll give it a whirl and submit a pull request. Thanks

should always using UTF when reading json docs

When piping in json files with a command such as:
cat data.json | jgrep -n -s id

I sometimes get errors such as:
100017401
10005352
Error - "\xE2" on US-ASCII

It seems like jgrep is reading files with the wrong encoding set.

Feature request: regex or wildcard matching on fields

I have json output that looks like this:

{
  "node1": {
    "foo": "bar"
  },
  "node2": {
    // blah
  },
  "node3": {
    "foo": "baz"
  }
}

I would love to be able to match on both node1.foo and node3.foo, preferably with the option to match certain values of that path as well.

Support for complex/nested start expressions ?

Hi, not sure how to put this correctly, but with data (and the jgrep invocation) here , wonder if its possible to simplify the invocation ?

Where did the -n option go?

Hi,
It seems that -n option introduced with 1.3.1 was still there in 1.3.2 branch, but is lost in the latest 1.3.3. Is there a reason for that?

BR,
Dawid

Extract specific matching document from array of matching documents

It would be handy if give JSON like this:

{
  "foo": {
    "bar": "baz"
  }
}

You could extract the subdocument:

{
  "bar": "baz"
}

jgrep currently will only return an array of matching items, e.g.:

$ jgrep foo.json -s foo
[
  {
    "bar": "baz"
  }
]

nagative matching

Would it be possible to support both these:

jgrep "! foo=1" 

and

jgrep "foo!=1"

as the same thing? atm the first works fine

Sub documents thats not hashes

Given a json input:

  [
    {
      "name": "radiant_load_balancer",
      "default_attributes": {
      },
      "_rev": "1-72a453357bb56d3978a1c478ab1f8d17",
      "json_class": "Chef::Role",
      "env_run_lists": {
      },
      "run_list": [
        "recipe[haproxy::app_lb]"
      ],
      "description": "radiant load balancer",
      "chef_type": "role",
      "override_attributes": {
        "haproxy": {
          "app_server_role": "radiant"
        }
      }
    }
  ]

$ cat t|jgrep -s run_list
Error - undefined method `keys' for "recipe[haproxy::app_lb]":String

Missing dependency for "json" gem

Hey! Just found this through the mention in the Devops Digest.

I assume most wouldn't notice this, as presumably it's already installed, but I recently wiped fresh :)

Still playing around but looks cool! Thanks

utility to validate an expression is valid

I'd like a way to check a expression is valid, so something like

def validate_expression(expression)
   Parser.new(expression)
   nil
rescue
   $!.message
end

This gives me a way to softly pre-check an expression a user gives, of course I can do JGrep::Parser.new myself but having something on JGrep.validate_expression seems to communicate the publicness of that method better - I assume Parser is private

STDIN detection logic prevents programmatic execution

I'm attempting to use jgrep as a component in a product build. Unfortunately, the build always fails when run from Jenkins because of the STDIN detection logic around line 69 (the user running the does not have a TTY allocated, so jgrep assumes it should try to read from STDIN, even though I'm explicitly passing -i).

Something like this would fix the issue:

if options[:file]
    json = File.read(options[:file])
elsif ! STDIN.tty?
    json = STDIN.read
else
    raise "No json input specified"
end

Short hand to check presence of a key

Should be able to figure out if some data item is present:

Now I do:

jgrep "foo!=bar"

when I know there is never something called bar, this has the effect of finding ones where foo is present, would be nice to just be able to do:

jgrep foo

which should imply presense

Support for regex

It would be nice for expressions to support the =~ operator

Support filtering parsed objects

Today JGrep.jgrep parses the input, but in voxpupuli/facterdb#282 I want to further refine results.

As a start I opened #53 so I could understand the API.

One option is to inspect the first argument and only parse it if it's a string, but I'm generally not a fan of those kind of APIs. Factoring it out to a smaller method is probably better.

Before I start I'd like to hear the opinions on this.

example from http://jgrep.org/#howto fails

See comments starting with '==>':

In-document comparison

If a document contains an array, the '[' and ']' symbols can be used to define a comparison where statements are checked for truth on a per element basis which will then be combined. Consider:

[foo.bar1=1 and foo.bar2=2]

on:

[
  {
    "foo":  [
      {
        "bar1":1
      },
      {
        "bar2":2
      }
    ],
  },
  {
    "foo":  [
      {
        "bar1":0
      },
      {
        "bar2":0
      }
    ]
  }
]

==> invalid JSON!

will return:

[
  {
    "foo": [
      {
        "bar1": 1
      },
      {
        "bar2": 2
      }
    ]
  }
]

==> query applied to corrected JSON (',' after "foo" elem in first obj deleted) returns nothing; but querying with
"foo.bar1=1 and foo.bar2=2"
works.
jgrep-1.3.1 (installed by gem), ruby 1.8.7 (2010-08-16 patchlevel 302) [x86_64-linux]

Should take a starting position as argument

Many documents have this format:

  {
    "total": 495,
    "type": "FeatureCollection",
    "features": [
         # we want to grep the data here
    ]
  }

At the moment I have to do:

cat foo.json|jgrep -s features -f|jgrep ....

Would be good if this could just become:

cat foo.json|jgrep --start features "this and that"

should not exit on json parse errors

when someone want to use JGrep.jgrep as a library it shouldnt exit

https://github.com/ploubser/JSON-Grep/blob/master/lib/jgrep.rb#L69-72

unused methods

while making it all pretty I noticed the 2 methods JGrep.hash_to_array and JGrep.array_to_hash they seem completely unused, they also seem a bit cringe worthy since all they do is print stuff to STDOUT and then return the string returned from puts. They also exit.

Any idea whats up with these @ploubser something we can just yank?