Comments (11)
I looked into this, I think it's Jackson actually consuming the rest of the stream.
I checked that the StringReader had been consumed by calling (.read string-reader)
and verifying it was still -1 after parsing, even with trailing junk.
I also double-checked with another project that uses Jackson, and it ignores trailing junk also.
Is the trailing data causing problems for you?
from cheshire.
I looked into this, I think it's Jackson actually consuming the rest of the stream.
I checked that the StringReader had been consumed by calling
(.read string-reader)
and verifying it was still -1 after parsing, even with trailing junk.
Indeed. This solution was a guess, and it wasn't the right approach. I looked into it more closely and I'm pretty sure the answer is that you need to check that (.nextToken jp)
is nil
at the end of parse/parse
. Here's a REPL session to show what I mean.
user=> (def jp (.createJsonParser (JsonFactory.) (StringReader. "{\"foo\": 123}")))
#'user/jp
user=> (.nextToken jp)
#<JsonToken START_OBJECT>
user=> (.nextToken jp)
#<JsonToken FIELD_NAME>
user=> (.nextToken jp)
#<JsonToken VALUE_NUMBER_INT>
user=> (.nextToken jp)
#<JsonToken END_OBJECT>
user=> (.nextToken jp)
nil
user=> (def jp (.createJsonParser (JsonFactory.) (StringReader. "{\"foo\": 123}null")))
#'user/jp
user=> (.nextToken jp)
#<JsonToken START_OBJECT>
user=> (.nextToken jp)
#<JsonToken FIELD_NAME>
user=> (.nextToken jp)
#<JsonToken VALUE_NUMBER_INT>
user=> (.nextToken jp)
#<JsonToken END_OBJECT>
user=> (.nextToken jp)
#<JsonToken VALUE_NULL>
user=> (def jp (.createJsonParser (JsonFactory.) (StringReader. "{\"foo\": 123}junk")))
#'user/jp
user=> (.nextToken jp)
#<JsonToken START_OBJECT>
user=> (.nextToken jp)
#<JsonToken FIELD_NAME>
user=> (.nextToken jp)
#<JsonToken VALUE_NUMBER_INT>
user=> (.nextToken jp)
#<JsonToken END_OBJECT>
user=> (.nextToken jp)
JsonParseException Unrecognized token 'junk': was expecting ('true', 'false' or 'null')
at [Source: java.io.StringReader@58a606e3; line: 1, column: 33] com.fasterxml.jackson.core.JsonParser._constructError (JsonParser.java:1524)
I also double-checked with another project that uses Jackson, and it ignores trailing junk also.
Yeah, as I mentioned I saw that data.json
and clj-json
have this same behavior as well. It's easy for this to happen when building a batch parsing function from a streaming parser.
Is the trailing data causing problems for you?
Yes, we parse a very large amount of JSON data and we see all manner of corruption.
from cheshire.
Any updates?
from cheshire.
@cespare I could add a wrapper around the regular parsing functions that would verify the nil
on .nextToken
after parsing has completed, then it could optionally be used (to not incur the overhead for everyone, just people who need it)
Does that sound like it would work for you?
from cheshire.
Sure, that would let us fix this problem.
This should not be a wrapper, it should be the default behavior. It's misleading that parse-string
only looks for a valid JSON prefix in the given string. The extra cost is very small compared with doing JSON parsing in the first place, and well worth avoiding the confusion.
If the current behavior is somehow useful, then perhaps the function should be named parse-json-prefix
. More likely to be useful would be an API for reading successive JSON values from a reader (the current parse-string
cannot do that because the StringReader
is just thrown away after the first value is read).
from cheshire.
Any update on this? Would love to see a fix.
from cheshire.
Also would love to see a fix
from cheshire.
Me too! This was very surprising!
from cheshire.
+1 Just bit me also.
from cheshire.
Closed by #122
from cheshire.
I still think it's misleading that the default parse-string
function accepts a string that's invalid JSON, but at least there's a workaround for people after they're bitten by this :|
from cheshire.
Related Issues (20)
- Always update to latest version of Jackson, when available
- Any way to not escape the quotes HOT 2
- Suggestion: prove equivalence with clojure.data.json HOT 6
- Add support for capturing source position/locations when parsing HOT 2
- Allow key-fn like hook for value HOT 2
- Converting nested fields from java objects to clojure types doesnot work
- Custom conversion when parsing?
- `parsed-smile-seq` actually parses JSON
- Run tests against JSONTestSuite
- Resulting Array from parse not processed lazily HOT 1
- :key-fn no longer works in generate-string? HOT 1
- Using the Ubuntu Cheshire package with Clojure HOT 1
- lein unable to find cheshire on classpath HOT 1
- Security vulnerability in Jackson dependency HOT 4
- Can't convert object of type org.postgresql.util.PGobject HOT 1
- support jackson's fast number parsing/writing features HOT 1
- Update test.generative HOT 2
- Possible to support parsing copy-paste JSON? HOT 1
- Test failures with Java 21
- Deterministic parse and generate HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cheshire.