jqassistant/jqa-json-plugin

JSON Plugin for jQAssistant

jQAssistant Aggregator Repository

jQAssistant has been splitted into multiple repositories to allow more flexible releases.

Note	Want to raise an issue? If possible try to go to the affected repository. If not, please raise it in our Idea Hub.

Want to get started? Here are some links to help you:

• Get Started

• 101 Tutorials

• The jQAssistant Manual

If you have questions related to jQAssistant please get in touch with us via the jQAssistant forum or Stackoverflow.

The table below lists the repositories jQAssistant is made of.

Table 1. List of jQAssistant repositories

BOM	https://github.com/jqassistant/jqa-bom
Commandline Distribution	https://github.com/jqassistant/jqa-commandline-tool
Core Framework	https://github.com/jqassistant/jqa-core-framework
Java Plugin	https://github.com/jqassistant/jqa-java-plugin
JSON Plugin	https://github.com/jqassistant/jqa-json-plugin
Maven 3 Plugin	https://github.com/jqassistant/jqa-maven3-plugin
JUnit Plugin	https://github.com/jqassistant/jqa-junit-plugin
Neo4j Backend	https://github.com/jqassistant/jqa-neo4j-backend
Plugin for Apache Maven	https://github.com/jqassistant/jqa-maven-plugin
XML Plugin	https://github.com/jqassistant/jqa-xml-plugin
YAML 2 Plugin	https://github.com/jqassistant/jqa-yaml2-plugin

Tip

For convenience reasons the repositories are aggregated as sub-modules in this Git repository. You can clone it using git clone --recursive https://github.com/jQAssistant/jQAssistant.git. Once checked out you might need to run git submodule update --remote to update to the latest commits of the sub-modules.

jqa-json-plugin's Issues

Migrate the Plugin Documentation of the JSON Plugin to the new Reference Manual

Task Description

Migrate the existing documentation of the JSON Plugin to the new reference manul.

Definition of Done for the Implementers

• The reference manual contains the same information as the old user manual
• The section on the JSON Plugin in the old user manual points to the section for the plugin in the reference manual

Cope with null scalar values in arrays

Currently the JSON plugin is not able to cope with JSON documents like [null, 'a']. Parsing such documents results in an exception.

Plugin should be compiled for Java 7

Currently the compiler setting specifies Java 8, for compatibility reasons it should be Java 7 as the plugin is included in the default distributions.

The JSON parser should be able to handle encoding problems

The JSON parser is currently not able to handle all examples with character encoding problems provided by the JSON parsing test suite to handle. But the parser should be able to do this.

JSON Parser does not handle empty arrays correctly

Bug Description

The plugin fails to load some JSONs completely with a very common error message:

JSON file 'xxx.json' seems to be invalid and will be marked as invalid. Result graph might be incorrect.

Expected Behaviour

At first I would like to see a more contextual error message, at least an exception should give some hint about the effective problem 😉.

How can we reproduce the bug?

Craft a JSON (and parse it) with to many empty arrays like:

{
  a1: [],
  a2: [],
  ...
  a1000: []
}

As soon as you have more than 999 empty arrays you will get the error.

Investigation

Digging deeper into the problem I first introduced the output of the exception which led to the problem in

jqa-json-plugin/src/main/java/com/buschmais/jqassistant/plugin/json/impl/scanner/JSONFileScannerPlugin.java

Line 102 in d98b83c

LOGGER.warn("JSON file '{}' seems to be invalid and will be marked as invalid. Result graph might be incorrect.", path);

It turned out it is a home grown exception: Maximum nesting level reached. JSON is to deep nested. as defined here

jqa-json-plugin/src/main/java/com/buschmais/jqassistant/plugin/json/impl/parsing/NestingLevelCounter.java

Line 28 in d98b83c

throw new IllegalStateException("Maximum nesting level reached. JSON is to deep nested.");

Looking into my JSON it was large (~25000 lines) but was not very deeply nested in some place (hardly more than a depth of 15).

Adding some debug output revealed the error after some time. While the start of an Array increased the nesting level:

jqa-json-plugin/src/main/java/com/buschmais/jqassistant/plugin/json/impl/parsing/JSONNestingListener.java

Lines 9 to 10 in d98b83c

	@Override
	public void enterArray(JSONParser.ArrayContext ctx) {

The nesting level is only decreased again if the Array had some content and the last element was parsed:

jqa-json-plugin/src/main/java/com/buschmais/jqassistant/plugin/json/impl/parsing/JSONNestingListener.java

Lines 14 to 16 in d98b83c

	@Override
	public void exitArrayElements(JSONParser.ArrayElementsContext ctx) {
	nestingCounter.leave();

As far as I can see the wrong method from the base class is overridden here, it should be exitArray (symmetric to enterArray above, not exitArrayElements).
Changing this implementation fixed the problem for me.

Definition of Done for the Implementers

Remove this hint: these checkboxes can be checked like this: [x]

• We got a final feedback from the reporting user (if applicable)
• Unittests have been written (if applicable)
• Integration tests have been written (if applicable)
• Test coverage is the same or even better then before (if applicable)
• Documentation has been written (if applicable)
• Added a note on the new feature to the release notes (if applicable)

Use Github Actions to provide a simple CI/CD Buildchain

As core developer of jQAssistant, I want this project to be build on various plattforms with different JDK versions on each commit, so that I know if the change done my the commit does not break the build and the software is fully functional.

The following acceptance criterias must be met:

• mvn -DskipTests -Djqassistant.skip clean install must pass
• mvn -Djqassistant.skip clean install must pass
• mvn -P IT clean install must pass
• mvn -DskipTests -Djqassistant.skip=false clean install must pass

Mention the JSON Parsing Test Suite in the documentation

We use the files provided by "JSON Parsing Test Suite" for testing our parser. The should mention them in the documentation. Also the article behind the test suite.

The JSON plugin should support the exclusion of files

As other plugins like the XML plugin, the JSON plugin should allow the user to exlude specific files from the scan process.

Unify the Asciidoctor markup

Go over all documents of the documentation and unify the Asciidoctor markup.

Internal exception thrown while scanning an invalid JSON document

User reported bug: A faulty JSON document causes an internal exception and breaks the scan phase of jQAssistant.

[ERROR] Failed to execute goal com.buschmais.jqassistant:jqassistant-maven-plugin:1.3.0-SNAPSHOT:scan (default) on project parent: Execution default of goal com.buschmais.jqassistant:jqassistant-maven-plugin:1.3.0-SNAPSHOT:scan failed: Unexpected problem encountered while scanning: item='MavenProject: com.buschmais.jqassistant.core:parent:1.3.0-SNAPSHOT @ /Users/hwellmann/git/github/jqa-core-framework/pom.xml', path='/Users/hwellmann/git/github/jqa-core-framework/pom.xml', scope='PROJECT', pipeline='[com.buschmais.jqassistant.plugin.maven3.impl.scanner.MavenProjectScannerPlugin@30b982d9]'. Please report this error including the full stacktrace (continueOnError=false). Unexpected problem encountered while scanning: item='/Users/hwellmann/git/github/jqa-core-framework', path='/Users/hwellmann/git/github/jqa-core-framework', scope='NONE', pipeline='[com.buschmais.jqassistant.plugin.common.impl.scanner.DirectoryScannerPlugin@57ca7d58]'. Please report this error including the full stacktrace (continueOnError=false). Unexpected problem encountered while scanning: item='/Users/hwellmann/git/github/jqa-core-framework/rule/src/main/resources/template.json', path='/rule/src/main/resources/template.json', scope='NONE', pipeline='[com.buschmais.jqassistant.plugin.common.impl.scanner.FileResourceScannerPlugin@7025e20e, com.buschmais.jqassistant.plugin.json.impl.scanner.JSONFileScannerPlugin@19d5d16b]'. Please report this error including the full stacktrace (continueOnError=false). Failed to parse the file /Users/hwellmann/git/github/jqa-core-framework/rule/src/main/resources/template.json. Failed to parse /rule/src/main/resources/template.json at line 1 due to token recognition error at: 'x': LexerNoViableAltException -> [Help 1]

Remove Antlr from JSON plugin

Reenable the analysis of the JSON plugin

Since the release of jQAssistant 1.3.0 - which contains now also the JSON plugin - the next version of this plugin is scanned by this release. Currently the scan of this plugin fails always as jQAssistant also scans out test JSON documents and fails because of a bug in version 1.3 of the JSON plugin.

Therefore we disable the jQAssistant plugin. But it must be enabled again if we are able to exclude our test documents from the scan. This requires that issue #6 has been solved.