konveyor/analyzer-lsp

Add-on that is focused on providing analysis based on the Language Server Protocol.

Analyzer Rule Engine

In this project, we are writing a rule engine that can use pluggable providers for rules to make a consistent way to execute rules for Konveyor.

One of the primary drivers for this repository is adding providers for specific languages using the Language Server Protocol. Today these providers are in tree, but we will be moving them out in the future.

Quick Demo

If you would like to run a quick demo we have a Dockerfile that has all the dependencies.

To run this demo build the containers:

podman build -f Dockerfile -t quay.io/konveyor/analyzer-lsp
podman build -f demo.Dockerfile -t test-analyzer-engine

This will build the engine, and include the current set of rules and examples in the container to be used.

To run the rules (rule-example.yaml) against the examples, and save the output to the demo-output.yaml file:

podman run -v $(pwd)/demo-output.yaml:/analyzer-lsp/output.yaml:Z test-analyzer-engine

Running from source code

To run the engine from source code, you need to:

• Configure providers. By default, providers are configured in provider_settings.json. See Providers for instructions on configuring providers.
• Configure rules. By default, rules are present in rules_example.yaml. See Rules for details on rule format.

Once the providers are configured, you can run:

go run cmd/analyzer/main.go

CLI Options:

Flags:
      --analysis-mode string        select one of full or source-only to tell the providers what to analyize. This can be given on a per provider setting, but this flag will override
      --context-lines int           When violation occurs, A part of source code is added to the output, So this flag configures the number of source code lines to be printed to the output. (default 10)
      --dep-label-selector string   an expression to select dependencies based on labels. This will filter out the violations from these dependencies as well these dependencies when matching dependency conditions
      --enable-jaeger               enable tracer exports to jaeger endpoint (default true)
      --error-on-violation          exit with 3 if any violation are found will also print violations to console
  -h, --help                        help for analyze
      --jaeger-endpoint string      jaeger endpoint to collect tracing data (default "http://localhost:14268/api/traces")
      --label-selector string       an expression to select rules based on labels
      --limit-code-snips int        limit the number code snippets that are retrieved for a file while evaluating a rule, 0 means no limit (default 20)
      --limit-incidents int         Set this to the limit incidents that a given rule can give, zero means no limit (default 1500)
      --no-dependency-rules         Disable dependency analysis rules
      --output-file string          filepath to to store rule violations (default "output.yaml")
      --provider-settings string    path to the provider settings (default "provider_settings.json")
      --rules stringArray           filename or directory containing rule files (default [rule-example.yaml])
      --verbose int                 level for logging output (default 9)

• See label selector for more info on --label-selector option.

Code Base Starting Point

Using the LSP/Protocal from Golang https://github.com/golang/tools/tree/master/gopls/internal/lsp/protocol and stripping out anything related to serving, proxy or anything. Just keeping the types for communication

Using JSONRPC2 from google.org/x/tools/internal. Copied and removed anything to do with serving.

Code of Conduct

Refer to Konveyor's Code of Conduct page

analyzer-lsp's Issues

Rules v1beta1

Publish first version of rule format with documentation.

Figure out base image for just the engine

Once we determine how external providers can be added, we will want to figure out all of this.

Should error on duplicate rule ids.

We probably should have some validation on rule ids during rule parsing

Add comments to non-intuitive rules

Until we decide on the final schema for the rules and generate docs, would it be possible to add a comment about the rule if its not so intuitive. Thanks in advance :)

Prototype integrating into Konveyor

This is to track work on the first iteration of integration work for Analyzers <-> Konveyor. Mainly things I am looking at is passing stuff coming from the Hub as an input to analyzers and pushing back results as a JSON back into the Hub.

POC Dependency analysis

Document provider settings

We need to document provider configuration settings.

Handle dependency analysis

Analyzers need an ability to perform dependency analysis.

Establish feature parity with WindUp for Java Analysis

This is intended as a tracking issue to follow work on-going for establishing feature parity WindUp for the Java Analysis.

Assumption is we will use the rules & tests WindUp provides from https://github.com/windup/windup-rulesets/tree/master/rules/rules-reviewed to help establish feature parity.

Break Rule `message` into title and an actual message that supports rich text

We need to modify message field to have a title in addition to the actual message. This is needed in the report.

Return schemas for Extras for a Violation and Incident

We have ability to return schemas for template context info created by providers. We need something similar for Violations that captures extra information put in there so that end users can parse the extras correctly

Update README

We need to get README to a point where a user / contributor can run an analyzer independently by following instructions.

Actually Implement the Java References Evaluation

Get all the files in a deployable place, in the repo

Add the ability for other providers to use pipelines (java/golang etc).

Output dependencies as a linked list

Add string comparison condition to builtin provider

It will help extract text from the document and compare it with a value/regex, returning a violation on match.

Add unit test and output verification gh actions for the demo

Inter-rule pipelining

We addressed inter-condition pipelining in Demo A. This issue captures the need for pipelining within rules.

Document Violations

Running new Dockerfile results in an error in entrypoint script

Error :

line 0: missing '['

POC Windup rules

Let's start thinking about how we want to get windup rules in analyzers

Discuss intra-conditions chaining by default inside of a or/and block

@pranavgaikwad @fabianvf Copying this issue from PR review. We merged w/o fixing this but want to make sure we don't forget it and have the conversation.

Pipelining in Rules

We need to enable and support pipelining in the rule

This will track all the work left outstanding:

• Initial PR: #20
• #22
• #23
• #24

Decide on templating language and/or implementation

Today we do not have a defined approach to how to pass templated context values into condition information fields.

Right now, this is the responsibility of the provider, but when moving to external providers, I believe that this needs to become the responsibility of the engine.

I think that for now, we can test out the functionality we do have and this will have to be part of the external provider work.

CLI Options

Add support for following CLI options:

Option	Description	Tracking Issue
--rules	Input one or more Rulesets / Rules	Completed #128
--output-file	Output file location	Completed
--label-selector	Label selector logic to include/exclude	#130
--no-dep	Skip dependency analysis	#135
--exlude-deps	Don't analyze dependencies	#136
--limit-incidents	Limt the number of incidents per rule	#153
--limit-cond-snip	Limt the number of incidents per rule that have code snippet found	#154

Support yaml rules

Currently rules can only be written in json but yaml is a superset of json that is more fun to write and read so we should support that too

Adding `Not` keyword when parsing rules

We need to make it so that folks can make sure that something DOES NOT exist.

Consider adding a category field for violations

There is value in classifying violations based on factors such as severity, urgency etc. In windup, there is a separate field category to define those characteristics about a violation. We should consider having something similar in analyzers.

We need to figure out what File URI / URL is supposed to based on

We need to determine the root here and standardize the output

rework the all go files rule in demo testing

Everytime a go source file is added or removed it causes the demo test to fail. We should update that to be less specific so that it doesn't need to be constantly updated

Reporting errors on partial matches

We need to report errors occurred when creation of violations when a rule partially matches or otherwise. Windup has an Optional category for violations that address this.

Re-design the output, such that we get all the rules that were executed

Track source as fileuri when reporting dependencies

Currently we don't point to where the dependencies are coming from, which could be a useful piece of information. We should track and report that. We should also change the Location field in the dependency.Dep to match Location in the rest of the code base (ie it should point somewhere in the codebase) and find a different term to report what it currently gathers for java (or add it to an extras field for arbitrary metadata)

We need to determine how to do out of tree providers

This is a large amount of work, and something that I think is important to get done ASAP.

Add Location information for dependencies

In the future, it might be nice to have a location from the dependency file where a dependency can be changed.

Clean up running LSP, when we exit

Unit testing for rule parsing

We nee do to make sure that we don't break this.

Output tags in violations

Generate a violation for application tags

Java provider needs to support IN option that contains file paths

Adding Tracing in engine and providers

One of the concerns could be the performance of a given provider. We will want the ability to quickly diagnose if the time taken is in the engine or a given provider.

We will also want to see the traces from our providers to see if anything is taking longer than it should.

Consolidating Categories

Current State

Today there are 4 Categories in the windup project.

1. Potential
2. Mandatory
3. Information
4. optional

The potential is supposed to mean that the Rule ran successfully, and there may or may not be an issue. We can't tell.

Mandatory is that the Rule ran correctly and the Rule states this must be fixed

Information is that the Rule ran correctly, and the Rule states that this is just for information, which could mean there is a problem in the future but not something you must fix.

Optional is that a rule ran unsuccessfully, and it might have a violation and may not.

Proposed State

Consolidating these and making it easier for both end users to understand what the category is supposed to mean and what a rule writer should set is the goal of this issue.

For the Rule's categories, I am proposing two mutually exclusive buckets.

1. Mandatory will be when the Rule states that this must be fixed based on the ruleset. This will only be set for rules that have run successfully
2. Potential will be when the Rule states this is potentially an issue. This will only be set for rules that have run successfully

We need to create a new bucket for partial matches/errors. I would prefer to call this something like Errors, which will be different than the violations that use the categories above. In this world, we would have two new error categories:

1. Partial Match -> This Rule ran semi-successfully, and some conditions matched, but there were failures in others. This should include the error message and other stuff
2. Failure -> this Rule did not run successfully in any way.

We also need a third bucket of rules for rules that ran successfully but did not have any violations. I believe this should just be something like PassedRules or something.

We should discuss the names here so that once we implement it, we can point back to this.

Adding in folks that this will probably have an impact on:
@rromannissen @jortel @ibolton336 @pranavgaikwad @fabianvf

We need to choose a logger, and determine default logging behavior

A logger that can be enabled to run with user-readable logs and structured logging as well as levels to aid in debugging I think are the primary requirements

Add ability in that java language LSP add-on to decompile jar files, if needed

Using: https://github.com/JetBrains/intellij-community/tree/master/plugins/java-decompiler/engine which can be pulled from here: https://mvnrepository.com/artifact/com.jetbrains.intellij.java/java-decompiler-engine the java LSP bundle to decompile and then re-initialize the project.

Re-design output to accomodate Info type violations

Violations in output need an additional type to distinguish Info type output

Windup integration

LSP analyzers must support windup rules. The goal is to support 80% of existing windup rules. In the first phase, windup rules can be converted to analyzers manually. In the next phase, a windup shim will facilitate running windup rules with analyzers.

Some java rules are not being triggered

See this commit for test data. Both added rules should be triggered according to the test data, but I can't see any references in the results.

Make CLI flags for overriding the settings file path

We need to set up cobra or go flags to make the settings files overridable.

Rule authors should be able to assign categories to tags

Rule authors should be able to categorize technology tags.

Find the correct way for a provider/capability to document its template context.

Today, a provider is responsible for adding extra information for its particular capability. We need an easy way for these things to provide this information, such that someone can write rules against this information.