konveyor/windup-shim

Compatibility layer to make windup XML compatible with analyzer-lsp

Code of Conduct

Refer to Konveyor's Code of Conduct here.

Building the Base Image

The shim will use the analyzer-lsp as the base image. You will need to build the base by:

1. Clone analyzer-lsp.

git clone [email protected]:konveyor/analyzer-lsp.git

2. Build the base image

cd analyzer-lsp && podman build -f Dockerfile -t analyzer-lsp

Running the demo

1. Build the Image

podman build -f Dockerfile -t windup-shim

2. Run the image

podman run -it windup-shim test /windup-rulesets/rules/rules-reviewed

windup-shim's Issues

Java addon rules

These rules in the rules-java addon in windup are fundamental for some technology tagging rules. We should convert them:

• HardCodedIP - this is giving us one violation in demo
• DiscoverArchiveLicenseFilesRuleProvider - this finds license for the project
• DiscoverArchiveManifestLicenseFilesRuleProvider - finds manifest
• DiscoverPropertiesFilesRuleProvider - finds properties files
• JavaSourceFilesRuleProvider - finds java files, this can be skipped, needs evaluation

Error in some builtin rules: could not run grep with provided pattern exit status

Following rules result in this error

• component-changes-00019
• java-generic-information-00010
• xml-removed-components-00008
• os-specific-00001

It appears that in all these rules, the wildcard characters are interpreted as wildcards, but the intention is to escape them instead.

Named capture groups should be converted correctly

See example rule:

- category: mandatory
  customVariables:
  - name: class-name
    nameOfCaptureGroup: class-name
    pattern: org.hibernate.cfg.(?P<class-name>(AbstractPropertyHolder|AccessType))?
  description: |-
    Class {class-name} within the org.hibernate.cfg package has been moved
    This class within `org.hibernate.cfg` has been moved; see link for more information.
  effort: 1
  labels:
  - konveyor.io/target=hibernate6.2
  - konveyor.io/target=eap8
  - konveyor.io/source
  - hibernate
  links:
  - title: Hibernate 6 migration guide - API/SPI/Internal distinction
    url: https://docs.jboss.org/hibernate/orm/6.2/migration-guide/migration-guide.html#api-internal-cfg
  message: This class within `org.hibernate.cfg` has been moved; see link for more
    information.
  ruleID: hibernate-6.2-00040

In the above rule, <class-name> is invalid, regexp doesnt like "-" in variable name, we should convert these into camel case instead.

Convert Source and Target technologies from Windup rules into labels on LSP rules

This is a work item in the shim in supporting the larger labels selector in LSP engine. We need to convert the old windup technology metadata into well defined labels (with prefixes).

Handle Hint.Tag field

Currently this information is discarded, need to capture it and figure out where it belongs in the analyzer-lsp

Enable shim to automatically commit converted rulesets to the repo on PR merges

Open a PR to https://github.com/konveyor/rulesets/ when a PR merges

[BUG] PACKAGE location not working as expected

Windup java conditions with no location are currently being added a PACKAGE condition. This doesn't seem to yield any results in the code (see konveyor/analyzer-lsp#413, which works when IMPORT is used instead of PACKAGE).

We should probably use IMPORT instead of PACKAGE.

Handle Classification.Description field

Currently this information is discarded, need to capture it and figure out where it belongs in the analyzer-lsp

Fix ordering of technology tagging rulesets

For instance, database-technology-usage ruleset needs to run after database ruleset.

Capture links and category from Windup rules

The hint field in the Windup Rules has category-id and links in addition to the message. We need to grab those and copy them to right fields in LSP rules. For the "potential" issues we need to evaluate how many cannot be converted to right tagging rules.

Converting windup hint messages into valid markdown

We need to convert all those message strings from html to valid markdown for displaying in the UI.

Enable global CI for shim

We should enable the global CI in this repo to test the full analysis flow via the addon. The test case for this should cover the following scenarios at the minimum:

• Upload a custom XML rule and run analysis against tackle-testapp with targets cloud-readiness, linux
• Do not upload any custom rules and run analysis against tackle-testapp with targets cloud-readiness, linux.

Identify rules where we need to use package searches

In the catchall rules, it looks like the rules should be doing package searches. We should first, identify these rules. Then we should update them either via shim or manually.

This is an example where we want a package search https://github.com/konveyor/rulesets/blob/01dcbeddca64c6fd22614d5b33590fd682b1fc18/default/generated/fuse/138-sonic-catchall.windup.yaml#L25

This is an example where we do not want it https://github.com/konveyor/rulesets/blob/01dcbeddca64c6fd22614d5b33590fd682b1fc18/default/generated/eap6/59-websphere-jms.windup.yaml#L32

Strip out non-capturing groups and escaped colons

grep does not support non-capturing groups ((?:), for instance:

jmle@fedora project $ grep -o -n -E '(?:[\\\/]+[^\n\t]+)+' ...
grep: warning: ? at start of expression
jmle@fedora project $ grep -o -n -E '([\\\/]+[^\n\t]+)+' ...
5:\\da

So they should be stripped out. It also does not properly treat escaped colons (\:), giving another warning:

grep: warning: stray \ before :

Handle (Hint|Classification).Quickfix field

Currently this information is discarded, need to capture it and figure out where it belongs in the analyzer-lsp

Fully capture ruleset metadata

Need to evaluate the following fields, determine whether they belong in the analyzer-lsp, and update the code to include them in the ruleset.yaml:

• Dependencies (should be able to drop this)
• SourceTechnology (should convert to tag)
• TargetTechnology (should convert to tag)
• Phase (could convert to tag)
• ExecuteAfter (used only in windup-rulesets/rules/rules-reviewed/technology-usage/spring-catchall.windup.xml)
• ExecuteBefore (seems unused in windup)
• OverrideRules (seems unused in windup)

OWNERS.md Missing

OWNERS.md Missing per https://github.com/konveyor/community/blob/main/MAINTAINERS.md guidelines

Convert Hibernate rules

This is remaining work from #59

Improve convert command to handle failures better

• If there are no files found that are windup rules, we need to continue doing what we are doing today i.e. to ignore them and print to stdout the names of those files.

• If we do find a windup ruleset that is not valid, we need to print to stdout AND return a non-zero exit code so that addon. We can push this behavior behind a flag.

Handle (Hint|Classification).Effort field

Currently this information is discarded, need to capture it and figure out where it belongs in the analyzer-lsp

Create tag and cut a release

Error matching version range

While executing podman run -it windup-shim test /windup-rulesets/rules/rules-reviewed, the following version ranges are not recognised:

• [5]
• [2.0,2.1,2.2,2.3]

Handle Filecontent.Filename field

Currently this information is discarded, need to capture it and figure out where it belongs in the analyzer-lsp

Java EE addon rules

These rules in the java-ee are also important for tagging as well as facts.

Here are some important ones I found:

• DiscoverJPA(Configuration|Annotations)
• DiscoverEJB(Configuration|Annotations)
• DiscoverWebXML
• DiscoverSpringConfigurationFile
• DiscoverHibernate (Tracked in #71)

Some of these rules store data into graph model which is later used in the reports. This is the information that we want to display into the "Facts" table. For example, the JPA rule goes and finds JPA configuration files, searches for name of the JPA persistent units and stores them in graph. We want a generic way of storing this data (probably via Variables?) such that it can be displayed in UI.

Explicit (wildcard) source labels.

For rulesets that contain rules with a combination of labels for a target. Some rules have explicit sources but others pertain to all sources. In windup, the Rule without an explicit source, is matched for all sources.

Consider the following use case based on Shim generated rules.

Labels selected by the user (in the UI):

• konveyor.io/source=S1
• konveyor.io/target=T1

Generated ruleset.

Rule A:

• labels
  • konveyor.io/source=S1
  • konveyor.io/target=T1

Rule B:

• labels
  • konveyor.io/target=T1

Rule C:

• labels
  • konveyor.io/source=S2
  • konveyor.io/target=T1

The intention is for rules A & B (but not C) be matched. How to build a selector expression to accomplish this?

A selector of (konveyor.io/source=S1 && konveyor.io/target=T1) will only match Rule A.
A selector of (konveyor.io/source && konveyor.io/target=T1) will match all (3) rules A,B,C which is not intended.

---

Potential solution:

The shim generated Rule-B as:

Rule B:

• labels
  • konveyor.io/source
  • konveyor.io/target=T1

Consider converting groovy rules in the technology-usage ruleset

"where" clause not being properly translated

Some where clauses are not appearing as translated in the YAML rules. See for instance this example in the Failing rules document with a link to the original windup rule.

Rules that look at .jar files for tagging should be converted into actual dep rules

Find example https://github.com/windup/windup-rulesets/blob/124a9d6122dc6732fb7729b56623a301ac713a0b/rules/rules-reviewed/technology-usage/mvc.windup.xml#L37-L47

Parse <a href=> elements from messages and convert them to actual links in new format

I observed that some links are part of a message. There is a dedicated field for links in the Rule format, we should convert/copy them to links. We can keep the original tags as-is for parity.

Tracking issue for Field Capture Work

• #52
• #51
• #50
• #49
• #48
• #53

Support for "rules-generated"

The rules-generated folder of rules withing the windup rulesets project is not being taken into account when calculating the total percentage of windup rules that are passing. These rulesets are also important, since they contain some rules belonging to the quarkus migration path.