eclipse-che4z/che-che4z-lsp-for-cobol

COBOL Language Support provides autocomplete, highlighting and diagnostics for COBOL code and copybooks

che-che4z-lsp-for-cobol's Issues

Provide Compiler Directives Support

Description

Michelle wants to have a syntax check for the COBOL compiler directives that are used in her code.

Acceptance Criteria

• There is syntax check performing for the compiler directives
• The solution is covered with test
• Test the solution on all platforms and supported IDE's

Technical Details

The current state of this capability is that now there is a CobolUnsupportedFeaturesIgnorerImpl that removes the lines with the compiler directives. Also, there is special logic in the preprocessor to recognize them. It means that the logic of CobolUnsupportedFeaturesIgnorerImpl should be changed in order to prepare the compiler directives to be processed in the preprocessor.

Introduce Completion Suggestions for Paragraphs

Description

As a Michelle, I would like to use autocomplete feature so I can see all my paragraphs

Acceptance criteria

• Autocomplete works for paragraphs in a current file
• Should work in all the supported IDEs
• The list of paragraphs should appear after the list of keywords
• The solution is covered by tests

Technical Notes

Provide a new delegated implementation of AbstractCompletion

Enhance copybooks support

Description:

Enhance and improve the copybooks support - TBD

Acceptance criteria

• Built the text of the copybook inside the program text to analyze it as a one complete program

Paragraph usage on a continuation line is not recognized

Description:
Paragraph usage (i.e. in a PERFORM statement) is not recognized if the paragraph name is on the next line that is marked as a continuation line.

Steps to reproduce:

1. Open VSCode or Theia with an installed extension
2. Create a .cbl file that will contain the following piece of code:

000000 PROCEDURE DIVISION.
000001 PERFORM
000002- PARAGRAPH-NAME
000003 PARAGRAPH-NAME.

3. Make a right-click on a PARAGRAPH-NAME on Line 000002
4. Select "Find all references"

Expected behavior:
Two usages are shown

Actual behavior:
Only the second usage is shown

Add content areas A and B support

Description:
As a software engineer, I would like to differentiate between areas in COBOL as fixed format implies so I can validate more input and show correct diagnostic messages.

Acceptance criteria:

• Area A and Area B text preprocessing
• Validate items which depend on area A or B

Definition of Done:
This US will be completed once all dependencies are completed and use cases are implemented.
Example of use case:​​​​​​ 

1. IDENTIFICATION DIVISION should start in Area A
2. Differentiate between areas
3. Validate where IDENTIFICATION keyword begins
4. Publish diagnostic

Add source type information in the diagnostic object

This modification is required in order to avoid the problem we got on Che instance where instead of the source attribute is displayed to the user some internal object reference, as shown below:

Here the reference from the LSP protocol specification: https://github.com/microsoft/language-server-protocol/blob/gh-pages/_specifications/specification-3-14.md#diagnostic

"01" should begin in area "A"

Validate if '01' in the variable definition is in the correct position

Update LSP client and documentation

Update the client-server communication and update readme and contributing

Add JaCoCo to sonarCloud

Provide JaCoCo maven configuration to sonar-cloud in jenkins file.

Introduce Dependency Injection

Description

As a software engineer, I would like to have a dependency injection support so I can build the dependencies between the components in an easy way.

Acceptance criteria

• TPSR is verified

• DI is configured for the service and core parts

• DI is applied to the current dependencies

• The solution is tested on all the platforms

• Licence information is updated

Technical notes

As a software engineer, I would like to have a lightweight DI framework, not some enterprise-level one. Have a look at Guice.

Handle missing java with error message

when java is not installed then an error has to be shown to the user

please mark as enhancement

Introduce Completion Suggestions for Copybook Names

Description

Michelle wants to have all the copybook names that are used in the current COBOL file, as well as those that present in the workspace to be suggested if autocomplete is invoked.

Acceptance Criteria

• Copybook names that are already used in the current document are listed after paragraphs and before snippets with kind "Reference"
• Copybook names that are present in the current workspace are listed after used ones with kind "File"
• No documentation providing is required
• The solution is covered with tests
• The solution is tested on all the platforms

Increase Code Coverage for Databus

Description

As a software engineer, I would like to have my code covered with tests.

Acceptance criteria

• Code coverage for lsp-core-domain module is more 80%. The current value is 0%
• Coverage is made with unit tests, not use cases in service in order to keep the coverage module-related.

Automatically update copyright date

Concatenate the error message

Problem messages contain a string "_generated_diagnostic_collection_name#" that is longer than the error message itself

Detecting Errors.

If there is already a detected error, new errors are not displayed anymore. (No period in Configuration Section)
Also quick fix does not fix the errors.

Update the plugins properties

Description:
Update the plugins properties (publisherID, plugins name, version) for LSP for COBOL
Please also check value for the author and description

Acceptance criteria:

publisherID = BroadcomMFD
name = lsp-for-cobol
displayName = LSP for Cobol
version = 0.8.0
categopry="Programming Languages"

Rally ID: US608918

Test automation Integration with Eclipse pipeline

Description:
Michelle, the open-source contributor, wants to run an automatic regression test suite from Eclipse infrastructure so that she improve her code testing.

Acceptance criteria:

• The COBOL test automation is open-source and available for the community
• Automatic tests can be performed by contributors
• Test results for each run are stored and available for later use
• Testing data is available in the repository

Definition of Done

• Definition of Done should be covered by tasks and labels in Github, so every story should have following by the time it is considered done:
• Story has been implemented according to the description 
• Unit testing was done (Preferably by TDD approach, coverage should be at least 70%)
• Code review has been done by at least one another team member
• Manual testing has been done
• Documentation was added and reviewed (if needed)
• Pull request was approved
• The story has been merged into development branch
• Testing was done on a development branch
• All the Acceptance Criterias were met and PO accepted the story

Introduce Copybooks Support

Description

As a software engineer, I would like to have semantic analysis based on referred copybooks so I can perform the analysis more accurately having the complete semantic context.

Acceptance criteria

• Scan the predefined folder and search for a copybooks
• Support only known extensions (cbl, cpy, cobol, cob)
• Analyze a copybook and retrieve a semantic context
• Test the solution on all platforms and supported IDE's

Technical Notes

As an example: Create a copybook file that contains a definition of a variable. In the main/root COBOL file, import/copy the copybook and use the variable, defined in the copybook, for a statement in PROCEDURE DIVISION.

https://about.sourcegraph.com/blog/part-2-how-sourcegraph-scales-with-the-language-server-protocol

Java version matching

Trying to match Java version by parsing 'java -version' output should respect other JRE/JDK implementations. This would fail to match Java version if OpenJDK is used.
https://github.com/eclipse/che-che4z-lsp-for-cobol/blob/09830e49a941c0575b79416163418f5c257c3184/clients/cobol-lsp-vscode-extension/src/extension.ts#L69
Example output:

# java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (IcedTea 3.13.0) (Alpine 8.222.10-r0)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

Perform Engineering Excellence

Description

As a software engineer, I would like to have my code aligned to the code excellence standards so I can easily maintain code and introduce new features

Acceptance criteria

• The code is aligned to the defined code style and naming convention (see below)
• The packages are renamed to represent the Broadcom branding

Code style guide

1. Interface names should not contain any markers (like "I" at the start) because it is not necessary for clients to know that type is an interface. Implementations should contain the interface name and a word that describes the purpose of this class, or "Impl" at the end if it is planned to have only one implementation at the moment (only if you cannot imagine something more descriptive).
   For example, CobolTextDocumentService implements TextDocumentService.

2. If some class implements some pattern (i.e Singleton, Strategy, etc.) then it should be specified in the name.
   For example, CobolErrorStrategy.

3. Class names should not be too generic or contain some "er"-words - Manager, Server, etc. unless it is some specification or domain requirement.
   For example, MessageManager is bad, Messages is good. If you have a problem with naming, maybe you should split the class, to let every part have only one purpose or abstraction level.

4. Methods should contain only one responsibility and one abstraction level. Also, the names should be descriptive.

5. Each public method should have a Javadoc comment that describes the parameters and the results (i.e. does it accept or return null if it may be important, that checks for the parameters does it apply). Also, if there is some complicated logic, it should be described in the comment. The best practice would be also to add code examples to the class Javadoc comment.

6. There should be no author section at the comments.

7. Unit tests that are use cases should contain the description and purpose of the case.

8. You should avoid in-code comments unless it is necessary.

9. Try to use as fewer method parameters as you can. The best practice is to make the method name describing why the parameters are necessary, for example - getUserBy(long id).
   Avoid (until is not strictly necessary) to pass null as parameter

10. Always extract complicated logical sentences.

11. Use Lombok where it is possible to reduce boilerplate code.

Update Readme

Update readme file using Cobol Language Support instead of LSP for COBOL extension

Provide file logging for the server

Description:
Provide a rotating file logging that will store files in the "server" folder not more than one month and limit the size of one file to 10MB and total to 1GB. The default log level should be set as "ERROR".
The enhancement should work with the existing Logback configuration.

Introduce a File Engine Based on File System

Description

As a software engineer, I would like to have a common file engine that would be available from service and core parts and store all the files so the application will have one entry point to manage the Cobol files

Acceptance criteria

• File engine is available for service and core parts
• It has only one-directional dependency - engine cannot control service or core for better component decoupling
• Works directly with the file system using the calling directory as root
• Allows access to the copybooks
• Performs cashing of copybooks
• Solution is tested on all the platforms

Add Watcher to Trigger Analysis if New Copybooks Appear

Description:

Michelle wants her analysis to be automatically run if new copybooks appear.

Acceptance Criteria:

• The server is triggered if there are any changes or new files inside the COPYBOOKS directory
• The copybooks cache is invalidated
• The analysis is applied for the documents

SKIP* considered as Syntax error

https://www.ibm.com/support/knowledgecenter/SSQ2R2_9.1.1/com.ibm.ent.cbl.zos.doc/PGandLR/ref/rlcdsski.html

It is written in the working storage area like this:

01 Var1
     03 Var2
...
     03 Var25
SKIP2
01 Var26
     05 Var 27

Add Mockito for unit test to mock antlr methods

There is a requirement to fulfill the code coverage, in order to do so we need to call some methods of the visitor. As the visitor uses the parser context we need to mock it using Mockito.

Add Uncaught Exception Handling for Syntax Analysis

Description:
As a part of quality assurance, I would like to have all the exceptions appearing while the syntax and semantic analysis to be caught and logged.

Acceptance Criteria:

• All the exceptions that are thrown by Language Engine are caught and logged as errors.

Technical Details:
We use CompletableFuture framework to run the analyzing tasks asynchronously. It also means that all the uncaught exceptions will be hidden. To prevent it, we may use
CompletableFuture.whenComplete((res, ex) -> ex.printStackTrace()); to log the exception stack.
The best methods to apply it are MyTextDocumentService#analyzeDocumentFirstTime() and MyTextDocumentService#analyzeChanges().

Increase Code Coverage for Language Engine

Description

As a software engineer, I would like to have my code covered with tests.

Acceptance criteria

• Code coverage for lsp-core-cobol-parser module is more 80%. The current value is 63.5%
• Coverage is made with unit tests, not use cases in service in order to keep the coverage module-related.

Integrate SonarCloud with Jenkins pipeline

Use SonarCloud to scan source code and perform code smells within the Jenkins pipeline

put COBOL in uppercase

All instances of COBOL should be uppercase.

StringIndexOutOfBoundsException on MyDocumentModule

Description:
There are some edge cases not covered in MyDocumentModel#retrieveToken(Position position, Line route) method. Please, add some checks before calling the substring. Also, update the test.

Steps to reproduce:
TBD

Fix Code Smell Found by the Static Analysis

Description:

As a software engineer, I would like to have my code of good quality. Fix the code smell found by SonarCloud in order to raise the maintainability and prevent issues.

Acceptance criteria:

All the confirmed code smells are fixed.

Update package.json name and repository informations

Description: Modify the package json in order to include repository informations

Acceptance criteria: Apply the required modifications on the specific fields:

• name
  cobol-language-support
• displayName
  Cobol Language Support
• description:
  COBOL Language Support standardizes the communication between language tooling and your code editor using the Language Server Protocol (LSP) .
• repository URL
  "repository": {
  "type": "git",
  "url": "https://github.com/eclipse/che-che4z-lsp-for-cobol.git"
  }
• bug URL
  "bugs": {
  "url": "https://github.com/eclipse/che-che4z-lsp-for-cobol/issues"
  }

Syntax highlighting for variables and paragraphs

Description

Michelle wants to get a Syntax highlighting option also for variables and paragraphs so that she can see the usage of those elements immediately with a background made by the client.

Acceptance criteria

• Syntax highlighting feature is applied for variables and paragraphs in the currently opened document.
• Tested on all platforms and supported IDEs

Technical Notes

• The expected behavior is to show the highlighting only for the dynamic/semantic elements (i.e. variables, copybooks, and paragraphs)
• Refer to the Language Server Protocol specification https://microsoft.github.io/language-server-protocol/specification#textDocument_documentHighlight
• Reuse the existing semantic context definition for highlight.
• Provide a new delegate

References are displayed twice

I am testing the FIND ALL REFERENCES and it is showing me the result twice.

Same with Peek Definition

Support ++INCLUDE for copybooks

LSP marks ++INCLUDE as a syntax error.

Make Analysis Thread-safe

Description:
As a software engineer, I want to have all the components that work in the multithread environment to be thread-safe. We may have a ConcurrentModificationException on Analysis class if several threads would run it simultaneously.

Acceptance Criteria:

• Class com.ca.lsp.cobol.service.delegates.validations.Analysis is thread-safe

Add gifs in Readme file

A Readme file will show short gifs with syntax awareness examples.

Replace Usage of Deprecated Method StringUtils.getLevenshteinDistance

Description

There is a usage of a deprecated method StringUtils#getLevenshteinDistance() in CobolVisitor. It is recommended to replace it with LevensteinDistance of Apache commons-text.

Technical details

TPSR verification is required.

No Log4J configuration found

Description:
There is an error message appearing in a log console while running the extension. See below for the detailed message.

Steps to reproduce:

1. Open VSCode with the extension installed.
2. Go to View -> Output
3. In the Source drop-down select "LSP extension for Cobol Language"

Expected Behavior:
No error message

Actual Behavior:
An error message appears:
ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2

Asterisk is in different columns when CTRL + / is used for commenting

I tried commenting my cobol code using CTRL+/ and the asterisk goes to different columns

Variable type change from group item to elementary item -> enhance syntax check for undefined PIC value

If a variable type changed at the compile/run time because of the poor definition of level data and the group item is not followed by a nested variable (a child) then our LSP should apply a check in order to see if the PIC value is defined.

In the example above we should see a syntax error on the VAR3 definition.

ArrayIndexOfBoundsException Appears if there are Copy Statements with Different Cases

Description:
In case if there are several COPY statements in a row with the same name, ArrayIndexOfBoundsException is thrown. Code example:

       IDENTIFICATION DIVISION.    
       PROGRAM-ID. TEST1.        
       DATA DIVISION.              
       WORKING-STORAGE SECTION.    
       01  PARENT.   COPY STRUCT.    
       COPY Struct.

The content of "STRUCT":

       02  PARENT2.
           03  CHILD1         PIC 9   VALUE IS '0'.
           03  CHILD2         PIC 9   VALUE IS '1'.
           03  CHILD3         PIC 9   VALUE IS '2'.

Error message:

java.util.concurrent.CompletionException: java.lang.ArrayIndexOutOfBoundsException: -1
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncRun.run$$$capture(CompletableFuture.java:1643)
	at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java)
	at java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1632)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
	at java.util.ArrayList.elementData(ArrayList.java:422)
	at java.util.ArrayList.remove(ArrayList.java:499)
	at com.ca.lsp.core.cobol.semantics.CobolVariableContext.buildVariableStructure(CobolVariableContext.java:83)
	at com.ca.lsp.core.cobol.semantics.CobolVariableContext.merge(CobolVariableContext.java:73)
	at com.ca.lsp.core.cobol.semantics.SemanticContext.merge(SemanticContext.java:33)
	at java.util.ArrayList.forEach(ArrayList.java:1257)
	at com.ca.lsp.core.cobol.preprocessor.sub.document.impl.CobolSemanticParserImpl.processCopybooks(CobolSemanticParserImpl.java:79)
	at com.ca.lsp.core.cobol.preprocessor.sub.document.impl.CobolSemanticParserImpl.processLines(CobolSemanticParserImpl.java:65)
	at com.ca.lsp.core.cobol.preprocessor.impl.CobolPreprocessorImpl.parseDocument(CobolPreprocessorImpl.java:89)
	at com.ca.lsp.core.cobol.preprocessor.impl.CobolPreprocessorImpl.process(CobolPreprocessorImpl.java:73)
	at com.ca.lsp.core.cobol.preprocessor.impl.CobolPreprocessorImpl.process(CobolPreprocessorImpl.java:54)
	at com.ca.lsp.core.cobol.engine.CobolLanguageEngine.run(CobolLanguageEngine.java:47)
	at com.ca.lsp.cobol.service.delegates.validations.CobolLanguageEngineFacade.analyze(CobolLanguageEngineFacade.java:56)
	at com.ca.lsp.cobol.service.delegates.validations.Analysis.run(Analysis.java:48)
	at com.ca.lsp.cobol.service.MyTextDocumentService.lambda$analyzeDocumentFirstTime$11(MyTextDocumentService.java:186)
	at java.util.concurrent.CompletableFuture$AsyncRun.run$$$capture(CompletableFuture.java:1640)
	... 6 more

Validation of Variables and Statements

I have a variable that is assigned as PIC 9(02).

It would be great if the extension

• can prevent me from moving "String data" to PIC 9 variables.
• can warn me that my variable will be truncated because It is only defined as PIC 9(02) and I am moving a value more than 2.
• validate missing periods in all my PERFORM inside the procedure. So far, it only validates the last PERFORM that I have.

Increase Code Coverage for Service

Description

As a software engineer, I would like to have my code covered with tests.

Acceptance criteria

• Code coverage for lsp-service-cobol is more 80%. The current value is 81.0%

Incorrect Error Position Underlining

Description

I have a definition of variables:

01 VISATOTAL2.
     02 PRNVISATOTAL.
          03 ABC.
             04 DEF.
                 05 HGI.

When I call HGI as in the image:

Actual behavior:

"HGI of DEF" is underlined

Expected behavior:

The error should be shown on VISATOTAL, one line after the call of HGI.

Introduce Referencing Support for Copybooks

Description

Michelle wants to have "Find all references" and "Go to Definition" support for copybooks.

Acceptance Criteria

• "Find all References" shows all the usages of the selected copybook name
• Verify that highlighting for the copybook names works out-of-the-box with this solution
• The solution is covered with tests
• The solution is tested on all the platforms

Create Changelog md file

Create Changelog file for the project

Remove deprecated parameter forkMode from pom.xml file

Description

There is a deprecated maven parameter used in lsp-service-cobol/pom.xml. See the warning message below:
The parameter forkMode is deprecated since version 2.14. Use forkCount and reuseForks instead.