strumenta/kolasu

Kotlin Language Support – AST Library

kolasu's Issues

Reimplement Node.transform(operation: (Node) -> Node, inPlace: Boolean): Node

There's a TODO there.

Consider creating method searchByPosition

This method should navigate the tree considering the position and the fact that a child should be contained in the parent also w.r.t. positions

Printing AST fails with private list fields

Also see #10

Handle null collections

Kolasu assumes lists and other collections are never null when mapping to EMF. However, nothing prevents the AST from containing null collections.

Errors should contain the type

Is the error lexical, syntactic or semantic?

InvalidPositions should check also the node itself, not only the descendants #12

Otherwise we risk having the root without a position, which causes problems

"isInterface" not recorded in EMF metamodel

This causes issues when importing a metamodel with interfaces from a language that doesn't allow multiple inheritance, because interfaces are recorded as abstract classes.

Publication: sources jar is empty

Move tests to assertk

Lately I've been using assertj a lot, and I must say that once you get over the "this is slightly longer syntax" phase, it offers a lot more ways to check stuff.

https://www.kotlinresources.com/library/assertk/

Mak ParserRuleContext.toPosition default parameter be true

fun ParserRuleContext.toPosition(considerPosition: Boolean = true)

Complete fun provideNodes(KClassifier?): Boolean

There's a TODO there.

Create comparison for nodes which permits to ignore certain collections

We could pass an object of type "IgnoreChildren" and the comparison could take that in account

Complete provideNodes(KTypeProjection): Boolean

There's a TODO there.

Get the package added to JCenter

So that it can be used more easily

Collection usage

I see LinkedList and HashMap and such. Why not use the Kotlin collections?

Several files with more or less the same functionality

Navigation, Processing, and Traversing do more or less the same. Should we put some stuff together?

Point/Position naming

"Position" sounds like something that is in 1 place. In Kolasu it is more of a range from one point to another.

And of course, Point should be Position as it is in JavaParser :-)

Don't include nodeType in nodeProperties

• Breaks XML serialization tests
• Fills EMF export with useless nodeType properties (that e.g. show up in Parser Bench)

EMF CLI: include issues

Draw tree using Graphviz

It is useful for documentation, debugging, and presentations

The operation lambda for Node.transform can process same node more than once

kolasu/core/src/main/kotlin/com/strumenta/kolasu/model/Processing.kt

Line 318 in d774ff6

operation(this)

Please consider this test from the Minicalc project:

    @test
    fun transformVarName() {
        val startTree = MiniCalcFile(listOf(
            VarDeclaration("A", IntLit("10")),
            Assignment(ReferenceByName("A"), IntLit("11")),
            Print(ValueReference(ReferenceByName("A")))))

        val expectedTransformedTree = MiniCalcFile(listOf(
            VarDeclaration("B", IntLit("10")),
            Assignment(ReferenceByName("B"), IntLit("11")),
            Print(ValueReference(ReferenceByName("B")))))

        assertEquals(expectedTransformedTree, startTree.transform {
            println("Hashcode: ${it.hashCode()}")
            when (it) {
                is VarDeclaration -> VarDeclaration("B", it.value)
                is ValueReference -> ValueReference(ReferenceByName("B"))
                is Assignment -> Assignment(ReferenceByName("B"), it.value)
                else -> it
            }
        })
    }

Note the println("Hashcode: ${it.hashCode()}") line that I added.

This is what's being printed if we run this test with the original implementation of the Node.transform function:

• Hashcode: 1422860865
• Hashcode: 1568352
• Hashcode: 48577
• Hashcode: 48577
• Hashcode: 1568352
• Hashcode: 2006568
• Hashcode: 48608
• Hashcode: 48608
• Hashcode: 2006568
• Hashcode: 499720
• Hashcode: 16120
• Hashcode: 16120
• Hashcode: 507408
• Hashcode: 1459116512

As you can see, some of the nodes are processed twice.

Now please see what's printed in the case I comment out the operation(this) at https://github.com/Strumenta/kolasu/blob/master/core/src/main/kotlin/com/strumenta/kolasu/model/Processing.kt#L318:

• Hashcode: 48577
• Hashcode: 1568352
• Hashcode: 48608
• Hashcode: 2006568
• Hashcode: 16120
• Hashcode: 507408
• Hashcode: 1459116512

We don't see duplicates anymore, however, the test passes in either case.

P.S. I first noticed this behavior in the 0.1.1 version. I tried to use the newest version but it seemed to behave similarly.

EMF: support multiple EPackages

Publishing using GitHub registry

https://help.github.com/en/github/managing-packages-with-github-package-registry/configuring-gradle-for-use-with-github-package-registry

Natively handle LocalDate/Time

CLI support

Provide optional tools to build a command-line interface for your parser.

Move to com.strumenta group on Maven

To do that we need to ask the authorization to Sonatype

Move to strumenta package

Update gradle

Have position include the source the node is coming from

This is useful to handle inclusions or transformations with nodes coming from multiple files

Issue severity

Issues could be errors, warnings, notes, ...

Printing AST fails with private fields

Javadoc with Dokka

Implement other collections in Node.transformChildren or throw exception

Marker for synthetic/generated nodes

Some nodes are calculated, for example the CharacterType node in the RPG parser. We could mark them appropriately to indicate they have a reason for not having a position

Ensure properties are not duplicated

There are cases in which we override properties and we end up with two properties definition.

In parser bench this is visible:

Option to generate metamodel with or without Kolasu included

For backward compatibility, the default is to include the Kolasu metamodel. However, when exporting to Parser Bench or other software that is already aware of Kolasu, it doesn't make sense to include it.

Printing AST: shorter format for empty arrays

See for examples:

CompilationUnit {
  _allDataDefinitions = [
  ]
  allDataDefinitions = [
    DataDefinition {
      fields = [
      ]
      initializationValue = null
      name = Msg§
      type = StringType(length=12)
      muteAnnotations = [
      ]
      parseTreeNode = null
    } // DataDefinition
  ]
  compileTimeArrays = [
  ]
  dataDefinitions = [
    DataDefinition {
      fields = [
      ]
      initializationValue = null
      name = Msg§
      type = StringType(length=12)
      muteAnnotations = [
      ]
      parseTreeNode = null
    } // DataDefinition
  ]
  databaseInterface = com.smeup.rpgparser.interpreter.DummyDBInterface@6ad43d70
  directives = [
  ]
  entryPlist = null
  fileDefinitions = [
  ]
  inStatementsDataDefinitions = []
  main = [
    MainBody {
      stmts = [
        EvalStmt {
          expression = [
            StringLiteral {
              value = Hello World!
              parseTreeNode = null
            } // StringLiteral
          ]
          flags = EvalFlags(halfAdjust=false, maximumNumberOfDigitsRule=false, resultDecimalPositionRule=false)
          operator = NORMAL_ASSIGNMENT
          target = [
            DataRefExpr {
              variable = Ref(Msg§)[Unsolved]
              parseTreeNode = null
            } // DataRefExpr
          ]
          muteAnnotations = [
          ]
          parseTreeNode = null
        } // EvalStmt
        DisplayStmt {
          factor1 = null
          response = [
            DataRefExpr {
              variable = Ref(Msg§)[Unsolved]
              parseTreeNode = null
            } // DataRefExpr
          ]
          muteAnnotations = [
          ]
          parseTreeNode = null
        } // DisplayStmt
        SetOnStmt {
          muteAnnotations = [
          ]
          parseTreeNode = null
        } // SetOnStmt
      ]
      parseTreeNode = null
    } // MainBody
  ]
  minTimeOut = null
  subroutines = [
  ]
  timeouts = [
  ]
  parseTreeNode = null
} // CompilationUnit

Create transformDescendants(InPlace)

Since transformChildren only transforms the direct children in the node, there is not really a full-tree transformation function. That should be easy to make.

mapTree and transformTree don't recurse

It seems like these two only look at their direct children and don't recurse deeper. I assume that's a bug? Or maybe you need to do the recursion in the "operation" you pass?

Support optionally named

Also, adapt references to support this

Allow maps in nodes

http://wiki.eclipse.org/EMF/FAQ#How_do_I_create_a_Map_in_EMF.3F

ParseTree could be much simpler

ParseTree could be much simpler, see here

Better usability with Java

@JvmOverloads, remapping of useful constants, extension methods, etc.

Add operator Point + String

    operator fun plus(text: String) : Point {
        return when {
            text.isEmpty() -> this
            text.startsWith("\r\n") -> Point(line + 1, 0) + text.substring(2)
            text.startsWith("\n") || text.startsWith("\r") -> Point(line + 1, 0) + text.substring(1)
            else -> Point(line, charPositionInLine + 1) + text.substring(1)
        }
    }

Documentation

Improve documentation, ensure every class/method has KDoc comments attached.

Make Kolasu multi-platform

Complete Any?.toXML

There's a TODO there.

Intellij doesn't understand generated test parser

When loading the project all fresh, intellij does not set the source path to the generated test parser. It is in a rather peculiar location if you ask me...

Update Kotlin

Node.addBefore(Node) and Node.addAfter(Node)

I guess these will be useful eventually.