atlanmod/mogwai

Automatic translation from OCL to Gremlin

 Mogwaï

Mogwaï is an efficient and scalable query framework that takes benefits of Graph database query facilities to perform queries over NeoEMF models.

It relies on a model-to-model transformation from OCL to Gremlin, the Tinkerpop query language to manipulate Blueprints compliant databases. Generated Gremlin traversals are computed on the database side, bypassing limitations of EMF API and improving execution time and memory consumption performances.

Mogwaï is fully integrated in NeoEMF, and provides its own PersistentResource extension which augment the default one with a basic query API. This API can be used to provide OCL queries (both inline or file-based) to Mogwaï, that will translate them into the appropriate Gremlin expression(s) and return results reified into navigable EObjects if needed.

News: Validation Benchmark

We have developed a new model validation engine to improve the evaluation of constraints over large models, and reduce the time required to retrieve the instances to validate. You can validate an entire model by using the MogwaiResource#validate method, that will optimize the input OCL constraint and translate it into an efficient Gremlin script.

You can run the validation benchmarks from the benchmarks project. Note that you first need to run the ModelMutator application, that will create a mutated version of the benchmark models containing elements violating the benchmarked constraints. You can then run the different queries in the package. There are 3 variants of each query: QueryName_OCL (standard OCL computation), QueryName_StandardMogwai (standard allInstances and Mogwai translation of the constraint) and QueryName_OptimizedMogwai (constraint pre-processing and optimized translation).

Installation

You can download the last version of the update site here. NeoEMF (v1.0.2 or newer), ATL (v3.8.0 or newer), EMFTVM (v4.0.0 or newer), OCL.ecore (v3.3.100), and OCL Examples and Editors SDK (v6.3.0 or newer) have to be installed in your Eclipse environment to run Mogwaï.

Alternatively, you can get a copy of Mogwaï into your Eclipse workspace by following these instructions:

• Clone the repository
• Install IvyDE
• Import the following projects in your workspace: fr.inria.atlanmod.mogwai.*
• You can also build the update site locally by importing fr.inria.atlanmod.mogwai.update

Documentation

Online javadoc corresponding to the latest build on atl2gremlin branch.

Issues

The current version of Mogwaï (v0.0.1) is a prototype developed to assess the OCL-to-Gremlin transformation we have presented here, and the ATL-to-Gremlin extension presented here. Bugs and wrong translation of complex expressions may occur, and the repository is regularly updated with bug fixes and enhancements.

If you have problems running Mogwaï or if your input query is not translated properly, please submit an issue on the issue tracker

mogwai's Issues

Package and subproject names are a mess

Maven does not support ATL file compilation

This makes the compilation process complicated. A quick fix is to remove the maven build and use only Eclipse. It would disappoint @sunye ;), but otherwise there is no easy way to have a clean project when importing Mogwai

Rethink the project organization

Right now we have in core ATL/OCL/Gremlin queries and processors, but each one needs its corresponding transformation.* subproject to work.

A better organization should be found, for example:

• mogwai.core: all the abstract classes and Gremlin query processing classes
• mogwai.atl: ATL query processing classes and the corresponding transformation project
• mogwai.ocl: OCL query processing classes and the corresponding transformation project

CustomGremlinGroovyPipeline#undefined() only checks the first element of the pipeline

undefined() operation has been added to the CustomGremlinGroovyPipeline class because it is way more efficient than the groovy delegation implied by the definition of a new Gremlin step. However, the current implementation doesn't handle collections, and simply checks the first element of the pipeline.

I think this method should be moved somewhere else (for example as a static field in ModelDatastore), because it is not related to the pipeline itself, and could be computed easily with a traversal such as

[...]_().next().equals(ModelDatastore.UNDEFINED)

Mogwai Console only supports OCL query execution

The current implementation of the console does not handle ATL constructs or plain Gremlin. This could be interesting for interactive editing and model browsing.

Before implementing an ATL-based console we need to check if there is a real interest to do so. A dedicated ATL launcher could be enough and more adapted to the existing modeling scenarios.

Feature projects are not in the repository

mogwai.transformation.[atl/ocl] Activators are not used

I am not sure these Activator classes are required: they do not define any specific behavior, and are not related to the other classes of the plugins.

ModelDatastore#removeRef should return an Iterable

Same as #29: there may be multiple reference instances between the two elements. We should then return all the elements linked by the deleted references.

Remove ClassDiagram dependency in transformation.atl

This dependency is there to ease development, it cannot be resolved by client codes and has to be removed before publishing the update site.

Reproduce OCL Benchmark

Hi,

I have found the paper G. Daniel, G. Sunyé, and J. Cabot: "Mogwaï: a Framework to Handle Complex Queries on Large Models". I would like to reproduce the OCL benchmarks with Mogwai. The benchmarks can be found here: https://github.com/atlanmod/Mogwai/tree/master/benchmarks/fr.inria.atlanmod.mogwai.benchmarks
However, it seems that the benchmarks are inconsistent with the Mogwai API as some types like MogwaiOCLQueryBuilder, MogwaiQueryResult have been refactored.

Then I have found the OCL test cases provided here: https://github.com/atlanmod/Mogwai/tree/master/fr.inria.atlanmod.mogwai.transformation.ocl.tests
These test cases do not have compile errors. So I merged the OCL benchmarks into these test cases to get them running. I also copied the XMI models from here https://github.com/atlanmod/Mogwai/tree/master/benchmarks/fr.inria.atlanmod.mogwai.benchmarks/resources.

This is how my test case looks like:

public class RunMogwaiQueryTest {

  private static MogwaiResource resource;
  @BeforeClass
  public static void setUp() throws IOException {

    JavaPackage.eINSTANCE.eClass();
    EPackage.Registry.INSTANCE.put(JavaPackage.eNS_URI, JavaPackage.eINSTANCE);
    PersistenceBackendFactoryRegistry.register(MogwaiURI.MOGWAI_SCHEME,
        BlueprintsPersistenceBackendFactory.getInstance());
    ResourceSet rSet = new ResourceSetImpl();
    rSet.getResourceFactoryRegistry().getProtocolToFactoryMap()
    .put(MogwaiURI.MOGWAI_SCHEME, MogwaiResourceFactory.getInstance());

    URI xmiURI = URI.createURI("resources/models/org.eclipse.gmt.modisco.java.kyanos.xmi");
    URI mogwaiURI = MogwaiURI.createMogwaiURI(new File("resources/db/set1.graphdb"));

    if(!new File("resources/db/set1.graphdb").exists()) {

      try {
        ModelImporter.createNeoMogwaiResourceFromXMI(xmiURI, mogwaiURI);
      } catch (IOException e) {
        MogwaiLogger.error("Cannot create the Mogwai resource ({0})", mogwaiURI);
        throw e;
      }
    }

    resource = (MogwaiResource) rSet.createResource(MogwaiURI.createMogwaiURI(new File(
        "resources/db/set1.graphdb")));

    resource.load(Collections.emptyMap());
  }

  @Test
	public void textElementInJavadocQuery(){


		String expression = "self.compilationUnits.commentList->"
                + "select(each | each.oclIsTypeOf(Javadoc))->"
                + "collect(o | o.oclAsType(Javadoc).tags).fragments->"
                + "select(each | each.oclIsTypeOf(TextElement))->"
                + "asSequence()";		


		MogwaiQuery query = OCLQueryBuilder.newBuilder().fromString(expression)
				.context(JavaPackage.eINSTANCE.getModel()).build();
		NeoEMFQueryResult result = resource.query(query);

		System.out.println("mogwai: textElementInJavadocQuery size: "+result.getResults().size());
	}  

}

I can execute this as a regular JUnit Test (not a plugin test). In this case I get the following error:

javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: self for class: Script2
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:333)
	at org.codehaus.groovy.jsr223.GroovyCompiledScript.eval(GroovyCompiledScript.java:41)
	at javax.script.CompiledScript.eval(Unknown Source)
	at fr.inria.atlanmod.mogwai.processor.GremlinScriptRunner.runGremlinScript(GremlinScriptRunner.java:120)
	at fr.inria.atlanmod.mogwai.processor.AbstractQueryProcessor.runGremlinScript(AbstractQueryProcessor.java:370)
	at fr.inria.atlanmod.mogwai.processor.AbstractQueryProcessor.process(AbstractQueryProcessor.java:199)
	at fr.inria.atlanmod.mogwai.processor.AbstractATLProcessor.process(AbstractATLProcessor.java:70)
	at fr.inria.atlanmod.mogwai.processor.OCLQueryProcessor.process(OCLQueryProcessor.java:71)
	at fr.inria.atlanmod.mogwai.neoemf.processor.NeoEMFOCLQueryProcessor.process(NeoEMFOCLQueryProcessor.java:68)
	at fr.inria.atlanmod.mogwai.processor.OCLQueryProcessor.process(OCLQueryProcessor.java:1)
	at fr.inria.atlanmod.mogwai.query.MogwaiQuery.process(MogwaiQuery.java:99)
	at fr.inria.atlanmod.mogwai.query.MogwaiQuery.process(MogwaiQuery.java:74)
	at fr.inria.atlanmod.mogwai.neoemf.util.NeoEMFQueryHandler.query(NeoEMFQueryHandler.java:158)
	at fr.inria.atlanmod.mogwai.neoemf.resource.DefaultMogwaiResource.query(DefaultMogwaiResource.java:92)
	at fr.inria.atlanmod.mogwai.neoemf.resource.MogwaiResource.query(MogwaiResource.java:119)
	at fr.inria.atlanmod.mogwai.transformation.ocl.tests.execution.RunMogwaiQueryTest.textElementInJavadocQuery(RunMogwaiQueryTest.java:105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
Caused by: groovy.lang.MissingPropertyException: No such property: self for class: Script2
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:50)
	at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:49)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:231)
	at Script2.run(Script2.groovy:5)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:330)
	... 39 more

It seems to me that OCL expressions containing self reference are not supported right now (or that the context is not set properly). So I modified the expression like this:

String expression = "Model.allInstances()->any(true).compilationUnits.commentList->"
            + "select(each | each.oclIsTypeOf(Javadoc))->"
            + "collect(o | o.oclAsType(Javadoc).tags).fragments->"
            + "select(each | each.oclIsTypeOf(TextElement))->"
            + "asSequence()";

This seem working, although I see the following error:

javax.script.ScriptException: java.lang.NullPointerException: Cannot invoke method inE() on null object
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:333)
	at org.codehaus.groovy.jsr223.GroovyCompiledScript.eval(GroovyCompiledScript.java:41)
	at javax.script.CompiledScript.eval(Unknown Source)
	at fr.inria.atlanmod.mogwai.processor.GremlinScriptRunner.runGremlinScript(GremlinScriptRunner.java:120)
	at fr.inria.atlanmod.mogwai.processor.AbstractQueryProcessor.runGremlinScript(AbstractQueryProcessor.java:370)
	at fr.inria.atlanmod.mogwai.processor.AbstractQueryProcessor.process(AbstractQueryProcessor.java:199)
	at fr.inria.atlanmod.mogwai.processor.AbstractATLProcessor.process(AbstractATLProcessor.java:70)
	at fr.inria.atlanmod.mogwai.processor.OCLQueryProcessor.process(OCLQueryProcessor.java:71)
	at fr.inria.atlanmod.mogwai.neoemf.processor.NeoEMFOCLQueryProcessor.process(NeoEMFOCLQueryProcessor.java:68)
	at fr.inria.atlanmod.mogwai.processor.OCLQueryProcessor.process(OCLQueryProcessor.java:1)
	at fr.inria.atlanmod.mogwai.query.MogwaiQuery.process(MogwaiQuery.java:99)
	at fr.inria.atlanmod.mogwai.query.MogwaiQuery.process(MogwaiQuery.java:74)
	at fr.inria.atlanmod.mogwai.neoemf.util.NeoEMFQueryHandler.query(NeoEMFQueryHandler.java:158)
	at fr.inria.atlanmod.mogwai.neoemf.resource.DefaultMogwaiResource.query(DefaultMogwaiResource.java:92)
	at fr.inria.atlanmod.mogwai.neoemf.resource.MogwaiResource.query(MogwaiResource.java:119)
	at fr.inria.atlanmod.mogwai.transformation.ocl.tests.execution.RunMogwaiQueryTest.grabats09(RunMogwaiQueryTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
Caused by: java.lang.NullPointerException: Cannot invoke method inE() on null object
	at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:77)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:45)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
	at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:32)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at Script2.run(Script2.groovy:5)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:330)
	... 39 more

The following tables show the result sizes reported in the paper for the MoDisco and JDT test model and what I have reproduced.

MoDisco:

Testcase	Paper	Reproduced Mogwai	Reproduced OCL Interpreter	Reproduced IncQuery/Viatra
textElementInJavadoc	12359	12359 ✔️	12359 ✔️	12359 ✔️
throwsExceptions	0	0 ✔️	0 ✔️	0 ✔️
invisibleMethods	134	0 ❌	134 ✔️	134 ✔️
grabats09	0	0 ✔️	0 ✔️	0 ✔️

JDT:

Testcase	Paper	Reproduced Mogwai	Reproduced OCL Interpreter	Reproduced IncQuery/Viatra
textElementInJavadoc	54201	54201 ✔️	54201 ✔️	54201 ✔️
throwsExceptions	1155	1155 ✔️	1155 ✔️	1155 ✔️
invisibleMethods	3927	0 ❌	3927 ✔️	3927 ✔️
grabats09	92	0 ❌	42 ❌	42 ❌

I am grateful for any support to reproduce the results reported in the paper.

Duplicated Ivy.xml in each sub-project

Ivy.xml files are duplicated in all the sub-projects, and are painful to use when cloning the repository and configuring the dependencies.

Using an other dependency / project life-cycle solution such as maven would be a good idea, but there are few challenges to tackle:

• we need to compile ATL transformations in the Maven build process
• we need a complex, 2 step Tycho-based build to generate the update site and resolve Eclipse-based dependencies (same as NeoEMF)

For now the former is an issue, because there is no usable solution to compile ATL transformations in a maven build. The later is similar to what we use in NeoEMF

Collect iterator variables are not translated

For now iterator variables in collect() body are not translated, meaning that it is not possible to access a collect iterator variable in its body.

Example (simplified from Train Benchmark):

OCL
`package railway

context Sensor

def:
    connectedSegments : Bag(TrackElement) = Sensor.allInstances()->collect(sensor |
    sensor.monitors->collect(segment1 |
        segment1.connectsTo->select(segment2 |
            segment2.monitoredBy->includes(sensor))))

endpackage`

Gremlin

def metaSensor = g.getIndex("metaclasses",Vertex.class)[[name:"Sensor"]]; def metaSensorNode = (metaSensor.hasNext() ? metaSensor.next() : null); metaSensorNode.inE("kyanosInstanceOf").outV ._().outE("monitors").inV .**segment1**.outE("connectsTo").inV .filter{def segment2 = it;segment2.outE("monitoredBy").inV .gather.transform{it.contains(**sensor**);}.next();}._()._();

Two errors there:

• segment1 should not appear in the traversal (it has not been declared, and removing it would not change the result)
• sensor variable (collect iterator) in the contains has not been declared, and will throw a NPE

New vs. old collect() translation

The new collect() translation is more flexible and allows to define iterator variables. However, the gather/scatter pattern may be quite heavy, especially if the query contains several nested collects.

Need to perform some benchmarking to see if there is a translation better than the other. If it is the case it would be possible to have two kinds of translations: a simple one (outE/inV) if the collect iterator is never accessed, and a complex one (gather/scatter or similar) if the iterator is used in a nested operation.

This question comes from the discussion with @bluezio about Mogwaï performance compared to Hawk/Neo4j and Hawk/OrientDB

Generated code performance vs Dynamic groovy

Is it interesting to generate native code that is more complex but faster than dynamic Groovy implementation?

For example: collect(ref)->isEmpty is translated into getRef(ref).asList().isEmpty(). This implementation is not efficient: getRef returns an Iterable that is filled from the database. If we put it in a list we force the database to load everything just to check if there is at least one element in the list.

Another approach is to dynamically add Iterable.metaClass.isEmpty to the init Groovy script, with an optimized implementation: !this.hasNext(). But Groovy dynamic dispatch is very slow, especially for closure-based methods.

For now I use the first solution, but I don't think it is the most interesting in the general case, need to to some experiments.

Query parameter tests

Add support for context and return type to parametrized queries

MogwaiQueryResult: isReifiable() says "true" when returning a list of strings

I've tried using a collect() query that produces a collection of strings: in this case, isReifiable() returns true and the program crashes when it tries to reify the strings.

Instead, a better option would be to check if the elements of the collection are all vertices and then set up the "isReifiable" flag. Also, isReifiable is always false when returning a single value, which feels odd. Some OCL queries could indeed return a single result...

Reifying a metaclass throws a NPE

The reification process doesn't handle metaclass reification for now. This is not critical but should be handled.

ModelElementMetaClass should extend ObjectMetaClass

Both provide the same implementation for identity step (_()), and a ModelElementMetaClass can be considered as a subclass of a generic ObjectMetaClass

TransformationHelper.booleanFrom should be used in all boolean expressions

This method is only used in not() translation, but it should be used to cast select() body, and every operator that requires a boolean value

Benchmark with different versions of Neo4j

See discussion with @bluezio. The version used seems to have an important impact on the query execution time.

No support for some OCL expressions

Intersection
Set Difference
Including
Excluding
Includes
Excludes

Are not supported for the moment

AllInstances(Class) translation does not take into account subclasses of Class

AllInstances(Class) operation is mapped to metaClassNode.inE("kyanosInstanceOf").outV, and does not take into account Class subclasses.

This could be fixed by reusing the logic in the oclIsKindOf(Class) translation.

Keep MogwaiResource#getBackend method public?

This is useful for development, but may be dangerous to expose publicly

How to transform ATL script to Gremlin?

Hi
I'm trying to generate the Gremlin script/query from an ATL transformation and then run the generated query. I do so as I don't want to persist the result of the transformation and for an java ATLQuery only java MogwaiResource.transform() and java MogwaiResource.process() are supported (at least, MogwaiResource.query() did not work).

I'm using the following code to transform the ATL transformation to Gremlin script:

private static void queryATL2Gremlin2(MogwaiResource mogwaiResource, ATLQuery query) {
	ATL2Gremlin atl2Gremlin = new ATL2Gremlin();

	Module atlModule = (Module) query.getATLResource().getContents().get(0);
	String sourceMMName = atlModule.getInModels().get(0).getMetamodel().getName();
	String targetMMName = atlModule.getOutModels().get(0).getMetamodel().getName();
	Resource gremlinResource = atl2Gremlin.transform(query.getATLResource(), sourceMMName, query.getSourcePackage(), targetMMName, query.getTargetPackage());
	GremlinScript script = ((GremlinScript) gremlinResource.getContents().get(0));
	System.out.println(script);
		
	GremlinQueryBuilder builder = GremlinQueryBuilder.newBuilder();
	GremlinQuery gQuery =  (GremlinQuery)builder.fromString(script.toString()).build();

	QueryResult result = mogwaiResource.query(gQuery);

	System.out.println(result);
}

Unfortunately, the following exception is raised:

Exception in thread "main" java.lang.RuntimeException: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: tHelper for class: Script2
	at fr.inria.atlanmod.mogwai.processor.GremlinScriptRunner.runGremlinScript(GremlinScriptRunner.java:133)
	at fr.inria.atlanmod.mogwai.processor.AbstractQueryProcessor.runGremlinScript(AbstractQueryProcessor.java:374)
	at fr.inria.atlanmod.mogwai.processor.AbstractQueryProcessor.process(AbstractQueryProcessor.java:199)
	at fr.inria.atlanmod.mogwai.processor.GremlinQueryProcessor.process(GremlinQueryProcessor.java:56)
	at fr.inria.atlanmod.mogwai.neoemf.processor.NeoEMFGremlinQueryProcessor.process(NeoEMFGremlinQueryProcessor.java:78)
	at fr.inria.atlanmod.mogwai.neoemf.processor.NeoEMFGremlinQueryProcessor.process(NeoEMFGremlinQueryProcessor.java:1)
	at fr.inria.atlanmod.mogwai.processor.GremlinQueryProcessor.process(GremlinQueryProcessor.java:1)
	at fr.inria.atlanmod.mogwai.query.MogwaiQuery.process(MogwaiQuery.java:99)
	at fr.inria.atlanmod.mogwai.query.MogwaiQuery.process(MogwaiQuery.java:74)
	at fr.inria.atlanmod.mogwai.neoemf.util.NeoEMFQueryHandler.query(NeoEMFQueryHandler.java:177)
	at fr.inria.atlanmod.mogwai.neoemf.resource.DefaultMogwaiResource.query(DefaultMogwaiResource.java:100)
	at fr.inria.atlanmod.mogwai.neoemf.resource.MogwaiResource.query(MogwaiResource.java:122)
	at run.Transformation.queryATL2Gremlin2(Transformation.java:190)
	at run.Transformation.main(Transformation.java:103)
Caused by: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: tHelper for class: Script2
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:333)
	at org.codehaus.groovy.jsr223.GroovyCompiledScript.eval(GroovyCompiledScript.java:41)
	at javax.script.CompiledScript.eval(CompiledScript.java:92)
	at fr.inria.atlanmod.mogwai.processor.GremlinScriptRunner.runGremlinScript(GremlinScriptRunner.java:121)
	... 13 more
Caused by: groovy.lang.MissingPropertyException: No such property: tHelper for class: Script2
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:51)
	at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:49)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:295)
	at Script2.run(Script2.groovy:1)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:330)
	... 16 more

Then, I tried to initialize Gremlin with other init script available in the project without success (similar problem) :

	private static void queryATL2Gremlin(MogwaiResource mogwaiResource, ATLQuery query, Map<String, Object> qOptions) {
		ATL2Gremlin atl2Gremlin = new ATL2Gremlin();
		atl2Gremlin.enableATLDebug();

		Module atlModule = (Module) query.getATLResource().getContents().get(0);
		String sourceMMName = atlModule.getInModels().get(0).getMetamodel().getName();
		String targetMMName = atlModule.getOutModels().get(0).getMetamodel().getName();
		Resource gremlinResource = atl2Gremlin.transform(query.getATLResource(), sourceMMName, query.getSourcePackage(), targetMMName, query.getTargetPackage());
		GremlinScript script = ((GremlinScript) gremlinResource.getContents().get(0));
		System.out.println(script);
		
		MogwaiQuery gremlinInit = GremlinQueryBuilder.newBuilder()
				.fromFile(new File("materials/init.gremlin"))
//				.bind(ModelDatastore.BINDING_NAME, mapping)
				.build();

		mogwaiResource.query(gremlinInit);
		System.out.println("Init done");

		MogwaiQuery gremlinQuery = GremlinQueryBuilder.newBuilder()
				.fromString(script.toString())
//				.bind(ModelDatastore.BINDING_NAME, mapping)
				.build();
		
		// Put bindings here if we need them
		QueryResult result = mogwaiResource.query(gremlinQuery);
	}

Maybe you can give some hints on how I could achieve that?
Thank you.

Do we need to keep Gremlin.defineStep instruction in Gremlin.init?

I am not sure these methods are still called during the execution: MetaClass interception dispatches them when they are performed on Model Elements, and the CustomGremlinGroovyPipeline defines few of them for Pipeline-level calles.

However, CustomGremlinGroovyPipeline doesn't define all of them, probably because I dnd't have time to do it, but I need to check I didn't omit them on purpose.

Error-prone CustomGremlinGroovyPipeline constructors

The constructors CustomGremlinGroovyPipeline() and CustomGremlinGroovyPipeline(Object starts) do not initialize the PipesDatastore to use to access the model. Calling model related methods defined at the pipeline level such as getAtt and getRef will throw a NPE.

Is there a valid reason to keep these constructors?

Merge multiple OCL2Gremlin Instances into a global one

If several queries are performed on the same model multiple instances of OCL2Gremlin are created, while this is useless.

Creating a single OCL2Gremlin instance would make more sense, and allow to have transformation caches that avoid multiple translation from the same OCL input (see #16).

Static type reference in init.gremlin

init.gremlin contains static references to attribute / element classes in order to customize the metaclasses. This should be modularized to use any ModelDatastore, regardless its element / attribute type.

init.gremlin should be generic and should not be updated by end-user.

panamaRel is not used anymore

This project is an old version of the relational metamodel for the panama papers application example.

ModelDatastore#getParent should return an Iterable

Default mapping (not based on EMF) may contain multiple "parents" for a model element. The default implementation of getParent() should return an Iterable.

Note: the old method returning a single element can be kept because ATL semantic for getParent() doesn't allow multiple parents. However this is required for simple usage of the mapping

Refactor core module into multiple plugins

1 plugin = 1 query type with its own builder, processor, query type ...

Simplify gremlin.init script

The script contains multiple commented lines that are not usefull anymore. In addition, we need to benchmark if the dynamic methods and steps are more efficient than the ones provided by the CustomGremlinGroovyPipeline.

We also need to be clear on what is defined at the Groovy level and what is defined at the Java level.

Add isSingleResult() to MogwaiQueryResult

When accepting external OCL queries, we might not know in advance if the query will return a single value or a collection. However, right now it is not possible to check if that was the case or not: calling the "wrong" method will result in a MogwaiException, and resultSize() might still return 1 if the OCL query happens to produce a collection with 1 element.

MogwaiQueryResult needs a method that tells the client if the result is a single value or not.

fr.inria.atlanmod.mogwai.applications.panama.[neo4j/relational] should be renamed to match the data-store

The packages should be named neo4j3 and jdbc, respectively.

Cache transformation results to avoid multiple transaltion from the same OCL input

If the same OCL query is given multiple times to the framework the translation is called each time. This process takes time and is not necessary if the input OCL query is exactly the same.

Cannot run Train Benchmark query with 3 levels of collect

I'm trying to run several queries that have multiply nested collect() calls. For instance, this simplified version of the Train Benchmark RouteSensor query:

import railway : 'platform:/resource/railway.neoemf/model/railway.ecore'

package railway
    context Route
    def:
    routeSensor : Bag(Sensor) = Route.allInstances()
        ->collect(route | route.follows
            ->collect(swP | swP.target
                ->collect(sw | sw.monitoredBy)
            )
        )
endpackage

However, this fails with the following error:

IN!<unnamed>
'swP'
Cannot set feature nextElement to value swP.target, inter-model references are forbidden. Configure launching options to allow them.
def metaRoute = g.getIndex("metaclasses",Vertex.class)[[name:"Route"]];
def metaRouteNode = metaRoute.next();
metaRouteNode.inE("kyanosInstanceOf").outV._().outE("follows").inV.([outE("target").inV] as  Set)._()._().outE("monitoredBy").inV._()._()._();

javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: com.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.outE() is applicable for argument types: (java.lang.String) values: [target]
Possible solutions: get(java.lang.String), put(java.lang.String, java.lang.Object), dump(), use([Ljava.lang.Object;), eval(java.lang.String), getAt(java.lang.String)
    at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:333)
    at org.codehaus.groovy.jsr223.GroovyCompiledScript.eval(GroovyCompiledScript.java:41)
    at javax.script.CompiledScript.eval(CompiledScript.java:92)
    at fr.inria.atlanmod.mogwai.core.Mogwai.runQuery(Mogwai.java:60)
    at fr.inria.atlanmod.mogwai.core.Mogwai.performQuery(Mogwai.java:29)
    at fr.inria.atlanmod.mogwai.resources.MogwaiResourceImpl.query(MogwaiResourceImpl.java:36)
    at fr.inria.atlanmod.mogwai.resources.MogwaiResourceImpl.query(MogwaiResourceImpl.java:31)
    at mogwai.remoteQuery.app.Prototype.run(Prototype.java:103)
    at mogwai.remoteQuery.app.Prototype.main(Prototype.java:74)

It seems as if swP.target wasn't being transformed for some reason. Any ideas on what might be going wrong? If I limit it to the two outermost collect() calls it works fine.

Support for List of Lists and Tuples?

For now Mogwaï is not able to return List of Lists or List of Tuples.
In fact the Tuple type does not have a mapping at all, and cannot be used inside the query.

Including Enhancement

Including translation could be improved:
It is currently translated as gather.transform{it << object}.scatter
The transform step is useless: the closure can be applied on the gather step

MogwaiConsolePage should extend/reuse OCLConsolePage methods

The current implementation contains a lot of duplicated code from OCLConsolePage, but does not extends it because of visibility issues.

There is maybe a way to limit code redundancy by wrapping an instance of the OCLConsolePage and add auxiliary methods for Mogwai-based computations

Non-Ecore EDataTypes and oclIsKindOf do not work

I have set up a small prototype that loads the GraBaTs'09 set0.xmi into NeoEMF and then runs a slightly more detailed query than the one in this project's benchmarks:

import DOM : 'platform:/resource/jdtast.neoemf/model/JDTAST.ecore'

package DOM

context TypeDeclaration

def: singletons : Set(TypeDeclaration) =

  TypeDeclaration.allInstances()
  ->select(td |
    td.bodyDeclarations
      ->exists(md |
        md.oclIsTypeOf(MethodDeclaration)
        -- this cannot be translated by Mogwai, apparently
        and md.modifiers->exists(mod | mod.oclIsKindOf(Modifier) and mod.oclAsType(Modifier).public)
        and md.modifiers->exists(mod | mod.oclIsKindOf(Modifier) and mod.oclAsType(Modifier)._static)
        and md.oclAsType(MethodDeclaration).returnType.oclIsTypeOf(SimpleType)
        -- and this doesn't work, either - fullyQualifiedName does not appear in the translated query
        and md.oclAsType(MethodDeclaration).returnType.oclAsType(SimpleType).name.fullyQualifiedName = td.name.fullyQualifiedName
      )
  )

endpackage

Sadly, it doesn't seem to be translated correctly: see (1), (2) and (3).

def metaTypeDeclaration = g.getIndex("metaclasses",Vertex.class)[[name:"TypeDeclaration"]];
def metaMethodDeclaration = g.getIndex("metaclasses",Vertex.class)[[name:"MethodDeclaration"]];
def metaModifier = g.getIndex("metaclasses",Vertex.class)[[name:"Modifier"]];
def metaSimpleType = g.getIndex("metaclasses",Vertex.class)[[name:"SimpleType"]];
def metaTypeDeclarationNode = metaTypeDeclaration.next();
def metaMethodDeclarationNode = metaMethodDeclaration.next();
def metaModifierNode = metaModifier.next();
def metaSimpleTypeNode = metaSimpleType.next();

metaTypeDeclarationNode.inE("kyanosInstanceOf").outV.filter{
  def td = it;
  td.outE("bodyDeclarations").inV.filter{
    def md = it;
    md.transform {
      it.outE("kyanosInstanceOf").inV.next() == metaMethodDeclarationNode;}.next()
      && md.outE("modifiers").inV.filter{
        def mod = it; mod && mod.; -- (1)
      }.hasNext()
     && md.outE("modifiers").inV.filter{
       def mod = it;mod && mod.; -- (2)
     }.hasNext()
     && md.outE("returnType").inV.transform{
      it.outE("kyanosInstanceOf").inV.next() == metaSimpleTypeNode;
    }.next()
    && md.outE("returnType").inV.outE("name").inV == td.outE("name").inV; -- (3)
}.hasNext();
};

It feels as if the oclAsType(...) calls (required by the OCL parser) were preventing the translator from working correctly. Do you think this would be the case?

CustomGremlinGroovyPipeline#getString throws an IllegalStateException when the pipeline contains more than one element

I am not sure this restriction is actually needed. A getString method on a pipeline containing multiple elements could return a generic String built by appending the elements one after the other.

We need to be clear on this method: is it only a debugging method that helps to understand what is in the pipeline, or is it also used in the translations (for example of toString() calls in ATL)? In the later we have to be careful, because String comparisons can have unexpected behaviors if we use a generic pattern for multiple elements.

ATL2Gremlin and OCL2Gremlin launchers should be merged

Both classes perform the same job: launching an ATL transformation. We could at least create an abstract class that contains the common behavior of these two classes.

It could be useful in other projects that needs to launch ATL transformation.

MogwaiATLQuery should be independant of the backend

The transformation is the same regardless the models involved. There is no need to have them inside the query itself.

Remove ObjectMetaClass constructor parameters and construct it using super(Object.class)?

This could simplify the instantiation of the class, but also create important execution time overhead, because this metaclass would be used for any access objects, including Pipes, Pipelines, and Model elements.

We need to benchmark this before any modification.

QueryProvider / QueryContext ?

Need to add math operations

Only '+' is supported for now, we need to add '-', '*' ...