pyecore/setuptools-pyecore

A setuptools command for generating Python classes from Ecore models

PyEcore: A Pythonic Implementation of the Eclipse Modeling Framework

PyEcore is a Model Driven Engineering (MDE) framework written for Python. Precisely, it is an implementation of EMF/Ecore for Python, and it tries to give an API which is compatible with the original EMF Java implementation.

PyEcore allows you to handle models and metamodels (structured data model), and gives the key you need for building MDE-based tools and other applications based on a structured data model. It supports out-of-the-box:

• Data inheritance,
• Two-ways relationship management (opposite references),
• XMI (de)serialization,
• JSON (de)serialization,
• Notification system,
• Reflexive API...

Let's see how we can create a very simple "dynamic" metamodel (as opposed to static ones, see the documentation for more details):

>>> from pyecore.ecore import EClass, EAttribute, EString, EObject
>>> Graph = EClass('Graph')  # We create a 'Graph' concept
>>> Node = EClass('Node')  # We create a 'Node' concept
>>>
>>> # We add a "name" attribute to the Graph concept
>>> Graph.eStructuralFeatures.append(EAttribute('name', EString,
                                                default_value='new_name'))
>>> # And one on the 'Node' concept
>>> Node.eStructuralFeatures.append(EAttribute('name', EString))
>>>
>>> # We now introduce a containment relation between Graph and Node
>>> contains_nodes = EReference('nodes', Node, upper=-1, containment=True)
>>> Graph.eStructuralFeatures.append(contains_nodes)
>>> # We add an opposite relation between Graph and Node
>>> Node.eStructuralFeatures.append(EReference('owned_by', Graph, eOpposite=contains_nodes))

With this code, we have defined two concepts: Graph and Node. Both have a name, and there exists a containment relationship between them. This relation is bi-directional, which means that each time a Node object is added to the nodes relationship of a Graph, the owned_by relation of the Node is also updated (it also works the other way around).

Let's create some instances of our freshly created metamodel:

>>> # We create a Graph
>>> g1 = Graph(name='Graph 1')
>>> g1
<pyecore.ecore.Graph at 0x7f0055554dd8>
>>>
>>> # And two node instances
>>> n1 = Node(name='Node 1')
>>> n2 = Node(name='Node 2')
>>> n1, n2
(<pyecore.ecore.Node at 0x7f0055550588>,
 <pyecore.ecore.Node at 0x7f00555502b0>)
>>>
>>> # We add them to the Graph
>>> g1.nodes.extend([n1, n2])
>>> g1.nodes
EOrderedSet([<pyecore.ecore.Node object at 0x7f0055550588>,
             <pyecore.ecore.Node object at 0x7f00555502b0>])
>>>
>>> # bi-directional references are updated
>>> n1.owned_by
<pyecore.ecore.Graph at 0x7f0055554dd8>

This example gives a quick overview of some of the features you get for free when using PyEcore.

The project slowly grows and it still requires more love.

Installation

PyEcore is available on pypi, you can simply install it using pip:

$ pip install pyecore

The installation can also be performed manually (better in a virtualenv):

$ python setup.py install

Documentation

You can read the documentation at this address:

https://pyecore.readthedocs.io/en/latest/

Dependencies

The dependencies required by pyecore are:

• ordered-set which is used for the ordered and unique collections expressed in the metamodel,
• lxml which is used for the XMI parsing.

These dependencies are directly installed if you choose to use pip.

Run the Tests

The tests use py.test and 'coverage'. Everything is driven by Tox, so in order to run the tests simply run:

$ tox

Liberty Regarding the Java EMF Implementation

• There is some meta-property that could be missing inside PyEcore. If you see one missing, please open a new ticket!
• Proxies are not "removed" once resolved as in the the Java version, instead they act as transparent proxies and redirect all calls to the 'proxied' object.
• PyEcore is able to automatically load some model/metamodel dependencies on its own.

State

In the current state, the project implements:

• the dynamic/static metamodel definitions,
• reflexive API,
• inheritance,
• enumerations,
• abstract metaclasses,
• runtime typechecking,
• attribute/reference creations,
• collections (attribute/references with upper bound set to -1),
• reference eopposite,
• containment reference,
• introspection,
• select/reject on collections,
• Eclipse XMI import (partially, only single root models),
• Eclipse XMI export (partially, only single root models),
• simple notification/Event system,
• EOperations support,
• code generator for the static part,
• EMF proxies (first version),
• object deletion (first version),
• EMF commands (first version),
• EMF basic command stack,
• EMF very basic Editing Domain,
• JSON import (simple JSON format),
• JSON export (simple JSON format),
• introduce behavior @runtime,
• resources auto-load for some cross-references,
• derived collections,
• multiple roots resources,
• xsi:schemaLocation support for XMI resources,
• URI mapper like,
• EGeneric support (first simple version),
• URI converter like

The things that are in the roadmap:

• new implementation of EOrderedSet, EList, ESet and EBag,
• new implementation of EStringToStringMapEntry and EFeatureMapEntry,
• improve documentation,
• copy/paste (?).

Existing Projects

There aren't too many projects proposing to handle models and metamodels in Python. The only projects I found are:

• PyEMOF (http://www.lifl.fr/~marvie/software/pyemof.html)
• EMF4CPP (https://github.com/catedrasaes-umu/emf4cpp)
• PyEMOFUC (http://www.istr.unican.es/pyemofuc/index_En.html)

PyEMOF proposes an implementation of the OMG's EMOF in Python. The project targets Python2, only supports Class/Primitive Types (no Enumeration), XMI import/export and does not provide a reflexion layer. The project didn't move since 2005.

EMF4CPP proposes a C++ implementation of EMF. This implementation also introduces Python scripts to call the generated C++ code from a Python environment. It seems that the EMF4CPP does not provide a reflexive layer either.

PyEMOFUC proposes, like PyEMOF, a pure Python implementation of the OMG's EMOF. If we stick to a kind of EMF terminology, PyEMOFUC only supports dynamic metamodels and seems to provide a reflexive layer. The project does not appear to have moved since a while.

Contributors

Thanks for making PyEcore better!

• Mike Pagel (@moltob), who is also the author of pyecoregen and pymultigen (on which pyecoregen is based)
• Terry Kingston (@TerryKingston)
• Afonso Pinto (@afonsobspinto)
• Andy (@CFAndy)
• annighoefer (@annighoefer)
• Rodriguez Facundo (@rodriguez-facundo)
• Filippo Ledda (@filippometacell)
• Ewoud Werkman (@ewoudwerkman)
• Eugene (@4ekin)
• Alexandre Acebedo (@aacebedo)
• Jinhu (@jinhu)
• Pablo Campillo (@pablo-campillo)
• Jose Antonio Moral (@jamoralp)
• Marco Heinemann (@ubmarco)
• Christian Drabek (@cdrabek)
• Dan Yeaw (@danyeaw)
• Arkadiusz Michał Ryś (@arrys)

Additional Resources

• This article on the blog of Professor Jordi Cabot gives more information and implementation details about PyEcore.

setuptools-pyecore's Issues

Add logging output

The user isn't informed which model is generated by pyecoregen. If several models were found by setuptools-pyecore or the user specified more than one model the user doesn't know about the model which is currently generated. This gets more problematic if the generation fails for one of the found models but the user don't know which model is the faulty one.

Before the model generation is triggered the distutils logging framework should be used to print a user friendly generation message.

library model contains invalid content

The content of the library.ecore model is invalid. The content consists of html data.

Add Python 3.7 support

Add Python 3.7 support as soon as it is supported by Travis CI.
See the upstream issue travis-ci/travis-ci#9815

Looks like Travis CI won't support Python 3.7 in default images anymore. The official workaround is to switch to xenial and enable sudo like described here.

Add verbosity user option

A user option to configure the verbosity of the PyEcoreGenerator should also be provided in setuptools-pyecore. There exist three levels:

• warning
• info
• debug

The user should be able to pass this levels in a setup.cfg and on the command line.

Code generation overwrites static sample code

If the library.ecore model part of the sample project is generated it overwrites the checked in __init__.py file. The library metamodel should be generated into model folder of the project.

Refactor default output directory

If the user doesn't pass a default or model specific output path the package source directory will be used as a fallback. This behavior should be changed for two reasons:

• In case multiple equally named Ecore models are generated the output path isn't unique. pyecoregen will overwrite the generated code
• Intuitively the user will expect that the generated code is placed in parallel to the Ecore XMI file and not in the root folder of the package.

Clean up of resource throws exception in case resource couldn't be loaded

The _load_ecore_model tries to remove a Resource in every case. Also if a model couldn't be loaded (e.g. because of an invalid content) the removal of the Resource from the ResourceSet will be triggered. This results in the following exception:

Traceback (most recent call last):
  File "setup.py", line 22, in <module>
    long_description=open('README.md').read(),
  File "<path>\venv\lib\site-packages\setuptools\__init__.py", line 140, in setup
    return distutils.core.setup(**attrs)
  File "C:\DevTools\Python36\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\DevTools\Python36\lib\distutils\dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "C:\DevTools\Python36\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "<path>\.eggs\setuptools_pyecore-0.1.0-py3.6.egg\setuptools_pyecore\command.py", line 131, in run
    with self._load_ecore_model(ecore_xmi_file) as resource:
  File "C:\DevTools\Python36\lib\contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "<path>\.eggs\setuptools_pyecore-0.1.0-py3.6.egg\setuptools_pyecore\command.py", line 117, in _load_ecore_model
    rset.remove_resource(resource)
UnboundLocalError: local variable 'resource' referenced before assignment

Generate code from a Ecore model

In this ticket the core functionality of setuptools-pyecore should be implemented. In the run method of the PyEcoreCommand a new PyEcoreGenerator should be instantiated. User options should be passed to this instance and the Python Code of the Ecore model should be generated.

Redesign user interface

The current user interface isn't as intuitive as it should be to easily configure the code generation of multiple Ecore models. This is mainly because of the limited feature set provided by setuptools and distutils to specify config parameters (see config file doc).

Also the current user interface lacks the possibility to specify several models with equal names.

To overcome this limitations and simply the configuration the current user interface should be redesigned. The configuration should be split up in a basic configuration for default values (e.g. an output folder) and a model-specific configuration.
Basic configuration should be possible on the command line and in a config file. Model-specific configuration (e.g. specific output-folders, user-models, ...) should be limited to config files in the first place.To be as close as possible and compatible to the - ini like - configuration language used by setuptools it should be still possible to use the standard python config parser.

A very basic configuration file should look like this:

# default configuration
[pyecore]
output=gen
auto-register-package=True

# model specific configuration
[pyecore:library]
ecore-model=/path/to/library.ecore
output=gen/library
auto-register-package=False

Model specific configurations should override the default configuration. The second part of the section name (e.g. [pyecore:library] ) can be freely chosen by the user and isn't bound anymore to the root package of the Ecore model.

Refactor project structure

Move library sample from example to samples/library folder

Required dependencies aren't installed

During installation of setuptools-pyecore the following required dependencies aren't installed:

• pyecoregen
• pyecore

Improve documentation

Actually the documentation of setuptools-pyecore focuses on integration aspects and usage without giving the user an idea of the behavior. This aspect should be described in more detail.
Additionally a table of content should be added at the top of the project to simplify navigating in the documentation.

Sanity check for user input

The user input passed on the command line or in config file should be validated by setuptools-pyecore. A sanity check for the following parameters should be added:

• user-modules
• output

If the path to a output directory or to a user module isn't valid (e.g. doesn't exist) a warning should be thrown. For an user-module the code should still be generated without passing the user-module path to the EcoreGenerator. For an invalid output path the default output folder should be used.

Deploy setuptools-pyecore to PyPI and GitHub releases

The deployment of setuptools-pyecore should be automated. The package should be deployed to the following web services:

• GitHub Releases
• PyPI

For GitHub Releases the organization account of pyecore should be used. Only a binary distribution (wheel package) should be deployed.

For PyPi the personal account of ferraith should be used. A source and a binary distribution of setuptools-pyecore should be deployed.

User modules doesn't have a regular path

setuptools-pyecore handles user models as regular files with a valid windows or posix paths. This assumption is wrong. From pyecoregen doc:

If specified, the given string is interpreted as a dotted Python module path. E.g. --user-module my.custom_mod will make the generated code import mixin classes from a module my.custom_mod.

To fix this problem user modules passed on the cli or defined in a setup.cfg file shouldn't be handled anymore as paths. A simple string is sufficient.

Migrate build-jobs to travis-ci.com

From a build perspective setuptools-pyecore is a legacy project which is still hosted on travis-ci.org. The guys from Travis CI are currently replacing the legacy travis-ci.org platform with the new travis-ci.com platform (see Open Source on travis-ci.com). In Q3 also legacy projects will be migrated to travis-ci.com. In this ticket the migration should be tracked and necessary configuration changes (e.g. check and status api) should be applied to this project.