devonfw/cobigen

Code-based Incremental Generator

devonfw default community health files

devonfw is a standard development platform and consists of many repositories. This repository is for default community health files of this entire devonfw github organization.

cobigen's Issues

Add equals and hash methods to IncrementTo

For better supporting the user of the programming API, we should insert equals and hash methods to the transport objects. Please also check Tos other than IncrementTo.

One use case for IncrementTo might be a contains with the increments ID on the list of matching increments.

_Tasks:_

• Implement hash(), equals() and toString() methods for IncrementTo
• Implement hash(), equals() and toString() methods for TemplateTo

At least for TemplateTo you have to rework the equals method as it currently only compares the id, but it should compare all attributes.

complete cobigen-javaplugin release v1.1.0

Tasks

• Final Review
  • Check for changed dependencies and document them if necessary (especially if cobigen-core dependendency has been changed)
    • as dependencies has been changed, check new dependencies with legal and create new licensing document for new version
  • Check if all tests are running green & there are no ignored ones left (as the case check them)
  • Check maven build
• Merge to master branch
• Create documentation
  • Possibly update latest versions of components in the introduction (not yet listed)
• Deployment
  • Deploy to OASP Nexus
  • Deploy to network drive
  • Close Milestone and deploy to new GitHub Release

CI Integration of cobigen

It might be interesting to provide a maven and/or gradle integration of cobigen.
Possible use cases are generation of testdata-builder or re-generation of equals/hashcode methods during build.

Create templates for Oasp ETO/CTO/Dao/Entities

We should provide templates for generating Eto = Entity transfer objects / Cto = Composite transfer objects / Dao = Data access objects / Entities from their common interface. Take oasp/oasp4j-sample as orientation. The common interfaces can be found in the component package in common.api (see coding-conventions).

Related to oasp/oasp4j#7

Allow at least ?replace operation in templates configurations

In the templates configuration it is allowed to use FreeMarker like functionalities like ?cap_first on variables. Unfortunately it is not easily possible to use the FreeMarker engine on strings such that we can use the same engine here as in the templates.

Therefore we should keep the custom reimplementation of the variable operations and enrich them by three additional operations:

1. ?replace(String regex, String replacement)
2. ?removeSuffix(String suffix)
3. ?removePraefix(String praefix)

ADDITIONALLY: It should be possible to concatenate the operations, which is currently also not possible. This need arises as we are not able to assign any internal template variables here.

Testdata builder for non persistable data

Currently, we have one template set for persistable testdata builder. Nevertheless, there might be Bo or To objects as well, which should be generated for testing puproses. As these data do not have to be persisted, we should also provide a second template increment for testdata builder without the dependency to an entityManager.
Problem is, that the entityManager needs to be loaded in the running test suite and therefore cannot be easily ignored. Currently, it has be removed by hand after generation.

Improve programming API to prevent faulty usage

Using the programming API the user might call getMatchingIncrements or getMatchingTemplates to get the templates he wants to generate. To retrieve all possible templates, the user has to provide his input to these methods.
After that, when calling the generate method to trigger the generation for a template, the user has to pass the input again. If he passes an invalid input for the template due to an programming error, he might be confronted with UnknownContextVariableExceptions and others, which will take him some time to analyse his failure.

The easiest way to make the API more intuitive is to enrich the API by an internal state, which holds an input, which can be set via only one method.

<containerMatcher> declaration does not work properly

The <containerMatcher ...> currently does not work properly.
I changed the input for retrieving the matching triggers and correct templates, but then the variables will not be resolved. So what is left is, that the variables will be resolved by matching each derived input with a matcher.

Related Tasks:

• #43 Adapt documentation
• #44 (this)
• #46 Fix variable resolving

complete cobigen-eclipse release v1.1.0

Tasks:

• Integrate cobigen-core v1.1.0
• Integrate cobigen-javaplugin v1.1.1
• Integrate cobigen-xmlplugin v1.0.1

Release tasks:
_1. Preparation_

• Check for changed dependencies and document them if necessary (especially if internal cobigen dependendencies have been changed). As dependencies have been changed:
  • check new dependencies with legal and create new licensing document for new version
  • document the changes in the ChangeLog of the dependency tracking.
• Check running maven build
• Check if all tests are green and if there are no ignored ones left. As there are ignored ones, please check them if they can be removed or they only should be temporarily ignored. In the latter case fix them!
• Deploy snapshot build to OASP Nexus

_2. Merging_

• Merge development branch to master branch
  • Perform final Review of merged contents
    • if ok - push
    • if not - abort merge and fix on dev branch
• higher version number to release version

_3. Testing / Consolidation_

• Perform integration tests
  • especially for cobigen-eclipse if cobigen internal dependencies have been changed
• Check/Update documentation according to changelog to be released

_4. Deployment_

• Create a tag according to the naming conventions
• Deploy release build to OASP Nexus
• Deploy release build to network drive
• Close milestone and create new release on GitHub

_5. Follow-up_

• Merge master branch back to development branch which previously had been merged into master

Create generic xml input reader

We should create a generic xml input reader, which traverses the input xml and creates a similar object model out of it.
Example:

<root>
  <test attr="abc"/>
</root>

The attribute attr should be accessible in the object model using ${root.test.@attr} and also by

<#list root.testList as test>
${test.@attr}
</#list>

The first "simple" variable access should be only available if the document declares only one <test> node as in the example. But you might also imagine to have a document like this:

<root>
  <test attr="abc"/>
  <test attr="efg"/>
</root>

As we do not know the schema definition of the input xml, we do not know whether we should provide a list of nodes or a single variable path for the node access. So in general we should provide a list for every node besides the root node to be as generic as possible.

This issue has been motivated by oasp/oasp4j#77

Tasks:

• refactor tests to match latest test distinction pattern equivalently as on dev_javaplugin
• Implement model transformation nodes -> template
• implement attribute retrieval using *.@attr syntax
• implement PCDATA retrieval using *.# syntax
• add documentation in the wiki
• update changed dependencies

Documentation of new context configuration and ui changes

The Word user and developer documentation is outdated, as so much changed since last release.
Due to the migration to GitHub we should provide the new documentation right within the GitHub Wiki.

Issues to be documented:

• Template-Development
  • Java object model for FreeMarker
  • Hints for developing FreeMarker templates / links to the FreeMarker reference of build-ins and other interesting stuff.
• Generator-Usage
  • currently possible inputs
  • generation with one input
  • generation in batch mode (multiple inputs)
• Generator-Configuration
  • configuration folder structure
  • templates configuration
  • context configuration
  • logging

Document the currently implemented generic use cases of CobiGen

We should provide an additional chapter in the documentation, which focuses on the currently implemented use cases CobiGen has been used for. Especially, we should focus on use cases, which might be interesting for several projects like the testdata builder generation.
Some (non-exclusive) topics to be documented are:

• Quick start guide: might also include references to different other chapter in the documentation with a focus on the CobiGen eclipse plug-in user's needs

Use cases (generic):

• Testdata builder generation for RF & OASP
• CRUD generation for RF & OASP
• Hash- / Equals

Use cases (specific):

• Testdocumentation generation

Add super types to object model

For inherited entity structures as input, we might need to be able to recover this type hierarchy in the generated sources. The generation of Eto's from Entities as intendet by #25 states such a case.
I think we should differ the inherited from the implemented super types in the model. I would guess the following model structure for this adaption:

• pojo
  • extendedType
    • name
    • canonicalName
    • package
  • implementedTypes (List)
    • type
      • name
      • canonicalName
      • package

Implement fillMandatoryFields in testdata builder templates

Currently, the testdata builder are generated without any implementation in the fillMandatoryFields method. This method should create mandatory dependencies as well as fill default values into mandatory fields of an entity.

Filling / Creating all mandatory dependencies of an entity by default is the main perfomance driver when using testdata builder, so we should try to come up with a full functioning solution here.

The java input reader should not retrieve all java files within a package recursively

The java input reader should not retrieve all java files within a package recursively as this does not fit to the container matcher semantics for java triggers.
We should only consider simple file retrieval for now.
In general, I think there is currently no interesting use case as well as no easy way to implement the concept of recursive vs. non-recusive object retrieval for containers configured via the context configuration. Thus we simply should remove this in the java plugin.

Let plug-ins provide specialized FreeMarker template methods

We should enrich the model builder interface to enable plug-ins to enrich the FreeMarker model bei specialized FreeMarker methods. This is necessary to support java specific template methods like instanceOf and isAbstract. See #40 for more information.

Enable automatic plug-in discovery using java service loaders

The corn-library for finding specific classes on the classpath seams to have a strange dependency to com.sun:tools:1.5 which is not fulfilled at least on my system with a standard Oracle 1.7/1.8 JDK under Mac OSX 10.9.

Tasks:

• Remove the corn-library / google reflections
• Evaluate Java provided ServiceLoader API (http://docs.oracle.com/javase/tutorial/ext/basics/spi.html) to cope with this goal.

context variables not resolved for <containerMatcher> matching

When matching a container matcher, e.g., a Java package, the variable resoultion should be done on the objects the container provides. Currently, you will get a UnknownVariableException if you want to access variables matched by a matcher as the variables are not resolved correctly.

Improvement for entities in testdata builder templates

When persisting an entity, the generated code should pass back the instance of the actual object. Callers may need to access the object afterwards (e.g. access id field).

Merge of namespaces not working correctly

When merging two XML-files where one XML-file includes namespaces that the other file does not use, the merging throws away the schemaLocation information. This also means that the resulting XML-file is no longer valid since the schemas are no longer defined (the definition of a schema consists of the namespace definition and the schemaLocation information).

Improve Errorhandling for XML-Merge

If you try to merge XML documents, which do not directly start with the document type declaration, you will get the strange error message "The processing instruction target matching "[xX][mM][lL]" is not allowed". This is not a bug, but a valid error message from the XML processing tooling:
http://www.bennadel.com/blog/58-the-processing-instruction-target-matching-xx-mm-ll-is-not-allowed.htm

Nevertheless, the error message is quite unhandy. Check if it is possible to provide a better error message to guide the user of CobiGen. Write also a test checking the new error handling.

Reproduction (for test issues):

1. Create an xml file, which starts with an empty line and then with the xml language declaration <?xml version="1.0" encoding="UTF-8" ?>.
2. Now create a template you want to merge into the file.
3. Start generation of the declared template, such that it will be merged into the previously created file.

--> Now you should get this strange Error

Integrate CobiGen context menu also within ProjectExplorer

Some of our users are quite confused to get no feedback after CobiGen installation.
Currently we only provide an entry point in the context menu of the jdt PackageExplorer. But it might be a good choice to at least also support the ProjectExplorer as some of the users preferably use this as default.

Not all specified templates are generated

I observed an incomplete generation preview some times.

Steps to reproduce:

1. Use the cobigen-templates folder as configuration folder
2. Select an persistence entity (e.g. Table) as input and start CobiGen
3. Select increment "DAO's" on the left side

--> The following files will be available for selection:

• NamedQueries.java
• AbstractDomainDao.java
• DomainDao.java
• NamedQueries.hbm.xml
• jpa.xml

but due to the templates.xml the following files are missing:

• TableDao.java
• TableDaoImpl.java

Provide random data generator for testdata generator

When creating an object for testing issues, sometimes random data is needed. Therefore creating random testdata for (at least primitive) Attributes could be integrated into the generator (e.g. String: random alphanumeric)

Furthermore, when creating the builder from entities, it may also be possible to extract the JPA Metadata (field length, etc.). On this way the random data generator could produce more consistent data.

Property not defined while generating DAOs

@subashbasnet mentioned:

1.The listing of the Dao and DaoImpl files show up the in right-pane. The dao and daoImpl files to be generated are listed in the templates.xml. They show up in the list.

1. But upon clicking generation I've shown the below error occurs, and file generation stops abruptly.
   
2. Despite of the error following files are generated which are visible only after refreshing the respective project.
   NamedQueries.java
   AbstractDomainDao.java
   DomainDao.java
   NamedQueries.hbm.xml
   jpa.xml

But the below two files don't appear upon generation even if they appear in the list, may be because of abrupt ending of generation caused by the error.
TableDao.java
TableDaoImpl.java

Update licensing agreement

The license agreement has to be updated to

• new modularized structure of CobiGen
• the new name "CobiGen" (former APPS-Generator)
• updated and possibly new library dependencies
• adapt APPS-Generator license agreement according to license subject
• create one license for each product of the released versions
• check for english version of license agreement

API: Generate Increment

It would be nice to have an API to trigger generation for a whole increment. Otherwise the user has to interate over all templates by himself (which might be a good option, but possibly is not the standard use case).

Insert version number for configuration files

In order to be able to provide further automatic configuration upgrades, we should add a version number for all configuration files.
When upgrading the APPS-Generator to CobiGen itself, the configuration has to be upgraded once by hand.

Create hello world example

Might be interesting to have a "Hello World" example to try CobiGen.
Possibly based on the restaurant example application.

Create Apache Maven plugin

In my opinion, a very important point is the integration of the cobigen workflow into the build process of an application. Especially for the testdata generator this would be a nice feature. On this way the builders always get generated and changes in the datamodel are directly reflected (comparable to JPA metmamodel generation) inside the IDE (compilation errors, ...).

Therefore it would be nice to get a maven plugin for calling the cobigen generator.

Organise imports and format sourcecode is not triggered for batch generation

If generating from a single source all imports will be oranised and the source code will be formatted. But when running a batch generation for multiple inputs, it seems that organise imports as well as format source code is omitted.

Adapt documentation for <containerMatcher>

The context configuration allows to define <containerMatcher ...> nodes beyond the <trigger ...> node to support packages as inputs. This is currently omitted in the documentation.

Make CobiGen configuration folder configurable

It would be nice to have the ability to configure the current configuration folder for CobiGen. Thus it might be possible that the configuration folder is not even imported as a project into the currently running eclipse. It might be provided by default via SCM.

But this accompanies with a good error handling and user support to guide the user when no folder has been configured or the folder does not exist any more!

Use generics in testdata builder templates for collections

When genereting builders for dataobjects with collections, the generated java file has only setters for the raw types of the respective collections. This suppresses compile time failure detection, when adding items to the collections.

Example Current:
Attribute: private testlist List => Setter: testlist(List testlist)

Example Improvement:
Attribute: private testlist List => Setter: testlist(List testlist)

Upgrade Java Parser QDox to 2.0-M2

Upgrade Java Parser QDox to 2.0-M2 to gain from multiple Bugfixes and full Java 7 support.

Provide generic trigger

cobigen-core should provide a simple default trigger type 'generic', which will be triggered every time.
Thus is will be possible to generate infrastructure code which does not depent on any input.

Strange generation results for different source folders

Case: File to be generated not listed in the generation-pane
1st case:
In place of name of source folder as 'src/main/java' if I use 'src', 'main', 'src/java','main/src' i.e. anything besides the above mentioned path it works fine.

2nd case:
If the source folder is 'src' and package name begins with 'main' then the generation doesn't work,
but in every other cases it works fine.

Smart Util for QDox API

Would be nice if the java plugin can provide util functionalities, e.g., for getting the main type of a java class. The main type here means the type which has the same name as the source file of the type.

Type checks not functional any more (since first CobiGen release)

Due to the migration of the APPS-Generator's single application design to a more modularized architecture, the type checking template methods of the java plug-in have been destroyed.

The problem is, that the whole model will be converted to a DOM model to enable xPath expressions on the model within the template. As of the modularization, the type checking template methods are now part of the model itself and not part of the global variables of FreeMarker. Therefore, the needed ability to integrate a BeanModel for the util implementation in a generic way has been eliminated.

Thus we have to find a new approach to handle this. My current proposal would be to add another method for InputReaders to be called beside the createModel() method. This method then might provide multiple function extensions for the template. I would also suggest to use the right extension point of FreeMarker for functions.

• #45 Implement cobigen-core extension interface
• implement extension interface for the two methods
  • instanceOf instead of the previously called isSubtypeOf
  • isAbstract
• update documentation

Resources are generated which are not selected

If you select a resource or an increment, such that some resources in the file preview will be selected, and you unselect them again, you can generate resources although you have non selected.
Furthermore all previously selected resouces will be generated, no matter if they have been unselected again.

Integrate possibility of custom string mappings

There is one low priority use case we currently cannot cover:

An application might have a component like TableManagement. Due to naming conventions the component name will be integrated in the package structure written in lower case.
Information like this can again be retrieved by our context configuration, e.g., using regular expressions on the full qualified names of java classes. BUT: This only will give us the component name written in lower case.

The problem will now occur if we want to create a file, which contains the original component name in its name such like TableManagement.java. We cannot recover the original camel-case!

For such issues it might be interessting for the future to have the ability to declare custom string mappings, which can then be retrieved in the templates configuration and also in the templates itself.
Such mappings might be defined by a simple properties files within a templates folder. The syntax for mapping resolution might look like ${variables.component?mappedValue}.

CobiGen does not merge the Java extends relation

Inheritance relations will not be merged. Example:

Inputs:

• base class: does not declare an inheritance relation (implicitly inheriting java.lang.Object)
• patch class: declares an inheritance relation different from java.lang.Object

Result: The inheritance relation specified in the patch will not be merged into the base class.

Not all possible templates are previewed and generated

On different machines, the preview of templates is inconsistent. In comparison a different set of templates will be shown, but not all at one time.

Perhaps that is an issue with the e4 architecture? Perhaps check with eclipse 3.7 if this propblem persists. If not, we have to focus on the e4 migration.

Create ability to append classloader for PackageFolder inputs

Due to issue #44 we need the ability to append a ClassLoader to the PackageFolder input as packages have no Java native object to be represented.
The ClassLoader is needed to parse the children of the package in the context of a customized ClassLoader to enable correct type resolution.

eclipse 3.x compatibility

@subashbasnet mentioned:

On eclipse 3.7(Indigo) the project cannot be compiled.
Tried to compile by commenting out method accept() in class IResourceStub, the method a override of the eclipse properties of later version.
Upon maven build the following error occurs:
The type IResourceStub must implement the inherited abstract method IResource.accept(IResourceProxyVisitor, int, int)

@Generated annotation for generated Java code

For Java code, we should add the @generated annotation to each method or class, which have been generated by CobiGen.
In case of a re-generation of the contents (classes / methods), CobiGen should interprete an existent @generated annotation as an indicator for normal configured behavior (merge/override defined in the templates.xml).
Methods and classes which @generated annotation has been removed by the programmer should not be touched by CobiGen any more, although the templates configuration might define something different.

Faulty configuration not cause errors but read the configuration partially

Currently faulty configurations (templates / context) will not cause an error as long as there is the correct root element specified files.
For usability, it would be better, if the files are not read partially which causes strange behavior for the user, but better throw an error an provide the user hints to correct the configuration.

User interface for configuration version handling

Related to #8:

Additionally to adding the version tag in the root of the configuration files, we should provide a checking mechanism, when CobiGen tries to read the configuration.
In this case the user should be informed about an incompatible version of configuration for now. If no version number is set, the configuration is also considered to be not compatible!
Furthermore, we should introduce a nice mechanism for migrating configuration files in future, e.g. by xslt transformation, such that we can define one xslt transformation for each version upgrade (if necessary).

Definition of target path per generation

In case of the generation of small non architectural-bound artifacts like testdata builders for example, it does not make sense to define the destination path beside each template.
There might be the necessity for different builders (entity builder, To builder, Bo builder...), which then will have the same target generation location. Therefore it should be possible to leave out the destination paths for the templates and prompt the user to define a destination before generating.

This might have impact on the generation preview wizard, as the destination paths are not previously known.

Consolidation of documentation to match oasp/doc-gen conventions

Jörg extracted the documentation generation of OASP to a separate reusable project: https://github.com/oasp/oasp-docgen

We should consolidate the current documentation to match the conventions of this project to gain from its features. Furthermore, we should remove the customized tools-cobigen project if possible.

One interesting point might be the license disclaimer in the generated document, which will differ for CobiGen from the OASP ones.

@hohwille: is it currently possible to configure the disclaimer in the footer?