Channel API cant handle multiple versions of same artifact

[wildfly-extras]

Channel API cant handle multiple versions of same artifact

[TomasHofman]

I want to comment on the maven plugin point of view (as oppose to the extension).

Currently, executing the plugin would likely lead to this same issue. The plugin now works in a way that all project submodules are processed (their dependencies upgraded), but that's mostly because I wasn't sure what is the most suitable way the plugin should work. This shows that perhaps more reasonable default mode of operation would be that only the (execution) root module is processed, as @jfdenise suggested. The plugin could also make this configurable.

In general I agree that the plugin should provide enough flexibility to workaround such issues. I think we will also need an ability to exclude particular streams from being applied, avoid upgrading dependencies to the project submodules, etc. It very much depends on particular use case.

[jmesnil]

I'm not sure to understand what the issue is.
The Channel API is not meant to fetch artifacts used by tests, only those provisioned by WildFly.

Afaics, WildFly only requires a single version of Hibernate: https://github.com/wildfly/wildfly/blob/f8f05aede00ff12fad9a303f9b905f4ea90051eb/pom.xml#L433

[wolfc]

I think multi-version support for Hibernate came to an end with https://issues.redhat.com/browse/WFLY-15607

We may see a need for multi-version support in EAP 8.

Prior to this you would see the same as we would see on eap-7.4.x:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1-jboss-1:testCompile (default-testCompile) on project wildfly-ormtransformer: Compilation failure: Compilation failure: 
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/BitSetType.java:[38,8] cannot access javax.persistence.PersistenceException
[ERROR]   class file for javax.persistence.PersistenceException not found
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/ProcedureParameterExtractionAwareType.java:[26,8] org.jboss.as.hibernate.test.ProcedureParameterExtractionAwareType is not abstract and does not override abstract method extract(java.sql.CallableStatement,java.lang.String[],org.hibernate.engine.spi.SharedSessionContractImplementor) in org.hibernate.type.ProcedureParameterExtractionAware
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/ProcedureParameterExtractionAwareType.java:[33,5] method does not override or implement a method from a supertype
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/ProcedureParameterExtractionAwareType.java:[38,5] method does not override or implement a method from a supertype
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/MyPersistentCollection.java:[34,8] org.jboss.as.hibernate.test.MyPersistentCollection is not abstract and does not override abstract method setCurrentSession(org.hibernate.engine.spi.SharedSessionContractImplementor) in org.hibernate.collection.spi.PersistentCollection
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/MyPersistentCollection.java:[82,5] method does not override or implement a method from a supertype
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/MyPersistentCollection.java:[87,5] method does not override or implement a method from a supertype
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/MyPersistentCollection.java:[88,73] incompatible types: org.hibernate.HibernateException cannot be converted to java.lang.Throwable
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/MyPersistentCollection.java:[103,20] incompatible types: org.hibernate.HibernateException cannot be converted to java.lang.Throwable
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/BitSetTypeDescriptor.java:[79,9] incompatible types: org.hibernate.HibernateException cannot be converted to java.lang.Throwable
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/BitSetTypeDescriptor.java:[92,9] incompatible types: org.hibernate.HibernateException cannot be converted to java.lang.Throwable
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/ProcedureParameterNamedBinderType.java:[26,8] org.jboss.as.hibernate.test.ProcedureParameterNamedBinderType is not abstract and does not override abstract method nullSafeSet(java.sql.CallableStatement,java.lang.Object,java.lang.String,org.hibernate.engine.spi.SharedSessionContractImplementor) in org.hibernate.type.ProcedureParameterNamedBinder
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate-transformer/src/test/java/org/jboss/as/hibernate/test/ProcedureParameterNamedBinderType.java:[33,5] method does not override or implement a method from a supertype
[ERROR] -> [Help 1]
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1-jboss-1:compile (default-compile) on project jipijapa-hibernate4-1: Compilation failure: Compilation failure: 
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_1/src/main/java/org/jboss/as/jpa/hibernate4/JBossAppServerJtaPlatform.java:[28,51] package org.hibernate.service.jta.platform.internal does not exist
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_1/src/main/java/org/jboss/as/jpa/hibernate4/JBossAppServerJtaPlatform.java:[29,51] package org.hibernate.service.jta.platform.internal does not exist
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_1/src/main/java/org/jboss/as/jpa/hibernate4/JBossAppServerJtaPlatform.java:[30,51] package org.hibernate.service.jta.platform.internal does not exist
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_1/src/main/java/org/jboss/as/jpa/hibernate4/JBossAppServerJtaPlatform.java:[37,91] package org.hibernate.service.jta.platform.internal does not exist
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_1/src/main/java/org/jboss/as/jpa/hibernate4/JBossAppServerJtaPlatform.java:[39,19] cannot find symbol
[ERROR]   symbol:   class JtaSynchronizationStrategy
[ERROR]   location: class org.jboss.as.jpa.hibernate4.JBossAppServerJtaPlatform
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_1/src/main/java/org/jboss/as/jpa/hibernate4/JBossAppServerJtaPlatform.java:[63,15] cannot find symbol
[ERROR]   symbol:   class JtaSynchronizationStrategy
[ERROR]   location: class org.jboss.as.jpa.hibernate4.JBossAppServerJtaPlatform
[ERROR] -> [Help 1]
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1-jboss-1:compile (default-compile) on project jipijapa-hibernate4-3: Compilation failure: Compilation failure: 
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_3/src/main/java/org/jboss/as/jpa/hibernate4/HibernatePersistenceProviderAdaptor.java:[87,58] cannot find symbol
[ERROR]   symbol:   variable USE_NEW_ID_GENERATOR_MAPPINGS
[ERROR]   location: class org.hibernate.cfg.Configuration
[ERROR] /var/jenkins_home/workspace/vbe-eap-7.4.x-build/workdir/jpa/hibernate4_3/src/main/java/org/jboss/as/jpa/hibernate4/HibernatePersistenceProviderAdaptor.java:[88,80] cannot find symbol
[ERROR]   symbol:   variable SCANNER
[ERROR]   location: interface org.hibernate.ejb.AvailableSettings
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :wildfly-ormtransformer

[baranowb]

Its not tests per se. When maven kicks in, it builds project tree and populates deps from info it can find - be it compile, runtime or tests scope, does not matter.
This issue actually covers two problems:

1. AFAIR - no way to discriminate major/minor/micro - whatever is in repos and defined in channel, will be sucked as latests.
2. Different integration parts of build may depend on different API versions, it should be possible to restrict search in some way

[jmesnil]

@baranowb can you please expand on the issue?

The Channel API main goal is to provision WildFly where we should have only 1 version of any Maven Artifact.
It seems that you encounter this issue when you are using the Channel lib to build WildFly (where we could have multiple versions of the same artifact for testing).

[baranowb]

For WFLY it might work as it does not need to bother with long lived versions. EAP on the other hand does not have luxury. Also, Im fairly sure one of key points in having channel defs is to be able to provision specific version/ranges so maven repository does not have to be sanitazed (as it is now)

[jmesnil]

@baranowb Can you show me an example of a WildFly/EAP version where we were shipping different versions of the same GA? I'm curious to see how they would be defined in the pom.xml that was used to build it.

The Channel API is meant to curate the latest version of a Maven GA artifact according to a version or version/range constraint but it is not designed to curate multiple versions of the same GA. I want to understand the scope of that requirement before making that change.

[jfdenise]

@baranowb , we have different artifactId for hibernate jipijapa versions, each one referenced in a slot per hibernate version. Artifacts GA are different for different versions. So the problem you identified is not at provisioning time, right?

The problem seems to be when building. We can build against different versions of the same GA. The extension/maven plugin that will resolve version from a channel and inject versions during the build will be impacted by that.

Taking hibernate as an example, we have multiple versions for hibernate-core used at compile time. To compile jipijapa4.1 we use hibernate-core 4.2.13.Final, for jipijapa4.3 we use hibernate-core 4.3.10.Final).
But these versions don't end in the provisioned server, just compile time. A user would have to provide his own hibernate version to work with the associated jipijapa.

[baranowb]

@jmesnil - not shipped, AFAIR provisioning by users was one of key aspects?
@jfdenise Sort of, provisioning can happen only after build/test phase - and for such cases multiple versions are needed for build/test phase.

[wolfc]

I would leave out the build phase. Yes we do have multiple versions of Hibernate in there, but this is because we want to support multiple API versions.
Which brings it to testing. For testing the Maven modules contain runtime test coverage for each of the different API contracts. So each module would need to see a different version of Hibernate matching the API version used in building.

[jmesnil]

So each module would need to see a different version of Hibernate matching the API version used in building

The different versions of Hibernate are used for testing. They should have a <scope>test</scope>, right?
The channel library must not be used to fetch any Maven artifacts but only those that are part of the runtime server.

Depending on their scope, some of these artifacts will not be fetched through the channel library.
If we want to use this lib to "inject" version into WildFly building/testing, there needs to have some checks on which maven artifacts this applies to.

[jfdenise]

@wolfc , where are these tests located? I can only find tests in the repo that test the latest hibernate version.

[wolfc]

https://github.com/wildfly/wildfly/tree/25.0.0.Final/jpa (and its EAP fork)

[jfdenise]

@wolfc , thank-you. That is where I found the various versions of hibernate (for example: https://github.com/wildfly/wildfly/blob/25.0.0.Final/jpa/hibernate4_3/pom.xml#L19) but I missed the test dir in hibernate4_3. The build and test inside this module are both depending the same dependency on hibernate-core. What is specific to tests?

[wolfc]

You can see in the stacktrace I posted earlier how each module is affected individually.
The difference between build & test is the fact that we want to build against a given API and test against the latest. In theory we should be able to use Hibernate 4.3.0.Final for the build and should use Hibernate 4.3.11.Final for testing (not 4.3.10.Final).

[jfdenise]

@wolfc , I think we are on the same page. By using a single version of hibernate-core (the one in the channel), we break the transformer test compilation that expect 5.1, and each jipijapa_4x that expect a given 4.x version. The way the hibernate4_3 module is designed, only a single version is used for both build (API contract) and tests.

But in the end, can't it be fixed in the maven plugin/extension that inject the versions? If it does the resolution in the parent project (raw thinking), the sub modules that override the version (the hibernate modules that do that) should override what the extension/plugin has injected. Similar to what occurs without the plugin. No?

[baranowb]

Depending on their scope, some of these artifacts will not be fetched through the channel library.
If we want to use this lib to "inject" version into WildFly building/testing, there needs to have some checks on which maven artifacts this applies to.

That's a fundamental flaw. Provisioning cant happen without testing(and in case of our CI, build phase). Because you cant be sure if provisioning of certain version wont break something. There are two distinct flows:

1. curation of possible versions( this includes testing against proposed versions)
2. on-site provisioning of WFLY/EAP

From POV of extension/plugin there is no way to determine which modules should not be processed(or so to say relevant way, other than 'scope') - its either all or non/blacklisting, which makes no sense.
In order to automate testing of certain version Channels have to allow such discrimination to happen, either via API or as outcome of yaml rules/defs?

[jfdenise]

@baranowb , do you have some examples of that with 7.4? The only reference I found on dealing with artifacts with multiple versions was hibernate, and in this case the versions are hardcoded in the submodule. So the hardcoded value should still win. Do we have other cases?
Thank-you.

[vsalbaba]

Any progress on resolving this issue? Will this be closed or are we planning changes to the channels specification?

[jmesnil]

We can keep this issue as an enhancement but we don't need to fix it for our initial release.

[vsalbaba]

We can keep this issue as an enhancement but we don't need to fix it for our initial release.

@baranowb Are you okay with that?