BIRT Framework fails to install in eclipse

[eclipse-birt]

BIRT Framework fails to install in eclipse

[SteveSchafer-Innovent]

I originally used 2020-06 but then I switched to 2021-06.

Thanks @ruspl-afed and @wimjongman for your helpful tips. You've filled in some of the gaps in my eclipse knowledge.

[ruspl-afed]

@SteveSchafer-Innovent Including to target is helpful to build BIRT itself, while org.apache.batik.anim.dom [1.14.0,2.0.0) should be visible by "host" eclipse on install time. It could be achieved by adding this bundle to some BIRT feature.

[SteveSchafer-Innovent]

I guess the first problem is finding that bundle. org.apache.batik.anim.dom doesn't exist in Orbit.

[ruspl-afed]

We have 2 ways:

1. add it to Orbit and consume from there
2. add it to BIRT
   In any case CQ needs to be created
   @wimjongman what you would prefer?

[ruspl-afed]

@SteveSchafer-Innovent is this problem specific to Eclipse 2020-06? I remember there was a special Eclipse Package with BIRT. I wonder how it was working there.

[wimjongman]

Adding to Orbit is quite painful...

Today, Tycho can consume directly from maven central. So I would prefer that. Otherwise, the most prudent thing to do is to get it into Orbit in some way. I would not do the fat jar approach if at all possible

[SteveSchafer-Innovent]

@wimjongman We have an unsatisfied OSGi dependency. The jar files provided by maven are not OSGi bundles. Also it looks like batik.anim is provided by maven as a whole and batik.anim.dom is a part of it. So I think that the batik.anim jar would need to be wrapped in a plugin that identifies itself as org.apache.batik.anim.dom and exports all the batik.anim.dom packages in the manifest, similar to what I did with axis (is that what you mean by "fat jar"?) The best way to do this is by adding it to Orbit, so that others who use these bundles won't run into the same problem. Or perhaps the dependency on org.apache.batik.anim.dom is a mistake that needs to be corrected.

@ruspl-afed 2020-06 is what I happened to have on-hand. I'll try a more recent version if there is one. Also there's the BIRT all-in-one package. I wonder if that would work.

[SteveSchafer-Innovent]

Actually the bundle doesn't need to exist because it's a package dependency. That dependency should be satisfied by the batik.anim bundle. I'll try to figure out why it isn't.

[SteveSchafer-Innovent]

I was able to get the framework installed but I had to install the following bundles first:

(Unless otherwise noted, these are from Orbit)
for batik: (all of these are needed to completely resolve)
org.apache.batik.anim_1.14.0.v20210324-0332.jar
org.apache.batik.dom.svg_1.14.0.v20210324-0332.jar
org.apache.batik.dom_1.14.0.v20210324-0332.jar
org.apache.batik.xml_1.14.0.v20210324-0332.jar
org.apache.batik.ext_1.14.0.v20210324-0332.jar
org.apache.batik.awt.util_1.14.0.v20210324-0332.jar
org.apache.batik.parser_1.14.0.v20210324-0332.jar
org.apache.batik.gvt_1.14.0.v20210324-0332.jar
org.apache.batik.script_1.14.0.v20210324-0332.jar
for poi:
org.apache.commons.collections4_4.4.0.v20200420-1700.jar
org.apache.commons.math3_3.6.1.v20200817-1830.jar
org.apache.poi.ooxml_4.1.1.v20200820-1148.jar
org.apache.poi.ooxml.schemas_4.1.1.v20200922-2105.jar
jetty-osgi-boot-10.0.5.jar (from maven https://mvnrepository.com/artifact/org.eclipse.jetty.osgi/jetty-osgi-boot/10.0.5)
javax.transaction_1.1.1.v201105210645.jar (this is a fragment)
jetty-deploy-10.0.5.jar (from maven https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-deploy/10.0.5)
org.apache.xmlbeans_3.1.0.v20200922-1359.jar

I did this by putting the jar files in the plugins directory and using the eclipse osgi console to figure out what unresolved dependencies there were. I hear I'm supposed to use p2 but that's beyond me.

I'm using eclipse 2021-06 for this.

What I need to know now is how to make this automatic. How do we get the birt framework to install these automatically if they are not already present?

[wimjongman]

What p2 repositories have you added to the installation before you do the installation?

At least the following repositories should be available:

• the birt repository
• the release train repository e.g. https://download.eclipse.org/releases/latest (which eventually includes birt again)
• The eclipse project repository e.g. https://download.eclipse.org/eclipse/updates/latest

Whatever cannot be resolved by these repos should be included in one of our features.

[SteveSchafer-Innovent]

I downloaded Eclipse IDE for Enterprise Java and Web Developers. In the past I've been able to install the BIRT framework in the JEE version of eclipse with no problem.

In terms of adding p2 repositories, I assume you are referring to Help -> Install New Software. I see https://download.eclipse.org/release/latest and https://download.eclipse.org/eclipse/updates/4.20 in the dropdown list. If I select either of them there are a lot of things that are not currently installed. It's possible that some of these items contain the bundles I need but how do I tell which ones? The names in the list don't really tell me.

[wimjongman]

Yes, those are correct. These repositories will also be examined when installing software, providing the 'Contact all update sites during install to find required software' box is checked.

[ruspl-afed]

Then if you will add another update site with BIRT (archive one?)
and
if you have Contact all update sites during install to find required software

it is assumed that the most of required bundles will be found somewhere and then we "just" need to add the rest of required bundles to the BIRT features.

[SteveSchafer-Innovent]

Okay well that's exactly what I did and it didn't work. 'Contact all update sites' is checked but it gives me the error I originally posted and it is necessary to manually add all those jars in order for it to work. So that means those jars need to be included in one of our features. I'm not sure how to do that. If you can point me in the right direction I'm sure I can figure it out.

[ruspl-afed]

I would add required bundles to the feature that is now installable, i.e. to org.eclipse.birt.feature
The easiest way to do this is to add an entry like

   <plugin
         id="org.apache.batik.anim"
         download-size="0"
         install-size="0"
         version="0.0.0"
         unpack="false"/>

to the bottom of feature.xml manifest

The added bundle should be "resolvable" by target platform, so you may need to add more entries to the "orbit" location of .target manifest. You may also add another location to the .target for the more recent Orbit if the current one does not contain what we need.

[ruspl-afed]

@SteveSchafer-Innovent you can have a look on this report https://download.eclipse.org/oomph/archive/reports/download.eclipse.org/releases/2021-06/index.html to understand what is currently published for SimRel. Unfortunately I don't know where to find the corresponding report for 2020-06.

[wimjongman]

If your goal is to do a test install, you can also add the orbit repo to the update sites.

• Select install new software
• Paste this URL : https://download.eclipse.org/tools/orbit/downloads/drops/R20210602031627/repository
• Wait for it to resolve
• Select your BIRT repo again and install as usual

The orbit URL can be found from the orbit site:

https://download.eclipse.org/tools/orbit/downloads/

[wimjongman]

Unfortunately I don't know where to find the corresponding report for 2020-06.

why do you mention 2020-06? It is old and should not be used.

[SteveSchafer-Innovent]

I now have a bit of a problem. I've added all the bundles to org.eclipse.birt.feature/feature.xml and now if fails to build:

[ERROR] Failed to execute goal org.eclipse.tycho.extras:tycho-source-feature-plugin:2.3.0:source-feature (source-feature) on project org.eclipse.birt: Could not generate source feature for project MavenProject: org.eclipse.birt.features:org.eclipse.birt:4.9.0-SNAPSHOT @ /disk1/home/innovent/git/birt-fork/birt/features/org.eclipse.birt.feature/pom.xml
[ERROR] Missing sources for plugins [org.apache.poi.ooxml.schemas_4.1.1.v20200922-2105]

If I remove org.apache.poi.ooxml.schemas it builds successfully but won't install in eclipse. That bundle is coming from Orbit and Orbit R20210602031627 is included in target. However Orbit has no source for that bundle. Is there a way to indicate there is no source?

I haven't been able to find to find documentation about feature.xml. Is there any that you know of?

[ruspl-afed]

We can try symptomatic treatment:
let's exclude this bundle here
https://github.com/eclipse/birt/blob/master/pom.xml#L268
like described here
https://www.eclipse.org/tycho/sitedocs/tycho-extras/tycho-source-feature-plugin/source-feature-mojo.html#excludes

[wimjongman]

We should probably make a separate "dependencies" feature to include third-party references.

[ruspl-afed]

We should probably make a separate "dependencies" feature to include third-party references.

I'm not so sure at the moment. I think that first we need to analyze the usage of 3rd party per BIRT component, otherwise we can add just more coupling.

[wimjongman]

I mean, instead of adding them to the org.eclipse.birt.feature

@SteveSchafer-Innovent , did you try to add the orbit repo as I suggested in #666 (comment)?

[SteveSchafer-Innovent]

@wimjongman I just tried it and I get this error when trying to install birt:

Cannot complete the install because one or more required items could not be found.
Software being installed: BIRT Framework 4.9.0.v202108222003 (org.eclipse.birt.feature.group 4.9.0.v202108222003)
Missing requirement: Jetty :: Deployers 9.4.15.v20190215 (org.eclipse.jetty.deploy 9.4.15.v20190215) requires 'java.package; org.eclipse.jetty.webapp [9.4.15,10.0.0)' but it could not be found
Cannot satisfy dependency:
From: BIRT Framework 4.9.0.v202108222003 (org.eclipse.birt.feature.group 4.9.0.v202108222003)
To: org.eclipse.equinox.p2.iu; org.eclipse.jetty.deploy [9.4.15.v20190215,9.4.15.v20190215]

To be clear, I just need to add the orbit repo to the repository list. I don't need to actually install anything, correct?

The above error is the same one I've been getting before trying this and it's interesting because the target file contains a location for jetty 9.4.15 but eclipse 2021-06 already contains jetty 10.0.5 bundles, but it's not all the bundles that are needed. Unfortunately, https://download.eclipse.org/jetty/updates doesn't contain anything higher than version 9. I'm still trying to find a URL for version 10 that can serve as a repository location. If you know of one please let me know.

[wimjongman]

Right! In 2021-06, Eclipse moved to Jetty 10. It means that jetty 9 is no longer in any of the three repos.

So in order for BIRT to resolve correctly, you also have to add [1], which is the jetty 9 P2 repo.

[1] http://download.eclipse.org/jetty/updates/jetty-bundles-9.x

[SteveSchafer-Innovent]

@wimjongman That location is in target but I still get the same error.

Oh wait, do you mean put that location in Help -> Install Software! I'll try that.

Excellent! Now it installs. But this means that there are extra actions that are needed in order to install the framework. Let's keep this issue open to discuss that.

[ruspl-afed]

@SteveSchafer-Innovent I'm not sure where to get the latest Jetty p2 update site, but may be this one could be useful https://download.eclipse.org/webtools/jetty/10.0.4/repository/

[SteveSchafer-Innovent]

With https://download.eclipse.org/webtools/jetty/10.0.4/repository/ specified in the target instead of 9.x, it fails to build. I needed to remove the org.eclipse.jetty.security and org.eclipse.jetty.xml units and now I'm stuck on this error:

[ERROR] Cannot resolve target definition:
[ERROR] Software being installed: org.eclipse.jetty.webapp 10.0.4
[ERROR] Missing requirement: org.eclipse.jetty.webapp 10.0.4 requires 'osgi.extender; (osgi.extender=osgi.serviceloader.processor)' but it could not be found

org.eclipse.jetty.webapp has a Require-Capability in its manifest that's causing this. 9.x also had this requirement but it was being satisfied somehow whereas now it's not. Let me know if you have any insights.

[wimjongman]

I found this issue: jetty/jetty.project#2153

[SteveSchafer-Innovent]

Still struggling with this. What I know at this point:
When I add the jars manually as described in this comment, I can install the birt framework which still depends on jetty 9.x in the target. After installation, I can only see jetty 10 bundles in the osgi console. But I haven't been able to accomplish this without manually installing the jars. I can add jetty 9.x in help->install new software and successfully install but then all the jetty 9.x bundles are included. If I try to change the target to depend on jetty 10, I can't get it to build. Following some of the suggestions in eclipse/jetty.project#2153 and bugzilla 494913 such as adding org.eclipse.equinox.ds or org.apache.felix.scr have not been fruitful. But I'm finding that the target file is very brittle and it's easy to break things with random changes. Next step is to figure out how to add Apache Aries SPI Fly and try to learn more about it, although I don't understand why it should be needed since jetty 9.x installs without it.

[ruspl-afed]

such as adding org.eclipse.equinox.ds or org.apache.felix.scr

@SteveSchafer-Innovent both should be in the Eclipse Platform already, i.e. a part of http://download.eclipse.org/releases/2021-06

@wimjongman should we try to switch BIRT to 2021-06 (or 2021-09) ? Perhaps it will help to resolve this one.

[SteveSchafer-Innovent]

Everything I've been doing recently is with 2021-06.

[SteveSchafer-Innovent]

@ruspl-afed you are right that org.apache.felix.scr is in 2019-06. org.eclipse.equinox.ds is not but this comment stated that org.apache.felix.scr replaced org.eclipse.equinos.ds.

Also this comment states that org.apache.felix.scr should provide the "osgi-extender" capability, but that doesn't seem to work in my case.

[wimjongman]

Alexander, I already work on this as part of #640 this compiles but tests are failing. All is build with jetty 10. I have not worked on it for a while, I hope to get some time next week.

[wimjongman]

Also, it does not run the report viewer. Jetty 10 is installed. After building, the all in one project can be found here: build\birt-packages\birt-report-all-in-one\target\products

[SteveSchafer-Innovent]

In target, I switched the jetty location to https://download.eclipse.org/webtools/jetty/10.0.5/repository/, and included a location for jetty-osgi-boot 10.0.5 from maven, and also org.apache.aries.spifly.dynamic.bundle from maven. That got rid of the osgi.extender error but now I have:

Missing requirement: org.eclipse.equinox.http.jetty 3.7.0.v20190305-2216 requires 'java.package; org.eclipse.jetty.http [9.4.0,10.0.0)' but it could not be found

I couldn't find any specific reference to that version of org.eclipse.equinox.http.jetty in the source code, so I don't yet know where it's coming from. Maybe https://download.eclipse.org/eclipse/updates/4.11/.

I'll take a look at #640.

[SteveSchafer-Innovent]

The target file in PR 640 is VERY different. I attempted to install the framework from 640 and got this:

Cannot complete the install because one or more required items could not be found.
Software being installed: BIRT Framework 4.9.0.v202109021750 (org.eclipse.birt.feature.group 4.9.0.v202109021750)
Missing requirement: org.apache.xmlgraphics:batik-transcoder 1.14.0.v20210324-0332 (org.apache.batik.transcoder 1.14.0.v20210324-0332) requires 'java.package; org.apache.batik.anim.dom [1.14.0,2.0.0)' but it could not be found

Not sure why that is. The orbit location should contain it. and it is referenced in includeBundles. Maybe it needs to be included in a feature.

[SteveSchafer-Innovent]

@wimjongman , I was able to get PR 640 framework to install successfully. I made the following changes:

Add these plugins to features/org.eclipse.birt.feature/feature.xml:
org.apache.batik.anim
org.apache.batik.awt.util
org.apache.batik.gvt
org.apache.batik.script
org.apache.commons.collections4
org.apache.commons.math3
org.apache.poi.ooxml
org.apache.poi.ooxml.schemas
org.eclipse.jetty.osgi.boot
javax.transaction
org.eclipse.jetty.deploy
org.apache.xmlbeans

Add org.apache.poi.ooxml.schemas in configuration excludes in tycho-source-feature-plugin in features/org.eclipse.birt.feature/pom.xml and /pom.xml

So PR 640 will fix this issue if you make those changes and once you get the testing issues worked out.

[wimjongman]

Not sure why that is. The orbit location should contain it. and it is referenced in includeBundles. Maybe it needs to be included in a feature.

Steve, when we use a package import, P2 cannot resolve it to a bundle. AFAIK, P2 does not know which bundle exports which package.

In order for a bundle to be installed, one of the following needs to be true:

• its dependencies need to be installed before
• or its dependency needs to be included
• or its dependency is required in the feature.

Including third-party bundles in our feature is not good.

However, if a bundle cannot be resolved we can require the bundle in the feature. This will tell P2 to pull in the bundle.

[SteveSchafer-Innovent]

@wimjongman So you are saying that the changes I made to feature.xml are not good.

The first option, dependencies need to be installed before, I think means the user would need to install something via Help -> Install New Software, or at least add a repository to the repo list prior to installing the BIRT framework, which is something we are trying to avoid. I think BIRT should install into JEE Eclipse without any additional steps. Let me know if you disagree.

The second option I think means embedding these jars into a plugin's lib folder in BIRT and exporting packages in its manifest. Is that correct? That's what I did for axis.

[ruspl-afed]

or at least add a repository to the repo list

Perhaps it could be solved with p2.inf instructions that we can include to BIRT feature

install into JEE Eclipse

Please clarify what Eclipse Package we are talking about.

The second option I think means embedding these jars into a plugin's lib folder

Oh, please, no.

[SteveSchafer-Innovent]

Eclipse package: https://www.eclipse.org/downloads/packages/release/2021-06/r/eclipse-ide-enterprise-java-and-web-developers

[wimjongman]

Fixed!

[claesrosell]

Magic?

[wimjongman]

Was this not fixed? I believe it was when we upped our requirement to 4.23. I will try later today. Would be good if we had a couple of independent confirmations.

[claesrosell]

I will test it as well. Hopefully tonight.

[wimjongman]

If I install a fresh 4.23M3 from [1] and then install the BIRT SDK from [2] all is fine. Signing issues are addressed here [3]

[1] https://download.eclipse.org/eclipse/downloads/
[2] https://download.eclipse.org/birt/update-site/snapshot/
[3] #817