Report viewer failing again.

[eclipse-birt]

Report viewer failing again.

[SteveSchafer-Innovent]

I've been looking at this and have narrowed it down a little bit. The 404 error is caused by org.eclipse.jetty.servlet.ServletHandler.Default404Servlet. When the request comes in, Jetty looks in a list of servlets and the Default404Servlet is the only thing in that list. Servlets can only be added to that list in ServletHandler.setServletMappings() and that method is never called by the ViewerPlugin or anywhere other than to add the 404 servlet. I don't really know where it should be called, so I'm going to see what an older version of BIRT looks like, where the viewer used to work. Maybe Jetty was different or maybe the ViewerPlugin was different. We'll see.

[hvbtup]

Hmm, it seems like Jasper is using the Jasper JSP compiler (which sounds somewhat silly btw, because we are running BIRT, not JasperReports).
Do we need JSP at all for BIRT?
It seems like this can be disabled or configured, see https://www.eclipse.org/jetty/documentation/jetty-9/index.html#configuring-jsp
But I'm a complete noob in this regard.
Maybe we need some help from the experts of the Jetty project...

[wimjongman]

I see in the build that we indeed have a step where jsp's are being compiled. I think that the webviewer is using jsp.

I don't know why but I have a feeling that this issue might be related.

[hvbtup]

Yes, that seems related. Probably you should contact Alexander Kurtakov. Just to clarify this. The problem is not Jetty in general, but specifically compiling JSP pages? Do we need JSP at all? From what I see in the BIRT sources, there are only 3 *.jsp files: * chartImage.jsp * control.jsp * Userpreference.jsp All three only contain static content and could be replaced with pure HTML files, as it seems. From this point of view we could just stop using JSP. Disclaimer: I am not using the Web Viewer – maybe there are other reasons for JSP… Henning Von: Wim Jongman ***@***.***> Gesendet: Montag, 3. Mai 2021 09:45 An: eclipse/birt ***@***.***> Cc: Henning von Bargen ***@***.***>; Comment ***@***.***> Betreff: Re: [eclipse/birt] Report viewer failing again. (#649) I see in the build that we indeed have a step where jsp's are being compiled. I think that the webviewer is using jsp. I don't know why but I have a feeling that this issue<https://bugs.eclipse.org/bugs/show_bug.cgi?id=453379> might be related. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#649 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AB2KM2HJH5AI4DCB2JFGZXTTLZIAJANCNFSM437I7HFQ>.

[SteveSchafer-Innovent]

There are 34 jsp files in the generated webapp. It would not be possible to replace them with html files. They are in viewer/org.eclipse.birt.report.viewer/birt/webcontent/birt.

[hvbargen]

I see.

At this time this (the viewer not working) is the most annoying bug in BIRT - I'd call it a showstopper.

I really think you should ask for help in the jetty-dev mailing list.

[wimjongman]

Please feel free to step in.

[hvbtup]

I would if I had any idea of the build process. But I never worked with Maven, Tycho, Eclipse, P2, M2 and so on - I really don't understand it. So I'm afraid I cannot help.

[wimjongman]

This was exactly my situation before I decided to dig in ;)

[hvbargen]

I looked at the comments on bug 453379 and this is intimidating - seems like Jetty is causing trouble for years.

However, in that bug, it was said that one reason for doing the things that way, was some conflict with the "help system".

I don't care about the help system as long as the BIRT IDE is unusable (I don't see why a help system needs JSP at all).
So, is it an option to add org.apache.jasper (not org.apache.jasper.glassfish) as a dependency somehow?

[hvbtup]

Maybe Jetty was different or maybe the ViewerPlugin was different.

AFAIK Jetty was different and included support for JSP, which now requires additional libraries. IOW the newer releases of Jetty no longer have "batteries included" (in Python speech).

[hvbtup]

Meanwhile I was trying to make Jetty log messages again (I'm pretty sure that would help us debugging this issue), but without success so far.
Jetty now uses SLF4J for logging. SLF4J ist more or less just an API for logging. It needs a binding to another library which actually performs the logging, such as java.util.logging, logback oder log4j. Without such a binding, it will log only exactly one message to tell you that it won't log any messages (using a no-op binding).
There are some of these bindings in Orbit, but I don't know how to make this work.
Perhaps someone with a bit more OSGI know-how could take a look at this.

[hvbtup]

I showed the SLF4J problem to a colleague who knows a bit more about Eclipse than I do, but we couldn't solve it.
If I understand the SLFJ documentation correctly, it should work, because slf4j-simple is available (and shows as active in the OSGI console).

[hvbtup]

I guess that the SLF4J problem hides the root problem. In the stacktrace (see log.txt from the OP), which was produced by Jetty 9, the error message was:

java.lang.ClassNotFoundException: org.apache.jasper.runtime.JspSourceImports cannot be found by org.eclipse.jetty.osgi.boot_9.4.37.v20210219

I guess that the situation is still the same with Jetty 10, but now it is hidden by the SLF4J problem, because Jetty just cannot log the error.

[hvbtup]

We still didn't get the SLF4J logger to work, but I found the following exception which is ignored (!?) in javax.servlet.jsp.JspFactory.class:

org.apache.jasper.runtime.JspFactoryImpl cannot be found by javax.servlet.jsp_2.2.0.v201112011158

So I think this is a project setup problem, we need to have the JSP implementation library available somehow - similar to the problem with Jetty 9.

[wimjongman]

For the logger to work, I have a theoretical idea.

SLF4j requires an implementation at runtime. This means that jetty needs a dependency on a logging implementation otherwise OSGi will not allow jetty to see anything, no matter how many implementation bundles are active.

You can try to create a fragment that has the jetty bundle as a master and which depends on an slf4j implementation. Master plugins are able to see all dependencies from their fragments.

[wimjongman]

org.apache.jasper.runtime.JspFactoryImpl cannot be found by javax.servlet.jsp_2.2.0.v201112011158

This issue is caused by the ViewerPlugin class. If you examine it you will see the setupJSPFactory method. This fails silently. I have been able to fix this by adding that package to the dependencies.

[hvbtup]

Yes, that's where I found the bug.
So does the Run > View as PDF now work for you? And what exactly did you do?

[wimjongman]

No, it doesn't make any difference. I will push my intermediate changes to a branch.

[hvbtup]

I added org.apache.jasper.glassfish (2.2.2) to the target platform and the plugin dependencies of org.eclipse.birt.report.viewer.
Stepping through in the debugger, I can see that the first attempt to load JspFactoryImpl fails (in JspFactory.getDefaultFactory()), but then the second attempt in ViewerPlugin setupJspFactory succeeds.

Unfortunately this still does not help. Probably something else is missing.

I found http://www.descher.at/descher-vu/category/eclipse/jetty/ where someone explains how he managed to get JSP up and running with Jetty 9. This may be a helpful resource, but I don't really understand it.

[hvbtup]

For those who can help with the SLF4J problem:
I think this can be narrowed down by debugging and adding a breakpoint at slf4j-api-2.0.0-alpha1-sources.jar\org\slf4j\LoggerFactory.java at the beginning of private static List findServiceProviders(). This is where the magic happens.

[adiinfo-77]

Hello

regarding https://github.com/eclipse/birt/blob/master/viewer/org.eclipse.birt.report.viewer/pom.xml:

i compared your POM with mine (Report viewer works in my setup).
Are you sure that you don't need to declare the dependencies in the plugin section? I would suspect that the required jar files are missing otherwise in the final deployment.

at the end you find my plugin section for org.eclipse.jetty.
i remember that originally jspc compiler was set to glassfish but you can also use tomcat.
In my setup, glassfish is commented out and tomcat jspc is the one that is used.

I hope this helps.
Andrej

		<plugin>
			<groupId>org.eclipse.jetty</groupId>
            <!--<artifactId>jetty-jspc-adi-maven-plugin</artifactId>-->
            <!--<version>9.0.7.v20131107</version>-->
            <!--<artifactId>jetty-jspc-adi-edge-maven-plugin</artifactId>-->
            <artifactId>jetty-jspc-maven-plugin</artifactId>
            <version>${jetty.version}</version>
			<executions>
				<execution>
                    <id>pre-compile-jsp</id>
                    <phase>process-classes</phase>
					<goals>
						<goal>jspc</goal>
					</goals>
					<configuration>
						<webAppSourceDirectory>${project.basedir}/birt</webAppSourceDirectory>
						<webXml>${project.basedir}/birt/WEB-INF/web-viewer.xml</webXml>
						<generatedClasses>${project.build.directory}/jsp/classes</generatedClasses>
						<useProvidedScope>true</useProvidedScope>
						<keepSources>true</keepSources>
						<classesDirectory>${project.build.directory}/birt/WEB-INF/lib/viewservlets.jar-classes</classesDirectory>                            
						<sourceVersion>1.8</sourceVersion>
						<targetVersion>1.8</targetVersion>
                        <!--<ignoreJspFragmentErrors>true</ignoreJspFragmentErrors>-->
					</configuration>
				</execution>
			</executions>
            <dependencies>
                <dependency>
                    <groupId>org.eclipse.jetty</groupId>
                    <artifactId>jetty-util</artifactId>
                    <version>${jetty.version}</version>
                </dependency>
                <dependency>
                    <groupId>org.mortbay.jasper</groupId>
                    <artifactId>apache-jsp</artifactId>
                    <version>8.5.35.1</version>
                </dependency>
                <dependency>
                    <groupId>org.mortbay.jasper</groupId>
                    <artifactId>apache-el</artifactId>
                    <version>8.5.35.1</version>
                </dependency>                    
                <dependency>
                    <groupId>org.apache.ant</groupId>
                    <artifactId>ant</artifactId>
                    <version>1.10.9</version>
                </dependency>
                <dependency>
                    <groupId>javax.servlet</groupId>
                    <artifactId>javax.servlet-api</artifactId>
                    <version>3.1.0</version>
                </dependency>
                <!--<dependency>
                    <groupId>org.eclipse.jetty.orbit</groupId>
                    <artifactId>org.apache.jasper.glassfish</artifactId>
                    <version>2.2.2.v201112011158</version>
                </dependency>
                <dependency>
                    <groupId>org.eclipse.jetty.orbit</groupId>
                    <artifactId>javax.el</artifactId>
                    <version>2.2.0.v201303151357</version>
                </dependency>                    
                <dependency>
                    <groupId>org.eclipse.jetty.orbit</groupId>
                    <artifactId>com.sun.el</artifactId>
                    <version>2.2.0.v201303151357</version>
                </dependency>
                <dependency>
                    <groupId>org.eclipse.jetty.orbit</groupId>
                    <artifactId>org.apache.taglibs.standard.glassfish</artifactId>
                    <version>1.2.0.v201112081803</version>
                </dependency>                    
                <dependency>
                    <groupId>org.eclipse.jdt</groupId>
                    <artifactId>org.eclipse.jdt.core</artifactId>
                    <version>3.24.0</version>
                </dependency>-->
                <dependency>
                    <groupId>org.eclipse.jetty.orbit</groupId>
                    <artifactId>org.eclipse.jdt.core</artifactId>
                    <version>3.8.2.v20130121</version>
                </dependency>
            </dependencies>                
		</plugin>

[ruspl-afed]

at the end you find my plugin section for org.eclipse.jetty. i remember that originally jspc compiler was set to glassfish but you can also use tomcat. In my setup, glassfish is commented out and tomcat jspc is the one that is used.

Hi Andrej @adiinfo-77 ,

Could you please propose a PR based on your setup?

Thanks in advance,
AF

[adiinfo-77]

Hi Alexander

i made the pull request.

I also had a look at the automatically generated maven build. are you aware of the fact that:

• you work simultaneously working with Jetty 10.0.6 and Jetty 9.4.4? Is this intended?
• the slf4j Logger Class is not loaded
  if Jetty 10 and Jetty 9 use different versions of slf4j (and i think this is likely), than you will have most likely classloader-problems there. Apart from other side effects when using jar's from Jetty 9 and Jetty 10.

i made a copy of the maven messages in question at the end of this comment.

Thank you
Andrej

[INFO] default instructions = {Export-Package=;version="${version}";-noimport:=true, Bundle-SymbolicName=wrapped.${mvnGroupId}.${mvnArtifactId}, g.slf4j.spi.SLF4JServiceProvider)"=, Bundle-Version=${version}, Bundle-Name=Bundle derived from maven artifact ${mvnGroupId}:${mvnArtifactId}:${mvnVersion}, DynamicImport-Package=, version=${version_cleanup;${mvnVersion}}, Import-Package=*;resolution:=optional, loader.processor)",osgi.serviceloader;filter=="(osgi.serviceloader=or, Require-Capability=osgi.extender;filter:="(osgi.extender=osgi.service}
[INFO] Resolving MavenDependencyRoots = [GroupId = org.apache.xmlgraphics, ArtifactId = fop, Version = 2.6, ArtifactType = jar, IncludeDependencyScope = ], IncludeDependencyScope = null, MissingManifestStrategy = GENERATE, IncludeSource = true...
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

[INFO] Resolving MavenDependencyRoots = [GroupId = org.eclipse.jetty, ArtifactId = jetty-continuation, Version = 9.4.44.v20210927, ArtifactType = jar, IncludeDependencyScope = ], IncludeDependencyScope = compile, MissingManifestStrategy = GENERATE, IncludeSource = true...

[INFO] Resolving MavenDependencyRoots = [GroupId = org.eclipse.jetty, ArtifactId = jetty-xml, Version = 10.0.6, ArtifactType = jar, IncludeDependencyScope = ], IncludeDependencyScope = compile, MissingManifestStrategy = GENERATE, IncludeSource = true...

[claesrosell]

I finally got the logging to work.
Since Jetty can use slf4j1.7.3, I :

• added the slf4j simple logger to the target platform (from the Orbit p2 repo).
• removed the slf4j 2.0.0 alpha 1 bundles from the debug configuration
• added slf4j 1.7.3 bundles to the debug configuration

On another note:
I really tried to get it to work with slf4j 2.0.0 alpha1. It should be possible, according to this post: https://www.mail-archive.com/[email protected]/msg00878.html .

This may be easier to solve with jetty 10.0.7 , which is using slf4j 2.0.0 alpha5, which in turn has the missing "Require-Capability" header in alpha1, but it does not seam like the simple logger (alpha5), has the corresponding "Provides-Capability". Some more info here: https://jira.qos.ch/browse/SLF4J-457

[adiinfo-77]

Hello

i had to close (delete) my pull request.

On double checking, it turned out that while the RCP Report Designer does start, the Jetty Http-Server does not.

The only working configuration i have is as described here: https://www.optimaw.ch/change-to-jdk11-lts/ (Point 5)
This configuration works only with Jetty 9.4.15 and a modified jetty-jspc-maven-plugin from the 9.0.x branch. And furthermore, only with the (very old) Glassfish JSPC-Compiler.

Sorry for the inconvenience
Andrej

[ruspl-afed]

Thank you for sharing your knowledge @adiinfo-77 !

We are looking forward to see more comments and PRs from your side.

[adiinfo-77]

Hi Alexander

when i did my work on updating BIRT, i struggled most of the time with the precompilation of the JSP files in the report viewer. I did never understand, why the precompilation was done a all! To my knowledge, JSP-Files must be copied to WEB-INF\src for automatic precompilation. But the BIRT build-process copies the jsp-files to WEB-INF\classes.
When i tried to switch from Glassfish to Tomcat Precompiler, Tomcat did obviously not precompile the JSP-files as these were not in WEB-INF\src.

Also, even if i got Tomcat to precompile these files by manual measures, the classpath in the resulting viewer.jar was wrong. With Glassfish precompiler, it was org.eclipse.birt (correct), with Tomcat the classpath was diffrent (i do not remember what it was, probably org.apache).

That's why i changed the source code of the old jetty-jspc-maven-plugin so that it could be used with Java 6 source code level. And that's why i have still to use Glassfish JSP Precompiler.

Maybe this helps a bit.
Andrej

[claesrosell]

I got the viewer working and have tried to make a clean "step by step" commit pull request for this. It was tricky so please try it out, if it passes build, that is, because I have only tested to run this from the PDE. :-)

[hvbtup]

Could not wait until Monday to try this out...
@claesrosell :

Yeah, this is great! The preview is actually working!

I tested this from within Exclipse only, not yet with the Maven build.
The SLF4J still not working. Did you commit the changes to make it work as well?

And there are some errors logged in the BIRT IDE, but this is a huge step forward.

[claesrosell]

I excluded the "fix" of the logging in this PR since I wanted to keep it as clean as possible.

[hvbtup]

I tried to build with mvn, but this failed. Maybe something is wrong with my environment... I'll create a separate issue for that.

[patric-r]

Thanks!
I pulled the recent changes, but I'm now getting java compilation errors:

[claesrosell]

Those dependencies should have been added to the target platform and since the CI build was successful I think that they were.
Try to open your .target file (org.eclipse.birt.target.target) and press "Reload target platform".
Hopefully that will solve the issue.

[patric-r]

Thanks, this fixed the issue!
Is there a way to do this automatically whenever the target platform definition changes?

[claesrosell]

You mean something like this ?: https://bugs.eclipse.org/bugs/show_bug.cgi?id=576844

[hvbtup]

I was able to generate the all-in-one-designer with mvn from within the Eclipse IDE.
The report viewer (inside the all-in-one-designer) is working here.
I tested with a quite complex PDF report, and at first glance everything is looking fine.

Great work!

However I noticed that starting the all-in-one-designer takes 2-3 minutes on my machine (Core i5, 8GB RAM, SSD) with high cpu. This is before the progress bar shows. The 4.2.1 all-in-one-designer only takes a few seconds to start.

Is this to be expected?

[claesrosell]

No, that is not expected. I guess that we need to investigate that.
Please file a new issue for that problem, we will need to profile the startup, to see what causes the increase in time. The start of jetty (used by the web viewer) does not happen until it is first used so it is not directly caused by that.

[wimjongman]

Is it also slow on the second launch?

[wimjongman]

Henning, can you also try if report parameters work?

[hvbtup]

Yes, report parameters work.
In my test case, I have two linked parameters. One of them is hidden and uses default values, the second one uses the first one to show a LOV. Not too complicated, but not trivial OTOH.