Error: An unexpected error occurred while trying to open file

[alumni]

Error: An unexpected error occurred while trying to open file

[MBaesken]

please see also the section 'New System Property to Control the Maximum Size of Signature Files' at
https://www.oracle.com/java/technologies/javase/17-0-8-relnotes.html#JDK-8300596
and
https://www.oracle.com/java/technologies/javase/11-0-20-relnotes.html#JDK-8300596

[FrankReisenhofer]

Hello @MBaesken ,

Mend has released a fix: whitesource/unified-agent-distribution#48

[RealCLanger]

Hi,

from your description it's not quite easy to guess what you are referring to, but maybe it is about usage of the SapMachine docker base images, specifically Ubuntu. We have changed the image a little bit for size optimization. This was the old Dockerfile and here is the new one. As you see, the installed packages ca-certificates and gnupg2 along with their dependencies are not part of the base image any more as they were only needed to download and install the sapmachine package.

So, maybe you have been using ca-certificates beforehand inadvertently and now this package is missing? To fix this, you could add the installation of the package to your own custom container build that bases on the SapMachine image?

Best regards
Christoph

[alumni]

Hi Christoph, I'm building my own Docker image using node:18.17.0-bookworm-slim (basically debian:bookworm-slim with node on top) and manually installing sapmachine-17-jre on top:

apt-get install curl ca-certificates gnupg2
...
apt-get install sapmachine-17-jre=$JAVA_VERSION

If I install sapmachine-17-jre=17.0.8 instead of sapmachine-17-jre=17.0.7, the WhiteSource JAR file will stop working. The docker images have nothing do to with it, just that there was a change between 17.0.7 and 17.0.8 that broke WhiteSource.

[RealCLanger]

OK, then it's maybe a thing in Java - do you have a callstack? I mean, where does "Error: An unexpected error occurred while trying to open file /opt/wss/wss-unified-agent.jar" come from? Is it already during VM startup?

[alumni]

This is what I have on my local machine:

$ java --version 
openjdk 20.0.2 2023-07-18
OpenJDK Runtime Environment SapMachine (build 20.0.2+9)
OpenJDK 64-Bit Server VM SapMachine (build 20.0.2+9, mixed mode, sharing)

$ java -jar wss-unified-agent.jar 
Error: An unexpected error occurred while trying to open file wss-unified-agent.jar

[FrankReisenhofer]

We found a workaround. Instead of using 3.9-sapmachine-11 we use the docker-image: 3.9.2-sapmachine-11.

[RealCLanger]

This is what I have on my local machine:

$ java --version 
openjdk 20.0.2 2023-07-18
OpenJDK Runtime Environment SapMachine (build 20.0.2+9)
OpenJDK 64-Bit Server VM SapMachine (build 20.0.2+9, mixed mode, sharing)

$ java -jar wss-unified-agent.jar 
Error: An unexpected error occurred while trying to open file wss-unified-agent.jar

Can we inspect the wss-unified-agent.jar ?

[alumni]

Can we inspect the wss-unified-agent.jar ?

An older version is in my first message, here is the latest:

https://github.com/whitesource/unified-agent-distribution/releases/latest/download/wss-unified-agent.jar

The issue happens in all versions after the last patch update.

[FrankReisenhofer]

Local execution is leading to the following error: [1]

Is this helpful for you?

[1]
Error: An unexpected error occurred while trying to open file .\wss-unified-agent.jar
java.io.IOException: Unsupported size: 8577957 for JarEntry META-INF/MANIFEST.MF. Allowed max size: 8000000 bytes
at java.base/java.util.jar.JarFile.getBytes(JarFile.java:810)
at java.base/java.util.jar.JarFile.getManifestFromReference(JarFile.java:421)
at java.base/java.util.jar.JarFile.getManifest(JarFile.java:408)
at java.base/sun.launcher.LauncherHelper.getMainClassFromJar(LauncherHelper.java:553)
at java.base/sun.launcher.LauncherHelper.loadMainClass(LauncherHelper.java:778)
at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:686)

[RealCLanger]

Hi,

we believe the issue is understood. The culprit is a security patch that came with the July update. It is this commit 8300596: Enhance Jar Signature validation which was backported to all maintained JDK versions. There also exists a release note. It adds a limit of 8 MB for jar signature sizes. And the MANIFEST.MF of the jar file that you are using, is larger than that.

So, if you run your Java application with parameter -Djdk.jar.maxSignatureFileSize=9000000, it'll run again without problems.

Cheers
Christoph

[RealCLanger]

Local execution is leading to the following error: [1]

Is this helpful for you?

[1] Error: An unexpected error occurred while trying to open file .\wss-unified-agent.jar java.io.IOException: Unsupported size: 8577957 for JarEntry META-INF/MANIFEST.MF. Allowed max size: 8000000 bytes at java.base/java.util.jar.JarFile.getBytes(JarFile.java:810) at java.base/java.util.jar.JarFile.getManifestFromReference(JarFile.java:421) at java.base/java.util.jar.JarFile.getManifest(JarFile.java:408) at java.base/sun.launcher.LauncherHelper.getMainClassFromJar(LauncherHelper.java:553) at java.base/sun.launcher.LauncherHelper.loadMainClass(LauncherHelper.java:778) at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:686)

@FrankReisenhofer How did you get that Exception trace? When I reproduce this locally, I only get the message Error: An unexpected error occurred while trying to open file .\wss-unified-agent.jar. But it would have helped if the full exception text would have bubbled up. Maybe there is some room for improvement...

Update: Ah, I see, you can get it with the option -Xdiag 😄

[FrankReisenhofer]

To address the issue with the Mend unified agent I raised an issue here:

whitesource/unified-agent-distribution#48

[RealCLanger]

We also created an OpenJDK bug: https://bugs.openjdk.org/browse/JDK-8312489

With that, I'll close this issue. But it's still open for comments.

[AlexanderWirthSAP]

Is it possible to apply this setting jdk.jar.maxSignatureFileSize to the whitesourceExecuteScan step of the SAP piper pipeline somehow?

[MBaesken]

Hi Alexander, that's a good question for the Piper colleagues (or the ones that created this 'whitesourceExecuteScan').

I checked the lengthy docu at https://www.project-piper.io/steps/whitesourceExecuteScan/
but it does not help me to answer this very basic question you asked.
There is a dockerEnvVars param ('Environment variables to set in the container') maybe this can be used to sneak in the mentioned system-property via JAVA_OPTIONS.
And what really sounds interesting is the jreDownloadUrl parameter (you could use that for using an older version of SapMachine , but maybe you already found this); but it is not what you asked for .

[FrankReisenhofer]

Hello Mathias, hello Alexander,

According to my knowledge the settings can not set from end-users of the piper WhitesourceExecuteScan.

That is why I opened:

SAP/jenkins-library#4481

[AlexanderWirthSAP]

Thank you Matthias and Frank,

I tried setting the property by setting JAVA_OPTS or JAVA_TOOL_OPTIONS via the installCommand setting, but it did not seem to work and I'm not sure whether the installCommand even gets executed.

Setting the dockerImage to "maven:3.9.2-sapmachine-17" as suggested by Frank was the easiest workaround so far. If the Jenkins library implements a permanent solution that would be ideal, so thank you for opening the ticket. 😄

[MBaesken]

There is now an adjustment PR that increases the allowed value from 8000000 to 16000000
8312489: Increase jdk.jar.maxSignatureFileSize default which is too low for JARs such as WhiteSource/Mend unified agent jar
openjdk/jdk#15072