doesn't find classes in jar file manifest's classpath

[classgraph]

doesn't find classes in jar file manifest's classpath

[lukehutch]

Thanks for the report. I think you're saying that FastClasspathScanner doesn't detect classes added using the Class-Path line in a jar's manifest file. Are you using the latest version in git? I checked in some additional classpath detection methods recently, but haven't made an additional release yet.

[tsujamin]

Yep that's what I meant. I'll pull from git now and test it

[tsujamin]

No luck on HEAD. I had to add " -Dmaven.test.skip=true" when I built fast-classpath-scanner because all the tests were failing on my machine...

[tsujamin]

The case where I specify the classpath at commandline (instead of with the Class-Path jar line) still works on HEAD

[lukehutch]

Weird. I'm looking at this now, and I don't see entries from either the "-cp" switch or from the Class-Path manifest line in either the java.class.path property, or in the list of URLs returned by the ClassLoader. I have no idea how to read these other classpath entries!

[tsujamin]

Apparently they just get loaded the same as the jar that refers to them:
"In general, classes referenced by a JAR-class-path entry are found as though they were part of the JAR file. The JAR files that appear in the JAR-class-path are searched after any earlier class path entries, and before any entries that appear later in the class path." - https://docs.oracle.com/javase/8/docs/technotes/tools/findingclasses.html

So maybe they are prefixed weirdly or something, I'm no expert on HotSpot's classloader implementaiton ;)

[tsujamin]

There's a wrapper class for manifests which could be used, but that would probably require attempting to load the manifest for each jar in the classpath. Not sure how expensive that would be.

https://stackoverflow.com/questions/1272648/reading-my-own-jars-manifest

[tsujamin]

There is one other way, but it depends on reflection and hotspots implementation: https://stackoverflow.com/questions/25729319/how-does-a-classloader-load-classes-reference-in-the-manifest-classpath

I'm happy to have a crack at supporting it this way, but no issue if you'd rather not add support for this feature

[lukehutch]

Good sleuthing. I think I'll use a variation of the "reading-my-own-jars-manifest" version. I have a question though, if you specify a jarfile with a non-absolute path in a Class-Path manifest field, how is it located? Is that relative path added to each existing classpath directory to try to locate the jarfile? Or do Class-Path entries require absolute paths? Or are they relative to the current directory at launch?

[tsujamin]

they are loaded relative to the jarfile containing the manifest being read. I've got a rough prototype of the later working at the moment, will link it in the morning

[tsujamin]

Yeah I've got it working using the reflection method, will PR a prototype tomorrow sometime.

[lukehutch]

Hey, sorry for the duplicate work, I just pushed a fix. Can you please pull the latest version and test both the "-cp" and "Class-Path:" methods with your codebase? Thanks!

(I decided against the reflection method, since that seems the diciest of all...)

N.B. There's a bit of trickiness, because jarfiles can refer to other jarfiles, so the solution has to transitively follow manifest references.

P.S. I also added a ".verbose()" method so you can debug what's happening in future.

[tsujamin]

Unfortunately the issue still exists on HEAD: http://pastebin.com/YuCA6Ref

The first call is using the "Class-Path:" method, while the second is using "-cp". Both end up with the same classpath at runtime

[lukehutch]

Strange, can you please repeat, but call .verbose() on the
FastClasspathScanner object after constructing it and before calling
.scan()?

On Wed, Aug 19, 2015 at 3:58 PM, Benjamin Roberts [email protected]
wrote:

Unfortunately the issue still exists on HEAD: http://pastebin.com/YuCA6Ref

The first call is using the "Class-Path:" method, while the second is
using "-cp". Both end up with the same classpath at runtime

—
Reply to this email directly or view it on GitHub
#20 (comment)
.

[tsujamin]

http://pastebin.com/R9Xg9mFc

[lukehutch]

Thanks. For the Class-Path version, is the detected classpath (containing only rpsl4j-app-0.3-SNAPSHOT.jar) what you expect? i.e. is that jarfile the one that contains the manifest with the Class-Path lines?

[lukehutch]

PS I just checked in extra logging for the Class-Path parsing, it would be great if you could please capture a verbose log again so I can get more insight into where this is failing. Thanks!

[tsujamin]

rpsl4j-app-0.4-SNAPSHOT.jar is the one with the manifest containing all the package dependencies.

The current HEAD appears to correctly read the manifest for Class-Path jar's, however it attempts to load them relative to the vm's pwd instead of relative to the jar containing the manifest: http://pastebin.com/NZThM2Va

[lukehutch]

Ah! I think you nailed it. Please see the latest commit. I don't have your
jar for reference, so I might be off by one directory level. Please test,
and if it's not correct but the fix is simple, it would be great if you
could send me a patch to prevent further round-trips. Thanks for your
persistence!

On Wed, Aug 19, 2015 at 10:25 PM, Benjamin Roberts <[email protected]

wrote:

rpsl4j-app-0.4-SNAPSHOT.jar is the one with the manifest containing all
the package dependencies.

The current HEAD appears to correctly read the manifest for Class-Path
jar's, however it attempts to load them relative to the vm's pwd instead
of relative to the jar containing the manifest:
http://pastebin.com/NZThM2Va

—
Reply to this email directly or view it on GitHub
#20 (comment)
.

[tsujamin]

yep 100% works now, thanks for adding support for this! :)

[lukehutch]

Excellent! That makes me happy, figuring out the classpath (beyond the simple java.lang.classpath) has always been a mystery to me, and working through this bug has taught me a heck of a lot about the internals. Thanks especially for all the informative links you sent about different potential solutions to this problem.

I made one more checkin, which shouldn't affect you, but I want to be sure (I disabled scanning of Java extension classes, which almost nobody should ever need). If you could please test just one more time, I'll push out a new release, which you'll then be able to depend upon using Maven. Sorry for the one additional request.

[lukehutch]

PS since I have your attention, I don't suppose you have any insights into bug #18? It's possible the fix to this bug #20 will address that too, but I'm not sure.

[lukehutch]

The Maven tests were failing for me too (re. your comment about having to add "-Dmaven.test.skip=true"), and this uncovered another curiosity about classpath handling: that Maven does not set all the necessary classloader URLs (or doesn't expose its classloader), but it does set java.class.path (so I need to merge from both classpath sources). This fix is checked in.

I'm pretty sure my latest changes won't break anything for you, so I pushed out version 1.8.0 to Maven Central. Please let me know if it's still working for you though.

Thanks again for your help, your bug report inspired a significant improvement to the way FastClasspathScanner works!

[tsujamin]

Yeah I've only been using maven for ~6 months, so I wouldn't have been of much help.

And the latest version still works for me. Thanks!

[lukehutch]

Just wanted to circle back on this. I discovered that the classpath entries in Class-Path attributes of manifest files are supposed to be URLs/URIs, not file paths. I have done a bunch of work recently to try to properly support file:// URIs. It would be great if you could please test the latest version again with your setup. Thanks!