Classpath resource APIs

[pathikrit]

Classpath resource APIs

[nscarcella]

Any progress on this? I would love to browse jar files this way...

[pathikrit]

@lu4nm3 : This month

[pathikrit]

see: https://github.com/spring-projects/spring-framework/blob/master/spring-core/src/main/java/org/springframework/util/ResourceUtils.java#L170

[acrisci]

Is anyone working on this? I can take it up if not.

[pathikrit]

@acrisci : Feel free to pick this up. Before you begin, can you outline your plan? Like what APIs would you like to add?

[acrisci]

I am kind of new to the jvm and scala and my knowledge of what makes for a good scala library interface is lacking so I might need some help with this :)

My use case I think is a rather common one. I have an application which I package as a jar and I want to get the contents of some file in the resources directory such as a config file or something.

When I simply run sbt run from the project root, this works fine:

val f = File(getClass.getResource("/some-file.txt").toURI)
println(f.contentAsString)

But when the application is packaged as a jar (such as with sbt assembly), it does not work:

Exception in thread "main" java.nio.file.FileSystemNotFoundException
at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
at java.nio.file.Paths.get(Paths.java:143)
at better.files.File$.apply(File.scala:543)
at example.Main$.main(Main.scala:9)
at example.Main.main(Main.scala)

The URI in the first case is the path to the actual file on the disk. In the second case, it is this:

jar:file:/home/myuser/projects/example/target/scala-2.11/default-assembly-0.1-SNAPSHOT.jar!/resource.txt

This obviously doesn't exist in the file system, so I don't believe it is possible to turn it into a proper better files File (although it would be nice if we could do that to reuse the interface).

So what I will propose first is a new class called Resource which can be made like so:

val resource = resource"/some-file.txt"
val resource = Resource("/some-file.txt")

That will load the file from the resource path similar to what you would expect could be gotten with getClass.getResource("/some-file.txt").

Loading resources from other classes would also be useful, and maybe that could be accomplished with a method such as Resource("/some-file.txt", classOf[SomeClass]) or something similar.

The Resource class would then implement as many methods from the File class as makes sense (given that the resource may not actually be a file in the file system). Useful methods would be contentAsString(), copyTo(), bytes(), lineIterator(), and others. Obviously only a small number of methods that are available on the File are going to make sense in this context though.

Then we can do cool stuff like this:

val myConfig = resource"/path-to-config.txt".contentAsString
// parse config somehow

instead of having to look up how to do that another way on google over and over again like i do now.

[pathikrit]

@acrisci: I think for all practical purposes, resources are just vanilla files that happen to live in a special place and loaded in a special way (i.e. inside the app's jar). Thus, if we classify all operations on a file into 2 camps - reads (e.g. .bytes or .size or .contentAsString) vs writes (e.g. .move, .delete), resources are just files that only supports the operations from the read camp.

Thus we can do this then:

trait ReadOperations {
   def contentAsString: String
   def bytes: Iterator[Byte]
   ....
}

trait WriteOperations {
   def delete(): Unit
   def moveTo(dest: File): File
   ....
}


class File extends ReadOperations with WriteOperations
class Resource extends ReadOperations

The above looks elegant but I ran into problems with certain read operations (e.g. listing a directory) which is not easy for Resources and thus I shelved progress on this issue. Maybe .list() should throw UnsupportedOperationException for Resources? But, that is pretty evil!

Would be happy to hear your thoughts on this.

[acrisci]

This blog post gives a fairly reasonable explanation on how to do that (untested). I think it would be less evil to throw a kind of NotImplementedException in cases like that so that people might be annoyed enough to come up with an implementation. But that solution still seems a bit ugly to me.

I'm starting to think that the best solution for this would be for java NIO to implement the jar file system but I'm not sure how likely that is to happen.

Another solution might be to divide the operations based on what can be generalized to any stream, and what is specifically a file system operation.

trait StreamOperations {
   def contentAsString: String
   def bytes: Iterator[Byte]
   ....
}

trait FileSystemOperations {
   def delete(): Unit
   def moveTo(dest: File): File
   ....
}
class File extends StreamOperations with FileSystemOperations
class Resource extends StreamOperations

Then we don't have to worry about nasty implementations of file system operations which aren't even that useful for resources to begin with.

This also can be reused for other streams in the future (STDIN, STDOUT).

[pathikrit]

SGTM. As a first attempt, just add a new Resource.scala class. I can work on the refactor of common APIs into separate traits.

[lu4nm3]

Is there a timeline for when this will be released?