I can't preprocess java-small

[jingranburangyongzhongwen]

I can't preprocess java-small

[urialon]

Hi,
How many cores and memory do you have on that machine?
I think that all training examples timed out, and there were no examples left for preprocessing.

Why do you want specifically to train on Java-small? can you use the Java-large version which is already preprocessed?

[jingranburangyongzhongwen]

I'm a soon-to-graduate undergraduate.
I want to make a few slight modifications.
The reason why I didn't use the Java-large version was that it's too big for me because of the restriction of computing resources.
Here is some information about my computer.

CPU: Intel® Core™ i7-8700 CPU @ 3.20GHz × 12
GPU: GeForce GTX 1070/PCIe/SSE2
RAM: 15.6 GiB

Besides, the error occurred after I ran bash preprocess.sh for a few seconds.

[urialon]

If it happens in a few seconds, it seems that some error occurs.
Do you have JDK?

Try to run the java jar directly to see what happens:

java -Xmx100g -XX:MaxNewSize=60g -cp JavaExtractor/JPredict/target/JavaExtractor-0.0.1-SNAPSHOT.jar JavaExtractor.App --max_path_length 8 --max_path_width 2 --dir /home/kf/code2seq/java-small/training/gradle --num_threads 6

[jingranburangyongzhongwen]

I got these messages:

(tensorflow-gpu) kf@lkf-P7xxTM1:~/code2seq$ java -Xmx100g -XX:MaxNewSize=60g -cp JavaExtractor/JPredict/target/JavaExtractor-0.0.1-SNAPSHOT.jar JavaExtractor.App --max_path_length 8 --max_path_width 2 --dir /home/kf/code2seq/java-small/training/gradle --num_threads 6
Exception in thread "main" java.lang.UnsatisfiedLinkError: /home/kf/anaconda3/envs/tensorflow-gpu/jre/lib/amd64/libnio.so: libnet.so: cannot open shared object file: No such file or directory
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1937)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843)
	at java.lang.Runtime.loadLibrary0(Runtime.java:870)
	at java.lang.System.loadLibrary(System.java:1122)
	at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:573)
	at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:571)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.nio.fs.UnixNativeDispatcher.<clinit>(UnixNativeDispatcher.java:571)
	at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67)
	at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39)
	at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46)
	at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39)
	at sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56)
	at sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
	at java.lang.Class.newInstance(Class.java:442)
	at sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48)
	at sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89)
	at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98)
	at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96)
	at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90)
	at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
	at java.nio.file.Paths.get(Paths.java:84)
	at JavaExtractor.App.extractDir(App.java:40)
	at JavaExtractor.App.main(App.java:32)

I haven't found a good solution yet.
Besides, I think I have Java:

(tensorflow-gpu) kf@lkf-P7xxTM1:~/code2seq$ java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

[urialon]

It's a problem with your java installation I think...
Maybe try to do the preprocessing outside of the anaconda environment and see if it resolves.

See here:
https://stackoverflow.com/questions/480764/linux-error-while-loading-shared-libraries-cannot-open-shared-object-file-no-s

[jingranburangyongzhongwen]

I did the preprocessing outside of the anaconda environment and it seemed that the problem had been solved.
I really appreciate your help and your patience.

[urialon]

Sure, no problem