Native I/O access for java.
Check out the examples for more information.
UNIX domain sockets (AF_UNIX) for Java
License: Apache License 2.0
Native I/O access for java.
Check out the examples for more information.
I'm using this lib to interact with hostapd through a unix datagram socket, the problem is that when I send a request to the server the server doesn't respond me and throw the error Transport endpoint is not connected
on sendto function, if UnixDatagramChannel is in blocking mode when calling receive
the code wait for something that doesn't come, if in non-blocking mode return 0
User's should be free to call Native.socket(domain, type, protocol)
etc. without having to copy the class and modify it.
For jruby/jruby#940, we need a factory method or constructor method exposed that can take an existing fd.
Facing the following exception when executed the jar file in AIX machine having IBM java 8 using jnr v0.33 -
jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr.ffi.provider.jffi.platform .ppc64.aix.TypeAliases
Exception in thread "main" java.lang.RuntimeException: invalid type: ssize_t
at jnr.ffi.provider.BadType.getNativeType(BadType.java:43)
at jnr.ffi.provider.jffi.InvokerUtil.getMethodResultNativeType(InvokerUtil.java:217)
at jnr.ffi.provider.jffi.InvokerUtil.getResultType(InvokerUtil.java:143)
at jnr.ffi.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:162)
at jnr.ffi.provider.jffi.AsmLibraryLoader.loadLibrary(AsmLibraryLoader.java:89)
at jnr.ffi.provider.jffi.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:44)
at jnr.ffi.LibraryLoader.load(LibraryLoader.java:325)
at jnr.unixsocket.Native.(Native.java:80)
at jnr.unixsocket.UnixServerSocketChannel.(UnixServerSocketChannel.java:43)
at jnr.unixsocket.UnixServerSocket.(UnixServerSocket.java:32)
at jnr.unixsocket.UnixServerSocketChannel.open(UnixServerSocketChannel.java:53)
at com.testing.jsrv.test.main(test.java:70)
I've writing a Clojure wrapper for Watchman at marcomorain/clj-watchman using jnr-unixsocket, and I've run into a problem that I was hoping you could help with @headius.
Watchman is a program that allows the user to set a watch on a directory, and it will alert the user when a file has changed.
You issue commands to Watchman by writing commands, formatted as JSON followed by a newline, into a unix socket, and then you can read results back, as JSON, from the socket. In this simple use-case the command and result are issued and returned in lockstep, and everything works fine.
Things get more complicated once you add a "watch" to a directory. When any file in the directory is modified, Watchman will write an event to the socket. So I set up a thread to call read on the socket in a loop to listen for these events. From my main thread I then write commands into the socket. My background thread reads results and will take the appropriate action.
The problem is that this causes a deadlock – my reader thread starts, and calls read on the socket:
"Thread-7" #18 daemon prio=5 os_prio=31 tid=0x00007ffacec91000 nid=0x3307 runnable [0x0000000130436000]
java.lang.Thread.State: RUNNABLE
at com.kenai.jffi.Foreign.invokeN3O1(Native Method)
at com.kenai.jffi.Invoker.invokeN3(Invoker.java:1061)
at jnr.enxio.channels.Native$LibC$jnr$ffi$1.read(Unknown Source)
at jnr.enxio.channels.Native.read(Native.java:95)
at jnr.enxio.channels.NativeSocketChannel.read(NativeSocketChannel.java:70)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:59)
- locked <0x0000000772292ff8> (a java.lang.Object)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
- locked <0x00000007722bfe28> (a sun.nio.ch.ChannelInputStream)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
- locked <0x00000007722bf818> (a java.io.InputStreamReader)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
- locked <0x00000007722bf818> (a java.io.InputStreamReader)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:313)
at com.marcomorain.watchman$read_response.invoke(watchman.clj:15)
at com.marcomorain.watchman$connect$fn__2408.invoke(watchman.clj:52)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:745)
This is doing a blocking read as intended. No commands have been issues, so there are no responses, so this thread blocks. This is correct behaviour.
Next I issue a command on my main thread. This tries to acquire a lock on the same socket, and I get deadlock:
"nREPL-worker-0" #16 daemon prio=5 os_prio=31 tid=0x00007ffacde14800 nid=0x5d03 waiting for monitor entry [0x000000012f98e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.nio.channels.Channels.writeFully(Channels.java:96)
- waiting to lock <0x0000000772292ff8> (a java.lang.Object)
at java.nio.channels.Channels.access$000(Channels.java:61)
at java.nio.channels.Channels$1.write(Channels.java:174)
- locked <0x00000007722b8f30> (a java.nio.channels.Channels$1)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
- locked <0x00000007722b92a8> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush(BufferedWriter.java:254)
- locked <0x00000007722b92a8> (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush(PrintWriter.java:320)
- locked <0x00000007722b9280> (a java.io.BufferedWriter)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:313)
at com.marcomorain.watchman$write_command.invoke(watchman.clj:21)
at com.marcomorain.watchman$execute_command.invoke(watchman.clj:26)
at com.marcomorain.watchman$log_level.invoke(watchman.clj:69)
at user$eval2419.invoke(form-init1868161862316182594.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6703)
at clojure.lang.Compiler.eval(Compiler.java:6666)
at clojure.core$eval.invoke(core.clj:2927)
at clojure.main$repl$read_eval_print__6625$fn__6628.invoke(main.clj:239)
at clojure.main$repl$read_eval_print__6625.invoke(main.clj:239)
at clojure.main$repl$fn__6634.invoke(main.clj:257)
at clojure.main$repl.doInvoke(main.clj:257)
at clojure.lang.RestFn.invoke(RestFn.java:1523)
at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__881.invoke(interruptible_eval.clj:67)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invoke(core.clj:624)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1862)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:51)
at clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__923$fn__926.invoke(interruptible_eval.clj:183)
at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__916.invoke(interruptible_eval.clj:152)
at clojure.lang.AFn.run(AFn.java:22)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
You can see both threads are locked on the object at 0x0000000772292ff8
.
Do you have any suggestions on how to work past this? – Is there a way to read from the socket without blocking writes?
Thanks in advance for any help you can offer!
The callstacks show the state of my code as of this commit: https://github.com/marcomorain/clj-watchman/tree/e6137c0b565541132ff9a58d9f30a092637b3976
Hi,
Is there any plan to support Z linux architecture ?
Such support would help us to use the fabric.io Docker maven plugin (See fabric8io/docker-maven-plugin#1213 )
Thanks
Hello!
When running on ppc64le, the test
fails correctly, expecting pid, but getting 1.linux x linux
myPid: 42170 # three real values
myUid: 1001
myGid: 1001
cc uid: 16383 # wrong client credentials
cc pid: 1
cc gid: -2070423904
sc uid: 16383 # samely wrong server credentials
sc pid: 1
sc gid: -2070421584
The values above are exemplar. They differs, canbe positive, but error is obvious.
I had debugged it into:
Which leads to jnr-ffi 's : https://github.com/jnr/jnr-ffi/blob/05cbacc8528e85b1dbff30be5b88cde4c6a9c6a7/src/main/java/jnr/ffi/Struct.java#L865 wehre it leaves my current knowlege a bit.
Suspicion is going to endians or basic size type. Will continue digging.
Note, I had run all jnr-* tests on this ppc64le machine and no other projects (including ffi) had any issues. Only this one unixsocket's CredentialsFunctionalTest
Good day!
Trying to use one of examples in unit test.
public class FrontTest {
@Rule
public final transient TemporaryFolder folder = new TemporaryFolder();
@Test
public void testRun() throws Exception {
// File path = this.folder.newFile("fubar.sock"); // 1
File path = new File("/tmp/fubar.sock"); // 2
path.deleteOnExit();
UnixSocketAddress address = new UnixSocketAddress(path);
UnixServerSocketChannel channel = UnixServerSocketChannel.open();
Selector sel = NativeSelectorProvider.getInstance().openSelector();
channel.configureBlocking(false);
channel.socket().bind(address); // 3
}
}
So, if file is created by line 2 - all is fine, no exceptions.
If I comment line 2 and uncomment line 1 there is and exception on line 3
java.io.IOException: bind failed: Address already in use
at jnr.unixsocket.UnixServerSocket.bind(UnixServerSocket.java:53)
at jnr.unixsocket.UnixServerSocket.bind(UnixServerSocket.java:43)
at foo.bar.FrontTest.testRun(FrontTest.java:51)
Do I miss something important?
Thank you!
I haven't found a way of shutting down a UnixServerSocket that's waiting for incoming sockets in accept(). It also doesn't respond thread interrupts?
The code looks a little like this - and the accept still hangs after the interupt.
Thread thread = new Thread(() -> {
UnixSocketAddress socketAddress = new UnixSocketAddress("/tmp/file.sock");
UnixServerSocket serverSocket = new UnixServerSocket();
serverSocket.bind(socketAddress);
while (true) {
UnixSocket sock = null;
try {
sock = serverSocket.accept();
// ... do stuff
} catch (Exception e) {
if (sock != null) {
sock.close();
}
}
System.out.println("I surrender");
}
});
thread.start();
Thread.sleep(2000);
thread.interrupt();
Thread.sleep(10000);
Hi,
When I use a wrapper PrintWriter over a UnixSocketChannel OutputStream, no exception is raised when I try to println(String) on the PrintWriter object while the socket is actually remotely closed.
To reproduce the issue, you need a simple client/server with a UnixSocketChanel.
Then a PrintWriter with :
PrintWriter w = new PrintWriter(Channels.newOutputStream(channel));
If you close the socket on the server side and then try to use the PrintWriter afterwards, no exception is raised.
I've seen that (PrintWriter)w.checkError() returns actually true, which shows the error. But I think a proper "SocketClosed" exception or something like that would be nice.
Thank you for your help
When I run the UnixServer sample on windows, it reports java.lang.UnsatisfiedLinkError: ÕҲ »µ½ ָ¶¨µ error.
The path of the UnixServer sample is jnr-unixsocket/src/test/java/jnr/unixsocket/example/UnixServer.java
Exception in thread "main" java.lang.UnsatisfiedLinkError: ÕҲ»µ½ָ¶¨µ
at jnr.ffi.provider.jffi.AsmRuntime.newUnsatisifiedLinkError(AsmRuntime.java:40)
at jnr.unixsocket.Native$LibC$jnr$ffi$0.socket(Unknown Source)
at jnr.unixsocket.Native.socket(Native.java:92)
at jnr.unixsocket.UnixServerSocketChannel.(UnixServerSocketChannel.java:38)
at jnr.unixsocket.UnixServerSocket.(UnixServerSocket.java:32)
at jnr.unixsocket.UnixServerSocketChannel.open(UnixServerSocketChannel.java:48)
at com.delilegal.IPC.UnixServer.main(UnixServer.java:23)
is there any plan to support DGRAM socket
I noticed that when reading from a Unix socket, if the return value of a native read
call is 0
then it is returned to the caller as -1
. Relevant code:
https://github.com/jnr/jnr-unixsocket/blob/master/src/main/java/jnr/enxio/channels/Common.java#L59
I'm left curious as to why particularly given that I'm having a problem with half-closing Unix sockets. My understanding is that a native read
could return zero on receipt of a FIN
. I'm not sure if my problem is related, but I'd still like to understand why the translation is performed.
Thanks.
When using JDK1.8, jnr-unixsocket v0.22 and below correctly sets osgi.ee;filter:="(&(osgi.e e=JavaSE)(version=1.8))" but v0.23 and above forces osgi.ee;filter:="(&(osgi.e e=JavaSE)(version=9.0))".
Given that JDK1.8 is still supported, can jnr-unixsocket v0.25-SNAPSHOT have its dependencies adjusted so that osgi.ee;filter:="(&(osgi.e e=JavaSE)(version=1.8))" persists and one is not forced to upgrade to JDK9?
The dependency involved that forces osgi.ee;filter:="(&(osgi.e e=JavaSE)(version=9.0))" is described here:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=537751
.. specifically "org.objectweb.asm" which is a transitive dependency of jnr-enxio.
It is "org.objectweb.asm" that forces JDK9.
UnixSocketChannel#stateLock
is not used. It certainly isn't need for things like #isConnected
but things like #finishConnect
don't look particularly thread-safe. Concurrent access to #state
seems to be possible since it's marked volatile
.
I've been working on adding this to jnr-unixsocket. I have a candidate ready, which I'll send as a pull request. Please let me know if it's acceptable to you. One thing I'm not sure about is your preferred handling of the feature being unsupported. In the current code, I am using UnsupportedOperationException for this case.
Expected:
jnr-unixsocket
and all of its dependencies should have compatible licenses.
Actual:
jnr-unixsocket
is Apache 2 license but it depends on jnr-posix
which is GPL. These are not compatible.
Reason:
The web3j library depends on this project.
I'm trying to make a server using the UnixServer example as a starting point. On my Mac, when I send more than two buffers worth to it, it just hangs and no longer reads more data. Here are two example commands that should work (and do work on Linux).
➜ ~ cat .emacs | socat UNIX-CONNECT:/tmp/fubar.sock -
➜ ~ cat .emacs | nc -U /tmp/fubar.sock
All I'm doing is running the UnixServer class straight out of the box. I'm using version jnr-unixsocket 0.28.
The funny thing is that if I make the ByteBuffer smaller than 1024 (like 512), it hangs after just 1 buffer read. All buffer sizes work fine on Linux.
Need it. Not sure how closely to integrate with Java's datagram support, but such integration may require us to finally align our socket classes with Java's.
Please release this project and it's dependency, jnr-enxio. We are using it as a dependency in our own project that we would like to release soon.
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.objectweb.asm.ClassVisitor, but class was expected
at jnr.ffi.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:101)
at jnr.ffi.provider.jffi.AsmLibraryLoader.loadLibrary(AsmLibraryLoader.java:86)
at jnr.ffi.provider.jffi.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:44)
at jnr.ffi.LibraryLoader.load(LibraryLoader.java:392)
This change in jnr-enxio shows how we'd like to start getting the C library, as a utility of jnr-ffi: jnr/jnr-enxio#12
We need to do the same in jnr-unixsocket.
The following code sends hello
from a client thread to a server thread over a UnixSocketChannel
, and the server prints out the character codes (pardon my Scala)
package mill.clientserver
import java.nio.channels.Channels
import jnr.unixsocket.{UnixServerSocketChannel, UnixSocketAddress, UnixSocketChannel}
object Main {
val tmp = java.nio.file.Files.createTempDirectory("") + "/sock"
val addr = new UnixSocketAddress(tmp)
def main(args: Array[String]): Unit = {
new Thread(() => server()).start()
new Thread(() => client()).start()
}
def server() = {
val io = UnixServerSocketChannel.open()
io.socket().bind(addr)
val in = Channels.newInputStream(io.accept())
while(true) println(in.read())
}
def client() = {
Thread.sleep(1000)
val ioSocket = UnixSocketChannel.open(addr)
val in = Channels.newInputStream(ioSocket)
val out = Channels.newOutputStream(ioSocket)
// new Thread(() => println(in.read())).start()
Thread.sleep(100)
out.write("hello\n".getBytes)
}
}
If you uncomment the new Thread(() => println(in.read())).start()
, the out.write("hello\n".getBytes)
appears to never make it to the server
thread, and nothing is printed. It appears blocking on the UnixSocketChannel
's in.read()
also disables out.write
. Not sure if this is expected or a bug, but another socket library I tried https://github.com/sbt/ipcsocket appears to allow writes while some other thread is read-blocked
UnixServerSocketChannel.accept()
should return null in non-blocking mode when there are no more client channels to accept. At the moment it throws a Exception which is wasteful as this is expensive.
After closing the socket from a different thread, channel write call was successful and writing to a recycled file descriptor belonged to a different connection. Here is the example tested on Linux. I couldn't reproduce the problem with TCP sockets.
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;
import jnr.unixsocket.UnixSocketAddress;
import jnr.unixsocket.UnixSocketChannel;
public class CloseTest {
public static void main(String[] args) {
try {
SocketChannel c1 = UnixSocketChannel.open(new UnixSocketAddress("/tmp/j1.sock"));
// SocketChannel c = SocketChannel.open(new InetSocketAddress("[::1]", 8000));
c1.write(ByteBuffer.wrap("hello".getBytes("utf-8")));
c1.close();
SocketChannel c2 = UnixSocketChannel.open(new UnixSocketAddress("/tmp/j2.sock"));
c1.write(ByteBuffer.wrap("world".getBytes("utf-8")));
c2.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
Listen on different sockets
nc -Ul /tmp/j1.sock
nc -Ul /tmp/j2.sock
Apologies if the description is inaccurate.
We use unix sockets with Jetty, which are powered by jnr-unixsocket. I filed an issue on the jetty project about seeing hundreds of exceptions related to a single connection (jetty/jetty.project#4865)
It seems that the maintainers think there is an issue in jnr-unixsocket that is causing the UnixSocketEndPoint attachment to the PollSelectionKey to be null?
Let me know if there is any additional information I can provide, I appreciate any help with this issue
I'm attempting to pass a file descriptor between two applications via dbus-java, but since this is not currently implemented correctly it seems as though I need to add support to jnr-unixsocket in order to properly pass the file descriptors between the processes. What's the best way to extend the classes in order to accomplish this task? It seems like this requires a completely new class, since the current implementation is only worried about writing messages using the write
system call.
The implementation of Common.write(ByteBuffer[] srcs, int offset, int length)
is faulty in case the channel is set in non-blocking mode.
When in non-blocking mode, there is no guarantee that the write()
actually writes any byte, so the loop over the buffers must exit as soon as the write is incomplete.
Integer values passed to setsockopt calls and retrieved from getsockopt calls are converted between Java and native code using ByteOrder.BIG_ENDIAN in Native.java, which results in incorrect values being passed for little-endian CPU architectures such as Intel processors.
For example, we were invoking
UnixSocketChannel.setOption(UnixSocketOptions.SO_SNDBUF, 262144);
... which ends up calling
Native.setsockopt.(fd, SocketLevel.SOL_SOCKET, UnixSocketOptions.SO_SNDBUF, 262144);
In Native.java, it converts the send buffer size using big endianness (regardless of the platform architecture):
ByteBuffer buf = ByteBuffer.allocate(4); buf.order(ByteOrder.BIG_ENDIAN); buf.putInt(optval).flip(); return libsocket().setsockopt(s, level.intValue(), optname.intValue(), buf, buf.remaining());
Running this on Intel/Linux ends up setting the send buffer size to 1024 byte (instead of 262144), as can be seen with strace:
18:31:15.483698 setsockopt(226, SOL_SOCKET, SO_SNDBUF, [1024], 4) = 0 <0.000032>
Obviously this undersized buffer could lead to severe performance degradation.
The same hard-coded endianness is found in getsockopt calls.
A simple fix is to replace
buf.order(ByteOrder.BIG_ENDIAN);
with
buf.order(ByteOrder.nativeOrder());
which works fine on any architecture (confirmed with strace on Intel/Linux).
Sometimes this smoke test fails, annoyingly often when I go to release.
Expected:
[INFO] --- exec-maven-plugin:1.4.0:exec (execute) @ jnr-unixsocket ---
connected to [family=PF_UNIX path=/tmp/fubar.sock]
read from server: blah blah
SUCCESS
Actual:
[INFO] --- exec-maven-plugin:1.4.0:exec (execute) @ jnr-unixsocket ---
Exception in thread "main" java.io.IOException: No such file or directory
at jnr.unixsocket.UnixSocketChannel.doConnect(UnixSocketChannel.java:130)
at jnr.unixsocket.UnixSocketChannel.connect(UnixSocketChannel.java:139)
at jnr.unixsocket.UnixSocketChannel.open(UnixSocketChannel.java:68)
at jnr.unixsocket.example.UnixClient.main(UnixClient.java:46)
Probably should look into why this happens. Re-running usually works.
Currently the default Unix Socket Type is set to SOCKET_STREAM
. I have a use case for SOCKET_SEQPACKET
, but unfortunately have to edit the source code myself in order to achieve this.
Is there perhaps a chance that the static connect
functions could be overloaded for the socket type to be passed as an option?
Hi, Thanks for awesome library ✨
I facing issue when use docker-maven-plugin.
Execution default-cli of goal io.fabric8:docker-maven-plugin:0.38-SNAPSHOT:build failed:
An API incompatibility was encountered while executing io.fabric8:docker-maven-plugin:0.38-SNAPSHOT:build:
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
fabric8io/docker-maven-plugin#1257 (comment)
Questions
When writing data asynchronously to the point of hitting flow control, the UnixSocketChannel.write
method correctly returns 0 bytes written, but it incorrectly updates the ByteBuffer
position so that the bytes are consumed.
The problem is demonstrated by this class in the eclipse jetty project. The test is a bit verbose, as it first confirms that client and server channels are working asynchronously. It then writes large buffers until flow controlled and the bug is indicated by the incorrect remaining value of the buffer after the write of 0 bytes.
I get the following output:
serverChannel=jnr.unixsocket.UnixServerSocketChannel@4dfa3a9d,
client=jnr.unixsocket.UnixSocketChannel@3fee9989 connected=true pending=false
serverSelected=1 [jnr.enxio.channels.PollSelectionKey@146ba0ac]
key=jnr.enxio.channels.PollSelectionKey@146ba0ac/SERVER c=false a=true r=false w=false
server=jnr.unixsocket.UnixSocketChannel@7085bdee connected=true pending=false
serverSelected=0 []
server read=0
clientSelected=0 []
client wrote=5
serverSelected=1 [jnr.enxio.channels.PollSelectionKey@73ad2d6]
key=jnr.enxio.channels.PollSelectionKey@73ad2d6/server c=false a=false r=true w=false ch=jnr.unixsocket.UnixSocketChannel@7085bdee
server read=5 'Hello'
clientSelected=0 []
server wrote=5
clientSelected=0 []
clientSelected=1 [jnr.enxio.channels.PollSelectionKey@2f686d1f]
key=jnr.enxio.channels.PollSelectionKey@2f686d1f/client c=false a=false r=true w=false ch=jnr.unixsocket.UnixSocketChannel@3fee9989
client read=5 'Ciao!'
So far so good.... now it gets strange...
server wrote 32768/32768 remaining=0
server wrote 32768/32768 remaining=0
server wrote 32768/32768 remaining=0
server wrote 32768/32768 remaining=0
server wrote 32768/32768 remaining=0
server wrote 32768/32768 remaining=0
server wrote 32768/32768 remaining=0
server wrote 0/32768 remaining=0
BUG!!!!!!!!!!!!!!!!
server wrote 229376 before flow control
clientSelected=1 [jnr.enxio.channels.PollSelectionKey@2f686d1f]
key=jnr.enxio.channels.PollSelectionKey@2f686d1f/client c=false a=false r=true w=false ch=jnr.unixsocket.UnixSocketChannel@3fee9989
client read=32 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
I can be naive, but this looks like a reasonable thing to do. Can anyone explain why wasn't this done?
From a first glance this cannot be done because of two reasons:
UnixServerSocketChannel
extends NativeServerSocketChannel
, while NativeServerSocketChannel
cannot extend ServerSocketChannel
, because it does not bind to socket-addresses but file-descriptors. Could UnixServerSocketChannel
extend ServerSocketChannel
directly while sharing code with NativeServerSocketChannel
in a different way?ServerSocketChannel#validOps()
are only SelectionKey.OP_ACCEPT
, the UnixServerSocketChannel#validOps()
are SelectionKey.OP_ACCEPT | SelectionKey.OP_READ
for some reasons I don't understand. I have tried changing it to SelectionKey.OP_ACCEPT
, but the tests still pass (so do NativeServerSocketChannel
tests). Why is SelectionKey.OP_READ
required there?I am using jnr-unixsocket version: 0.22
jnr-ffi', version: '2.1.5'.
when i run my project on ppc64le environment I get an error for unsupported architecture.
Error:-
Unhandled architecture/OS: ppc64le-Linux
at org.newsclub.net.unix.NarSystem.getAOLs(NarSystem.java:110)
Can you please point me to the appropriate versions.
Hi, I have encountered an issue on Solaris 5.11 on SPARC-T5 machines using Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
. I get this exception:
java.net.SocketException
at jnr.unixsocket.UnixSocket.setSoTimeout(UnixSocket.java:277)
Caused by: java.io.IOException: Option not supported by protocol
at jnr.unixsocket.Common.setSocketOption(Common.java:148)
at jnr.unixsocket.UnixSocketChannel.setOption(UnixSocketChannel.java:322)
at jnr.unixsocket.UnixSocket.setSoTimeout(UnixSocket.java:275)
... 7 more
when I try to change a socket option such as timeout. code works fine otherwise on Linux-amd64 and AIX-POWER.
The default socket timeout is a ridiculously long time. Is there any way to change it?
It's difficult to use this library in conjunction with the rest of the Java ecosystem because the standard abstract socket classes are not implemented. Was there a specific reason this isn't done? In other words, if I tried to do this myself, is there something non-obvious that would block me?
For reference, http://code.google.com/p/junixsocket/ supports the non-NIO socket interfaces.
I use jnr-unixsocket v0.18 under linux like this:
File path = new File("/var/run/test.sock");
UnixSocketAddress address = new UnixSocketAddress(path);
UnixSocketChannel channel = UnixSocketChannel.open(address);
but get an exception:
Exception in thread "main" java.io.IOException: Protocol wrong type for socket
at jnr.unixsocket.UnixSocketChannel.doConnect(UnixSocketChannel.java:127)
at jnr.unixsocket.UnixSocketChannel.connect(UnixSocketChannel.java:136)
at jnr.unixsocket.UnixSocketChannel.open(UnixSocketChannel.java:68)
If you create a UnixSocket
, set a read timeout and then:
read()
on one threadclose()
on another thread...the behaviour is different on OS X and Linux.
On OS X, the read is cancelled as soon as close()
is called and so close()
returns immediately. On Linux, the read blocks until the read timeout is reached, and then close()
returns.
This makes interrupting read operations impossible, and is the root cause of square/okhttp#4233.
I have created a test case that fails on Linux (but passes on OS X) in charleskorn@b0981ae.
As far as I can tell, jnr-ffi is fine with array-backed or direct buffers, so we can avoid the extra allocation in those cases.
The example UnixClient
appears to fail sporadically on Darwin. This frequently happens during release builds, but exactly once followed by a success. It could be a simple filesystem cleanup issue.
[INFO] --- exec-maven-plugin:1.4.0:exec (execute) @ jnr-unixsocket ---
Exception in thread "main" java.io.IOException: No such file or directory
at jnr.unixsocket.UnixSocketChannel.doConnect(UnixSocketChannel.java:130)
at jnr.unixsocket.UnixSocketChannel.connect(UnixSocketChannel.java:139)
at jnr.unixsocket.UnixSocketChannel.open(UnixSocketChannel.java:68)
at jnr.unixsocket.example.UnixClient.main(UnixClient.java:46)
Hi,
I cloned the project and built the project with JDK 8. And I ran the unit tests with JDK 17. All but one unit test BasicFunctionalityTest
stalls when the client tries to read from the channel.
We are using a similar pattern in our project's Unit tests and we see the same issue.
My Setup is as follows:
Could you help me with this? We are currently migrating our project to JDK 17, as a result, bumped up the version from 0.18
to 0.38.21
.
Older versions fail with the following error with JDK 17:
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider at jnr.ffi.provider.InvalidRuntime.newLoadError(InvalidRuntime.java:101) at jnr.ffi.provider.InvalidRuntime.findType(InvalidRuntime.java:42) at jnr.ffi.Struct$NumberField.<init>(Struct.java:872) at jnr.ffi.Struct$Unsigned8.<init>(Struct.java:1113) at jnr.unixsocket.SockAddrUnix$BSDSockAddrUnix.<init>(SockAddrUnix.java:187) at jnr.unixsocket.SockAddrUnix.create(SockAddrUnix.java:174) at jnr.unixsocket.UnixSocketAddress.<init>(UnixSocketAddress.java:47) at jnr.unixsocket.UnixSocketPair.<init>(UnixSocketPair.java:25) at jnr.unixsocket.BasicFunctionalityTest.setUp(BasicFunctionalityTest.java:36) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type
POINTER, original error message follows: java.lang.UnsatisfiedLinkError: Unable to execute or load jffi binary stub from
/var/folders/v8/fpwjm10x3mxck_1j2_1_13qm0000gn/T/. Set
TMPDIRor Java property
java.io.tmpdir to a read/write path that is not mounted "noexec".
Can't load library: /Users/xxx/IdeaProjects/jnr-unixsocket/jffi833868370737744349.dylib at com.kenai.jffi.internal.StubLoader.tempLoadError(StubLoader.java:424) at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:409) at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:278) at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:487)
In its MANIFST.MF, jnr-unixsocket
exports package jnr.enxio.channels
. However, the same package is exported by its dependency bundle jnr-enxio.
This results in ClassNotFoundExceptions in OSGi environments because the class loader will bind exclusively to one bundle and thus fail to load classes from the other bundle. See section 3.6.6 in the OSGi spec [1], for instance.
A clean fix would be splitting the classes in jnr-unixsocket -> jnr.enxio.channels
and jnr-enxio -> jnr.enxio.channels
into two different packages, e.g. jnr.enxio.channels.api
and jnr.enxio.channels.impl
.
A workaround without modifying package names could be to use Require-Bundle: jnr-enxio
in MANIFEST.MF of jnr-unixsocket
.
[1] https://osgi.org/specification/osgi.core/7.0.0/framework.module.html
I'm using jnr-unix-socket
in the jetty servlet container and am having problem with connect (see jetty/jetty.project#1281). In summary, if we have a multiple clients connecting at the same time, sometimes a connection attempt is lost and just hangs.
This can be demonstrated by having two or more shells executing the loop:
while :; do curl -v --unix-socket /tmp/jetty.sock http://localhost/ ; done
and soon enough all but one will hang with:
* Trying /tmp/jetty.sock...
this is with 0.18. I've tried both blocking and async connections and I see the same result.
I've not yet nailed this down as a JNR bug and it is probably that there is something wrong in jetty's usage. But I thought I'd open this issue to see if you can shed any light on it. Meanwhile I will work on a minimal reproduction.
I tried example server on Ubuntu 14.04(Release env) and MacOS(Develop env).
On Mac, NIO selection is wait for asleep until message received.
https://github.com/jnr/jnr-unixsocket/blob/master/src/test/java/jnr/unixsocket/example/UnixServer.java#L47
But on Ubuntu, NIO selection is not blocked and loop infinitely.
https://github.com/jnr/jnr-unixsocket/blob/master/src/test/java/jnr/unixsocket/example/UnixServer.java#L47-L67
In my guess, Is the reason that Ubuntu is not BSD?
https://github.com/jnr/jnr-enxio/blob/master/src/main/java/jnr/enxio/channels/NativeSelectorProvider.java#L55-L57
Please check whether I did mis-configured or not. Thanks.
I didn't find this project at first because I was looking for "unix domain sockets" (the official term) and neither the project description nor the README contain this string. AF_UNIX
is also a good search term.
A blocking call to accept cannot be interrupted, not even if the channel is closed. This is demonstrated by the following code:
import java.io.File;
import java.io.IOException;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import jnr.unixsocket.UnixServerSocketChannel;
import jnr.unixsocket.UnixSocketAddress;
public class UnixSocketAcceptInterrupt
{
public static void main(String... args) throws Exception
{
File file = new File("/tmp/test.sock");
if (file.exists())
file.delete();
final UnixServerSocketChannel channel = UnixServerSocketChannel.open();
channel.socket().bind(new UnixSocketAddress(file));
final AtomicBoolean run = new AtomicBoolean(true);
final CountDownLatch start = new CountDownLatch(1);
final CountDownLatch complete = new CountDownLatch(1);
Thread accept = new Thread()
{
@Override public void run()
{
start.countDown();
try
{
while(run.get())
{
try
{
channel.accept();
System.err.println("accepted");
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
finally
{
complete.countDown();
}
}
};
accept.setDaemon(true);
accept.start();
if (!start.await(5,TimeUnit.SECONDS))
System.err.println("Timeout waiting for start");
Thread.sleep(1000);
run.set(false);
accept.interrupt();
if (!complete.await(5,TimeUnit.SECONDS))
System.err.println("Timeout waiting for complete");
channel.close();
accept.interrupt();
if (!complete.await(5,TimeUnit.SECONDS))
System.err.println("Timeout waiting for complete after close");
}
}
Would be great if a stable JPMS module name can be provided, as simply as adding Automatic-Module-Name
to the manifest in the jar.
This will allow projects that are moving to proper JPMS modules to reference a stable JPMS name for jnr-unixsocket
in their module-info.java
.
I would suggest to not use com.github
as prefix, but perhaps just org.jnr
, but it's ultimately your choice.
Thanks!
I just stumbled over this while working on refactoring:
Fetching peer credentials is currently supported on Linux only. This can be improved.
#define SO_PEERCRED 0x1022 /* get connect-time credentials */
/* Read using getsockopt() with SOL_SOCKET, SO_PEERCRED */
struct sockpeercred {
uid_t uid; /* effective user id */
gid_t gid; /* effective group id */
pid_t pid;
};
LOCAL_PEERCRED Requested via getsockopt(2) on a SOCK_STREAM socket
returns credentials of the remote side. These will
arrive in the form of a filled in xucred structure,
defined in <sys/ucred.h> as follows:
struct xucred {
u_int cr_version; /* structure layout version */
uid_t cr_uid; /* effective user id */
short cr_ngroups; /* number of groups */
gid_t cr_groups[XU_NGROUPS]; /* groups */
};
The cr_version fields should be checked against
XUCRED_VERSION define.
The credentials presented to the server (the
listen(2) caller) are those of the client when it
called connect(2); the credentials presented to the
client (the connect(2) caller) are those of the
server when it called listen(2). This mechanism is
reliable; there is no way for either party to
influence the credentials presented to its peer
except by calling the appropriate system call (e.g.,
connect(2) or listen(2)) under different effective
credentials.
On all the above platforms, those are restricted to SOCK_STREAM type AF_UNIX sockets.
Using the TestClient and TestServer on
Linux Tile440 3.19.0-28-generic #30-Ubuntu SMP Mon Aug 31 15:52:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
on the client, printing out address correctly results in:
connected to [family=PF_UNIX path=/tmp/fubar.sock]
but on the server, if you print out the address from
int n = channel.read(buf);
UnixSocketAddress remote = channel.getRemoteSocketAddress();
System.err.printf("%x Read in %d bytes from %s%n",hashCode(),n,remote);
you get output like:
29ee9faa Read in 39 bytes from [family=PF_UNIX path=6�J�]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.