jNVMf is a Java library for remote access to NVMe storage. It implements client side of the NVMe over Fabrics (NVMf) specification. jNVMf has been tested to work with SPDK, Linux and Mellanox offloading targets. Futhermore it supports advanced NVMe/f features like fused operations, incapsule data and dataset management.
Building the source requires Apache Maven and a Java JDK with Java version 8. jNVMf depends on DiSNI while the java dependecy is automatically picked up from maven central you need to download and compile the native library of DiSNI. Check the DiSNI README for instructions on how to build the native library. To build jNVMf use:
mvn -DskipTests package or mvn -DskipTests install
To run the unit tests make sure the LD_LIBRARY_PATH contains libdisni (see above) and execute:
mvn test -Prdma -DargLine="-DdestinationAddress=10.100.0.5 -DlocalAddress=10.100.0.2 -Dnqn=nqn.2017-06.io.crail:cnode4420 -Dport=4420"
where destinationAddress is the target IP address, localAddress is the local IP of the RDMA device, nqn the NVMe qualified name of the controller you want to connect to and port the port of the target.
jNVMf provides a small benchmark tool. Run:
java -cp target/jnvmf-1.7-jar-with-dependencies.jar:target/jnvmf-1.7-tests.jar com.ibm.jnvmf.utils.NvmfClientBenchmark --help
to shows the available arguments.
For example:
java -cp target/jnvmf-1.7-jar-with-dependencies.jar:target/jnvmf-1.7-tests.jar com.ibm.jnvmf.utils.NvmfClientBenchmark -a 10.100.0.22 -p 4420 -g 4096 -i 3 -m RANDOM -n 10 -nqn nqn.2016-06.io.spdk:cnode1 -o bench.csv -qd 1 -rw read -s 4096 -qs 64 -H -I
- executes a
-m RANDOM-rw readtest - to a target at
-a 10.100.0.22,-p 4420with controller-nqn nqn.2016-06.io.spdk:cnode1 - all accesses are aligned to
-g 4096bytes - statistics are printed every
-i 3seconds-n 10times - output is written to
-o bench.csvin csv format - queue depth
-qd 1 - transfer size
-s 4096bytes - queue size
-qs 64entries - histogram
-H - RDMA inline data
-I
jNVMf offers a tool to run diagnostics. Print help with:
java -cp target/jnvmf-1.7-jar-with-dependencies.jar:target/jnvmf-1.7-tests.jar com.ibm.jnvmf.utils.NvmfDiagnostics --help
Example:
java -Djnvmf.legacy=true -cp jnvmf-1.7-jar-with-dependencies.jar:jnvmf-1.7-tests.jar com.ibm.jnvmf.utils.NvmfDiagnostics -a 10.100.0.22 -p 4420 -nqn nqn.2016-06.io.spdk:cnode1 -t asyncevent
- executes
-t asynceventtest - to a target at
-a 10.100.0.22,-p 4420with controller-nqn nqn.2016-06.io.spdk:cnode1
Sample usage of the jNVMf API to perform a read operation:
Nvme nvme = new Nvme();
/* target ip and port we want to connect to */
InetSocketAddress socketAddress = new InetSocketAddress("10.100.0.1", 4420);
/* controller NVMe qualified name */
NvmfTransportId tid = new NvmfTransportId(socketAddress,
new NvmeQualifiedName("nqn.2016-06.io.spdk:cnode1"));
/* connect to controller */
Controller controller = nvme.connect(tid);
/* enable controller */
controller.getControllerConfiguration().setEnable(true);
controller.syncConfiguration();
controller.waitUntilReady();
/* create a io queue pair with submission queue size 64 */
IoQueuePair queuePair = controller.createIoQueuePair(64);
/* allocate and register buffer to be used for transfer */
ByteBuffer buffer = ByteBuffer.allocateDirect(4096);
KeyedNativeBuffer registeredBuffer = queuePair.registerMemory(buffer);
/* create a new read command */
NvmReadCommand command = new NvmReadCommand(queuePair);
/* create a response */
Response<NvmResponseCapsule> response = command.newResponse();
/* set buffer */
NvmIOCommandCapsule commandCapsule = command.getCommandCapsule();
commandCapsule.setSglDescriptor(buffer);
/* set length, LBA and namespace identifier */
NvmIoCommandSqe sqe = commandCapsule.getSubmissionQueueEntry();
sqe.setStartingLba(0);
/* read one block*/
sqe.setNumberOfLogicalBlocks(1);
sqe.setNamespaceIdentifier(new NamespaceIdentifier(1));
volatile boolean done = false;
/* set callback */
command.setCallback(new OperationCallback() {
@Override
public void onStart() {
}
@Override
public void onComplete() {
System.out.println("command completed");
/* check response */
NvmCompletionQueueEntry cqe = response.getResponseCapsule().getCompletionQueueEntry();
StatusCode.Value statusCode = cqe.getStatusCode();
if (!statusCode.equals(GenericStatusCode.getInstance().SUCCESS)) {
System.out.println("Command not successful");
}
done = true;
}
@Override
public void onFailure(RdmaException e) {
System.out.println("There was an RDMA error when executing the command");
done = true;
}
});
command.execute(response);
/* wait until command is done */
while (!done);
PRs are always welcome. Please fork, and make necessary modifications, and let us know.
If you have questions or suggestions, feel free to post at:
https://groups.google.com/forum/#!forum/zrlio-users
or email: [email protected]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent