milvus-io/milvus-sdk-java

Java SDK for Milvus.

Milvus Java SDK

Java SDK for Milvus. To contribute to this project, please read our contribution guidelines first.

Getting started

Prerequisites

-   Java 8 or higher
-   Apache Maven or Gradle

The following table shows compatibilities between Milvus and Java SDK.

Milvus version	Java SDK version
2.0	2.0.4
2.1	2.1.0-beta4
2.2.0 ~ 2.2.8	2.2.0 ~ 2.2.5
>= 2.2.9	2.2.7 ~ 2.2.15
2.3.x	2.3.6
2.4.x	2.4.0

Install Java SDK

You can use Apache Maven or Gradle add Milvus SDK to your project.

• Apache Maven

   <dependency>
       <groupId>io.milvus</groupId>
       <artifactId>milvus-sdk-java</artifactId>
       <version>2.4.0</version>
   </dependency>

• Gradle/Groovy

  implementation 'io.milvus:milvus-sdk-java:2.4.0'

• Gradle/Kotlin

  implementation("io.milvus:milvus-sdk-java:2.4.0")

Examples

Please refer to examples folder for Java SDK examples.

Documentation

Troubleshooting

• If you encounter the following error when running your application:

  Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
  

  This is because SLF4J jar files need to be added into your application's classpath. SLF4J is required by Java SDK for logging purpose.

  To fix this issue, you can use Apache Maven or Gradle to download the required jar files.

  • Apache Maven

     <dependency>
         <groupId>org.slf4j</groupId>
         <artifactId>slf4j-api</artifactId>
         <version>1.7.30</version>
     </dependency>

  • Gradle/Groovy

    compile group: 'org.slf4j', name: 'slf4j-api', version: '1.7.30'

  • Gradle/Kotlin

    implementation("org.slf4j:slf4j-api:1.7.30")

milvus-sdk-java's Issues

Change proto package to io.milvus.grpc

problem about duplication

When I insert one vector twice, it seems that milvus doesn't cover the data with same vectorId and it keeps two copies of the same data

java sdk exception: how to use manual vectorId when insert vectors

When I insert vectors with vector IDs using Java SDK, there is a exception:
InsertResponse {Response {code = ILLEGAL_VECTOR_ID, message = "Table vector IDs are auto-generated. All vectors of this table must use auto-generated IDs."}, returned 0 vector ids}
however, python edition can use manual vector IDs to insert

insert method can pass in the same vectorId

InsertResponse insert(InsertParam insertParam) method is called whether at the same time or different times, it can pass in java.lang.Long type java.util.List composed with the same vectorId.

where is partitionTags in MilvusGrpcClient.java

Method “search‘’ in MilvusGrpcClient.java，can not support partitionTags

'SearchByIDParam(io.milvus.client.SearchByIDParam.Builder)' has private access in 'io.milvus.client.SearchByIDParam'

Release v0.2.0

• Remove timeout parameter
• Several API changes

Update pom for deploying to Maven central repo

Add IndexType IVF_PQ

fix connected() & add port range check & add @nonnull annotation & set maxInboundMessageSize

Add more documents

Following documents are needed.

• README
• ISSUE Template
• Contributing Guide
• CHANGELOG

[Feature] Rename Java SDK APIs

Update Java SDK APIs

Reference Rename sdk interfaces #2295

Add README

[Feature] Update Java SDK APIs

• Change getVectorByID to getVectorsByIDs
  Input collection name and a list of ids, return a list of corresponding vectors

• Add searchByIds
  Input collection name, ids, top_k, partition_tags and params, return search results

• Update collectionInfo
  Get string in JSON style

{
	"partitions":[
		{
			"row_count":10000,
			"segments":[
				{
					"data_size":5200000,
					"index_name":"IDMAP",
					"name":"1587627144905721000",
					"row_count":10000
				}
			],
			"tag":"_default"
		}
	],
	"row_count":10000
}

• Add hasPartition
  Input collection name and partition tag, output whether the tag exists in collection

Rename table in sdk

INFO: Created collection successfully!
CollectionMapping = {collectionName = uZoWJDFhTL, dimension = 128, indexFileSize = 50, metricType = IP}
Table UbiwhCJzKv created.

Release sdk version 0.5.0, for Milvus 0.7.0

Change GRPC proto (and related code) to increase search result's transmission speed

Add success status check in other Response

Add waitTime option in ConnectParam

Previously default wait time is 500 ms for client to establish a connection to server.
We should let users to decide how long to wait.

java.lang.NoClassDefFoundError: com/google/protobuf/GeneratedMessageV3

我在服务器上搭建了milvus0.8的cpu服务，访问正常。
下载了0.7的jar包依赖，

这个请问是什么问题？希望官方大大帮忙看看

Add more getters in SearchResponse & add normalize method in unittest

fix deleteByRange error message & update unittest

Update README and CHANGELOG

change default timeout to 24 hour

[BUG] Search failed with exception if no search result

test_search_range_no_result (io.milvus.client.MilvusGrpcClient@7b4619a3, IP_GpPOyWPTUN) java.lang.ArithmeticException: / by zero at io.milvus.client.MilvusGrpcClient.buildSearchResponse(MilvusGrpcClient.java:744) at io.milvus.client.MilvusGrpcClient.search(MilvusGrpcClient.java:349) at com.TestSearchVectors.test_search_range_no_result(TestSearchVectors.java:430) at com.MainClass.main(MainClass.java:142) ... Removed 22 stack frames
SDK_version: 0.3.1RC-SNAPSHOT

Force channel to request connection in connect()

Since when the channel is first created, it enters directly into IDLE state and it's not even trying to create a connection because of a lack of new or pending RPCs

Add more getters in ShowPartitionResponse

java-sdk 链接超时

docker容器已经启动成功。
使用当前github项目里的examples进行测试，修改成我自己的IP和端口号，
然后运行测试代码，会有超时异常的报错。
本地用telnet检测端口是开通状态，请问有可能是什么问题呢？

环境：
docker容器里的版本是0.6.0 ,
java-jdk版本0.4.1

Add Apache 2.0 license header to source code

ignore *.iml generated by IntelliJ IDEA and pass args to example main

1. ignore *.iml generated by IntelliJ IDEA
2. pass args to example main

Partition tag not working when inserting

Add javadoc

the Index still exists after it has been deleted

First, I create a Index on a table named "example", then delete the Index by calling dropIndex method.When I call the describeIndex method to make sure the index info was empty of the 'example' table, it will still return the index info that I created.

change IndexParam in DescribeIndexResponse to Index

Update examples

Implement new api for partition

Please check the #77 of milvus project. New api has been defined for table partition.

Format code with Google Java Style

Row count returned by collectionInfo is wrong

{"partitions":[{"row_count":7999,"segments":[{"data_size":4159480,"index_name":"IDMAP","name":"1588752091131169000","row_count":7999}],"tag":"_default"}],"row_count":7999}

Change connect waitTime to timeout

Just realized setting a connect wait time and let client wait for the entire duration for channel to establish connection instead of having a timeout is ridiculous. Changing it now.

当milvus服务出现问题时，我怎么来让我的客户端进行重连呢？

我现在的策略是采用的定时任务定时去检测milvus服务，当我检测到我的服务异常的时候，我就去断开客户端现在的连接，然后再去创建新的连接。但是我发现我在压测数据入库时，如果把把milvus重启，虽然能重新创建新的连接，但是数据不再入库了，请问这是什么原因呢？
@Scheduled(initialDelay = 1000L, fixedDelay = 1000L) public void reconnect() { boolean serverStatus = milvusDataSource.getServerStatus(); if (!serverStatus) { log.info("Disconnect Milvus."); boolean disconnect = milvusDataSource.disconnect(); if (disconnect) { log.info("Reconnect Milvus."); milvusDataSource.connect(host, port); } } }

MilvusClient sometimes will return "You are not connected to Milvus server" after it being connected to server for a long time

I have created a singleton instance of MilvusClient and connected to Milvus server already.
When there are no operations on MilvusClient for a long time, sometimes it will return "You are not connected to Milvus server" as Response message.
After review code, the GRPC create Channel:

  channel =
      ManagedChannelBuilder.forAddress(connectParam.getHost(), port)
          .usePlaintext()
          .maxInboundMessageSize(Integer.MAX_VALUE)
          .build();

Can we solve the problem by adding keepalive settings？like:

  channel =
      ManagedChannelBuilder.forAddress(connectParam.getHost(), port)
          .keepAliveTime(5, TimeUnit.MINUTES)
          .keepAliveWithoutCalls(true)
          .keepAliveTimeout(10, TimeUnit.MINUTES)
          .idleTimeout(24, TimeUnit.HOURS)
          .usePlaintext()
          .maxInboundMessageSize(Integer.MAX_VALUE)
          .build();

Make some functions and constructors package-private if necessary.

Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true

Sometimes after operations are done and in tear down process, server side will return severe log:
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.

How to reproduce: Happens randomly. In client test, always occurs in insertAsync() and preloadCollection() but not others.

May 09, 2020 10:54:02 AM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=73, target=250.250.250.250:19530} was not shutdown properly!!! ~*~*~*
    Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
	at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:94)
	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:52)
	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:43)
	at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:548)
	at io.milvus.client.MilvusGrpcClient.connect(MilvusGrpcClient.java:93)
	at io.milvus.client.MilvusClientTest.lambda$connectUnreachableHost$7(MilvusGrpcClientTest.java:162)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:55)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:37)
	at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:2952)
	at io.milvus.client.MilvusClientTest.connectUnreachableHost(MilvusGrpcClientTest.java:162)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

May 09, 2020 10:54:02 AM io.milvus.client.MilvusGrpcClient logInfo
INFO: Channel terminated

The reason is still unknown.

Change IVF_SQ8H to IVF_SQ8_H

Add examples

Add @Deprecated for DateRanges in SearchParam

add IVF_SQ8_H index type

Fix README format

can not create table

milvus-server: milvusdb/milvus:latest
version:v0.4.0
config:

change sleep time to 5000,

ConnectivityState connectivityState;
connectivityState = channel.getState(true);
logInfo("Waiting to connect...");
TimeUnit.MILLISECONDS.sleep(5000);
connectivityState = channel.getState(false);
if (connectivityState != ConnectivityState.READY) {
logSevere("Connect failed! {0},connectivityState:{1}", connectParam.toString(), connectivityState);
return new Response(Response.Status.CONNECT_FAILED);

just run MilvusGrpcClientTest.createTable method, and the log will be:

信息: Connected successfully!
ConnectParam {host='192.168.1.100', port='19530'}
十月 10, 2019 9:32:01 上午 io.milvus.client.MilvusGrpcClient logSevere
严重: Create table failed
TableSchema = {tableName = ecqruakdag, dimension = 128, indexFileSize = 1024, metricType = L2}
error_code: ILLEGAL_TABLE_NAME
reason: "Invalid table name: "
十月 10, 2019 9:32:01 上午 io.milvus.client.MilvusGrpcClient logSevere
严重: dropTable RPC failed:
Status{code=INTERNAL, description=Did not read entire message, cause=null}
org.opentest4j.AssertionFailedError:
Expected :true
Actual :false

fix client version in readme and src code