codislabs / jodis Goto Github PK
View Code? Open in Web Editor NEWA java client for codis based on Jedis and Curator
License: MIT License
A java client for codis based on Jedis and Curator
License: MIT License
下面是代码:
JedisResourcePool jedisPool = RoundRobinJedisPool.create().
curatorClient("192.168.25.247:2181", 20000).
zkProxyDir("/zk/codis/db_test/proxy").build();
Jedis jedis = jedisPool.getResource();
jedis.set("foo", "bar");
String value = jedis.get("foo");
System.out.println(value);
下面是日志:
[2015-12-23 11:02:39_434] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl [main.start:238] Starting
[2015-12-23 11:02:39_462] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:zookeeper.version=3.4.7-1713338, built on 11/09/2015 04:32 GMT
[2015-12-23 11:02:39_465] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:host.name=JQ0640.chinapnr.com
[2015-12-23 11:02:39_465] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.version=1.7.0_45
[2015-12-23 11:02:39_465] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.vendor=Oracle Corporation
[2015-12-23 11:02:39_465] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.home=C:\Program Files\Java\jdk1.7.0_45\jre
[2015-12-23 11:02:39_465] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.class.path=E:\JavaTestSpace\HuiFuExamPro\bin;E:\JavaTestSpace\HuiFuExamPro\lib\jodis-0.2.2.jar;E:\JavaTestSpace\HuiFuExamPro\lib\jedis-2.7.2.jar;E:\JavaTestSpace\HuiFuExamPro\lib\commons-pool2-2.4.jar;E:\JavaTestSpace\HuiFuExamPro\lib\slf4j-api-1.6.1.jar;E:\JavaTestSpace\HuiFuExamPro\lib\slf4j-log4j12-1.6.1.jar;E:\JavaTestSpace\HuiFuExamPro\lib\log4j-1.2.16.jar;E:\JavaTestSpace\HuiFuExamPro\lib\jackson-annotations-2.7.0-20151116.041133-14.jar;E:\JavaTestSpace\HuiFuExamPro\lib\jackson-core-2.7.0-20151126.183923-32.jar;E:\JavaTestSpace\HuiFuExamPro\lib\jackson-databind-2.7.0-20151125.063421-188.jar;E:\JavaTestSpace\HuiFuExamPro\lib\guava-18.0.jar;E:\JavaTestSpace\HuiFuExamPro\lib\cobertura-2.1.1.jar;E:\JavaTestSpace\HuiFuExamPro\lib\zookeeper-3.4.7.jar;E:\JavaTestSpace\HuiFuExamPro\lib\curator-client-2.5.0.jar;E:\JavaTestSpace\HuiFuExamPro\lib\curator-framework-2.5.0.jar;E:\JavaTestSpace\HuiFuExamPro\lib\curator-recipes-2.5.0.jar
[2015-12-23 11:02:39_465] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.library.path=C:\Program Files\Java\jdk1.7.0_45\bin;C:\windows\Sun\Java\bin;C:\windows\system32;C:\windows;D:\app\killy.zhang\product\11.2.0\client_1;C:\oracle\product\10.2.0\client_1\bin;D:\Program Files\Java\jdk1.6.0_22\bin;D:\Program Files\Java\jre6\bin;D:\Software\apache-maven-3.1.1/bin;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0;D:\Software\windows linux commands;D:\Program Files\TortoiseSVN\bin;D:\Program Files (x86)\WinSCP;D:\Program Files (x86)\Windows7Master;E:\技术研究\ANT\apache-ant-1.8.2\bin;C:\Program Files (x86)\Python27;.
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.io.tmpdir=C:\Users\KILLY~1.ZHA\AppData\Local\Temp
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:java.compiler=
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:os.name=Windows 7
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:os.arch=amd64
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:os.version=6.1
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:user.name=killy.zhang
[2015-12-23 11:02:39_466] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:user.home=C:\Users\killy.zhang
[2015-12-23 11:02:39_467] INFO org.apache.zookeeper.ZooKeeper [main.logEnv:100] Client environment:user.dir=E:\JavaTestSpace\HuiFuExamPro
[2015-12-23 11:02:39_470] INFO org.apache.zookeeper.ZooKeeper [main.:438] Initiating client connection, connectString=192.168.25.247:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@1f449a28
[2015-12-23 11:02:39_526] INFO org.apache.zookeeper.ClientCnxn [main-SendThread(192.168.25.247:2181).logStartConnect:1032] Opening socket connection to server 192.168.25.247/192.168.25.247:2181. Will not attempt to authenticate using SASL (unknown error)
[2015-12-23 11:02:39_530] INFO org.apache.zookeeper.ClientCnxn [main-SendThread(192.168.25.247:2181).primeConnection:876] Socket connection established to 192.168.25.247/192.168.25.247:2181, initiating session
[2015-12-23 11:02:39_564] INFO org.apache.zookeeper.ClientCnxn [main-SendThread(192.168.25.247:2181).onConnected:1299] Session establishment complete on server 192.168.25.247/192.168.25.247:2181, sessionid = 0x151c4d1663c0037, negotiated timeout = 20000
[2015-12-23 11:02:39_568] INFO org.apache.curator.framework.state.ConnectionStateManager [main-EventThread.postState:194] State change: CONNECTED
log4j:WARN No appenders could be found for logger (com.wandoulabs.jodis.RoundRobinJedisPool).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" redis.clients.jedis.exceptions.JedisException: Proxy list empty
at com.wandoulabs.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:286)
at com.huifu.killy.codis.CodisExam.main(CodisExam.java:21)
Unable to get lock channel for E:\Users\psteitz\pool\tags\POOL_2_4_RC1\target\cobertura\cobertura.ser.lock: \Users\psteitz\pool\tags\POOL_2_4_RC1\target\cobertura\cobertura.ser.lock (系统找不到指定的路径。)
lock file could not be deleted
可以支持连接多个zookeeper吗?我看源码好像没有实现
如果配置codis集群,可能需要配置多个zookeeper。
每次取出来的Jedis连接,没有还回去的接口,无法共享长连接池,一直使用一个对象很不靠谱。
你好,Jodis里面的包用了jedis2.7.3版本的。。。这个版本的jedis的JedisPoolConfig没有了setMaxActive和setMaxWait等方法,这两个方法改成其他什么方法了么?不通过jodis去连接,直接通过jedis连接的话,配置要怎么弄呢?
使用样例代码来测试时报错:
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:53)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:99)
at io.codis.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:205)
at JodisToMultiCodisProxy.main(JodisToMultiCodisProxy.java:14)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.ConnectException: Connection refused: connect
at redis.clients.jedis.Connection.connect(Connection.java:164)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:80)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1677)
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:87)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:868)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at redis.clients.util.Pool.getResource(Pool.java:49)
... 8 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at redis.clients.jedis.Connection.connect(Connection.java:158)
... 15 more
虚拟机上已关闭防火墙,在本地使用Zookeeper客户端连接虚拟机上的Zookeeper正常(单个Zookeeper,没有部Zookeeper集群),Zookeeper配置文件如下:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/zhudf/zookeeper-3.4.6/data
dataLogDir=/home/zhudf/zookeeper-3.4.6/log
clientPort=2181
我这边使用的jodis是0.3.0版本 jedis是2.8版本。因为2.8版本直接使用jedis.close()就可以了return pool里 。
但是我这边测试调用codis不支持的命令,正常我应该这样:
try {
logger.info("keys---"+jedisClient.keys("*"));
}catch (Exception e){
logger.error("jedis error");
}finally {
if (jedisClient != null){
jedisClient.close();
}
}
*These commands are disallowed in codis proxy, if you use them, proxy will close the connection to warn you. *
是不是说我这边不需要处理?不需要close? 那么这样的话 我如何写一个通用的base方法来统一处理呢?
如果客户端初始化已经成功了,后来突然的网络中断等问题,会发起重连。关于这些
Client session timed out, have not heard from server in 2003ms for sessionid 0x0, closing socket connection and attempting reconnect (ClientCnxn.java:1096)
Opening socket connection to server 10.201.3.38/10.201.3.38:2180. Will not attempt to authenticate using SASL (unknown error) (ClientCnxn.java:975)
怎么做次数和时间定制。或者取消重连。。
您好,请问java 客户端在连接的时候 如何连接 zklist
比如 kafka 在连接zk的时候 有个 zookeeper.connect 的写法是 ip:port,ip2,port,ip3:port
jodis 支持zklist吗
如果不支持,那客户端是如何连接多个proxy的
jodis只支持zookeeper吗?不支持etcd?
jar版本如下
jodis 0.2.2版本
jedis 2.7.3
apache-common-pools2-2.3
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
com/wandoulabs/jodis/RoundRobinJedisPool.resetPools()V @244: invokespecial
Reason:
Type 'redis/clients/jedis/JedisPoolConfig' (current frame, stack[5]) is not assignable to 'org/apache/commons/pool2/impl/GenericObjectPoolConfig'
Current Frame:
bci: @244
flags: { }
locals: { 'com/wandoulabs/jodis/RoundRobinJedisPool', 'com/google/common/collect/ImmutableList', 'java/util/Map', 'com/google/common/collect/ImmutableList$Builder', 'java/util/Iterator', 'org/apache/curator/framework/recipes/cache/ChildData', 'com/wandoulabs/jodis/CodisProxyInfo', 'java/lang/String', 'com/wandoulabs/jodis/RoundRobinJedisPool$PooledObject', '[Ljava/lang/String;', 'java/lang/String', integer }
stack: { uninitialized 206, uninitialized 206, 'java/lang/String', uninitialized 212, uninitialized 212, 'redis/clients/jedis/JedisPoolConfig', 'java/lang/String', integer, integer, integer, 'java/lang/String', integer, 'java/lang/String' }
try (Jedis jedis = jedisPool.getResource()) {}
这种写法,一个连接用完之后即关闭,没有像JedisPool那样用完return回pool里吗?这样的话岂不是会频繁创建新连接?
测试跑100W连续多线程塞数据都正常,但是停下放一段时间后,其他小批量的后台任务反而时不时报错:
2016-12-15 20:11:33.439 [Thread-990017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 990000-测试codis使用时间:8
2016-12-15 20:11:33.784 [Thread-991017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 991000-测试codis使用时间:8
2016-12-15 20:11:34.076 [Thread-992017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 992000-测试codis使用时间:3
2016-12-15 20:11:34.401 [Thread-993017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 993000-测试codis使用时间:7
2016-12-15 20:11:34.724 [Thread-994017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 994000-测试codis使用时间:9
2016-12-15 20:11:34.972 [Thread-995017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 995000-测试codis使用时间:13
2016-12-15 20:11:35.363 [Thread-996017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 996000-测试codis使用时间:5
2016-12-15 20:11:35.657 [Thread-997017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 997000-测试codis使用时间:8
2016-12-15 20:11:35.943 [Thread-998017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 998000-测试codis使用时间:2
2016-12-15 20:11:36.210 [Thread-999017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 999000-测试codis使用时间:1
2016-12-15 20:52:52.489 [localhost-startStop-1-SendThread(10.201.5.16:2180)] INFO org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 10004ms for sessionid 0x158fd66fd860020, closing socket connection and attempting reconnect
2016-12-15 20:52:52.489 [localhost-startStop-1-SendThread(10.201.5.16:2180)] INFO org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 13510ms for sessionid 0x158fd66fd86001f, closing socket connection and attempting reconnect
2016-12-15 20:52:53.988 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: SUSPENDED
2016-12-15 20:52:53.990 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: SUSPENDED
2016-12-15 20:52:54.407 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.17/10.201.5.17:2180. Will not attempt to authenticate using SASL (unknown error)
2016-12-15 20:52:54.515 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.17/10.201.5.17:2180, initiating session
2016-12-15 20:52:54.923 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.17/10.201.5.17:2180. Will not attempt to authenticate using SASL (unknown error)
2016-12-15 20:52:54.924 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.17/10.201.5.17:2180, initiating session
2016-12-15 20:52:55.625 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x158fd66fd860020 has expired, closing socket connection
2016-12-15 20:52:55.625 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x158fd66fd86001f has expired, closing socket connection
2016-12-15 20:52:55.742 [localhost-startStop-1-EventThread] WARN org.apache.curator.ConnectionState - Session expired event received
2016-12-15 20:52:55.823 [Curator-PathChildrenCache-0] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_SUSPENDED, no data
2016-12-15 20:52:55.823 [Curator-PathChildrenCache-1] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_SUSPENDED, no data
2016-12-15 20:52:55.824 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.201.5.16:2180,10.201.5.15:2180,10.201.5.17:2180 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@7ebe36cc
2016-12-15 20:52:55.825 [localhost-startStop-1-EventThread] WARN org.apache.curator.ConnectionState - Session expired event received
2016-12-15 20:52:55.825 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.201.5.16:2180,10.201.5.15:2180,10.201.5.17:2180 sessionTimeout=15000 watcher=org.apache.curator.ConnectionState@500764ed
2016-12-15 20:52:55.826 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: LOST
2016-12-15 20:52:55.826 [Curator-PathChildrenCache-0] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_LOST, no data
2016-12-15 20:52:55.826 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down
2016-12-15 20:52:55.827 [localhost-startStop-1-SendThread(10.201.5.15:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.15/10.201.5.15:2180. Will not attempt to authenticate using SASL (unknown error)
2016-12-15 20:52:55.827 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.17/10.201.5.17:2180. Will not attempt to authenticate using SASL (unknown error)
2016-12-15 20:52:55.828 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: LOST
2016-12-15 20:52:55.828 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down
2016-12-15 20:52:55.829 [localhost-startStop-1-SendThread(10.201.5.15:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.15/10.201.5.15:2180, initiating session
2016-12-15 20:52:55.829 [Curator-PathChildrenCache-1] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_LOST, no data
2016-12-15 20:52:55.829 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.17/10.201.5.17:2180, initiating session
2016-12-15 20:52:55.832 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server 10.201.5.17/10.201.5.17:2180, sessionid = 0x258faae399a0029, negotiated timeout = 15000
2016-12-15 20:52:55.832 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: RECONNECTED
2016-12-15 20:52:55.833 [Curator-PathChildrenCache-1] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_RECONNECTED, no data
2016-12-15 20:52:55.834 [localhost-startStop-1-SendThread(10.201.5.15:2180)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server 10.201.5.15/10.201.5.15:2180, sessionid = 0x58faae39ef0026, negotiated timeout = 20000
2016-12-15 20:52:55.834 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: RECONNECTED
2016-12-15 20:52:55.835 [Curator-PathChildrenCache-0] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_RECONNECTED, no data
2016-12-15 20:53:38.374 [Timer-6] ERROR c.k.finance.redis.service.RedisCodisServiceImpl - leftPopList 出错[key=MQ:600000-system_rank_msg-500000-test]
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:201) ~[jedis-2.8.0.jar:na]
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Protocol.process(Protocol.java:141) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Protocol.read(Protocol.java:205) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:216) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.BinaryJedis.lpop(BinaryJedis.java:1153) ~[jedis-2.8.0.jar:na]
at com.kingdee.finance.redis.service.RedisCodisServiceImpl.leftPopList(RedisCodisServiceImpl.java:986) ~[finance-lib-redis-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at com.kingdee.finance.jms.redismq.queue.manage.MessageTransactionManage.consumeMessage(MessageTransactionManage.java:63) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.RedisMQQueueSubscriber.subscriber(RedisMQQueueSubscriber.java:56) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.QueueSubscriberTimerTask.run(QueueSubscriberTimerTask.java:22) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at java.util.TimerThread.mainLoop(Timer.java:555) [na:1.7.0_79]
at java.util.TimerThread.run(Timer.java:505) [na:1.7.0_79]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_79]
at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_79]
at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_79]
at java.net.SocketInputStream.read(SocketInputStream.java:108) ~[na:1.7.0_79]
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195) ~[jedis-2.8.0.jar:na]
... 12 common frames omitted
2016-12-15 20:56:11.162 [Timer-6] ERROR c.k.finance.redis.service.RedisCodisServiceImpl - leftPopList 出错[key=MQ:600000-system_rank_msg-500000-test]
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:201) ~[jedis-2.8.0.jar:na]
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Protocol.process(Protocol.java:141) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Protocol.read(Protocol.java:205) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:216) ~[jedis-2.8.0.jar:na]
at redis.clients.jedis.BinaryJedis.lpop(BinaryJedis.java:1153) ~[jedis-2.8.0.jar:na]
at com.kingdee.finance.redis.service.RedisCodisServiceImpl.leftPopList(RedisCodisServiceImpl.java:986) ~[finance-lib-redis-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at com.kingdee.finance.jms.redismq.queue.manage.MessageTransactionManage.consumeMessage(MessageTransactionManage.java:63) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.RedisMQQueueSubscriber.subscriber(RedisMQQueueSubscriber.java:56) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.QueueSubscriberTimerTask.run(QueueSubscriberTimerTask.java:22) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT]
at java.util.TimerThread.mainLoop(Timer.java:555) [na:1.7.0_79]
at java.util.TimerThread.run(Timer.java:505) [na:1.7.0_79]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_79]
at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_79]
at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_79]
at java.net.SocketInputStream.read(SocketInputStream.java:108) ~[na:1.7.0_79]
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195) ~[jedis-2.8.0.jar:na]
... 12 common frames omitted
代码如下:
/**
* Build jedis resource pool.
* @param settingProperties
* @param curatorFramework
* @return
*/
private JedisResourcePool buildJedisResourcePool(RedisCodisSettingProperties settingProperties,CuratorFramework curatorFramework){
JedisPoolConfig jedisPoolConfig = getJedisPoolConfig();
logger.info("buildJedisResourcePool初始化读取 codisSetting 相关配置");
JedisResourcePool jedisPool = RoundRobinJedisPool.create()
.curatorClient(curatorFramework, true)
.zkProxyDir(settingProperties.getZkProxyDir())
.poolConfig(jedisPoolConfig)
.password(settingProperties.getAuthPwd()).timeoutMs(DEFAULT_POOL_TIMEOUT_MS).build();
return jedisPool;
}
/**
* Build zookeeper framework-style client.
* @param settingProperties
* @return
*/
private CuratorFramework buildCuratorFramework(RedisCodisSettingProperties settingProperties) {
logger.info("buildCuratorFramework初始化读取 codisSetting 相关配置,ZkAddress={},SessionTimeoutMs={},SleepMsBetweenRetries={}",
settingProperties.getZkAddress(), settingProperties.getSessionTimeoutMs(), settingProperties.getSleepMsBetweenRetries());
CuratorFramework curatorFramework = CuratorFrameworkFactory.builder()
.connectString(settingProperties.getZkAddress())
.sessionTimeoutMs(settingProperties.getSessionTimeoutMs())
.retryPolicy(new RetryNTimes(0, settingProperties.getSleepMsBetweenRetries())).build();
return curatorFramework;
}
/**
* 初始化Redis连接池,如果codisSettingBean为空,则使用默认值.
*
* @return
*/
private synchronized JedisPoolConfig buildJedisPoolConfig() {
if (this.jedisPoolConfig != null) {
return this.jedisPoolConfig;
}
RedisCodisSettingProperties codisSetting = this.codisSettingBean.getSettingProperties();
JedisPoolConfig config = new JedisPoolConfig();
logger.info("buildJedisPoolConfig读取 codisSetting 相关配置,MaxTotal={},MaxIdle={},MinIdle={}",
codisSetting.getPoolMaxTotal(), codisSetting.getPoolMaxIdle(), codisSetting.getPoolMinIdle());
config.setMaxTotal(codisSetting.getPoolMaxTotal());
config.setMaxIdle(codisSetting.getPoolMaxIdle());
config.setMinIdle(codisSetting.getPoolMinIdle());
this.jedisPoolConfig = config;
return this.jedisPoolConfig;
}
参数:
session_tms:20000
sleepms_retry:5000
pool_max_total:2000
pool_max_idle:100
pool_min_idle:50
pool_timeout_ms:6000
您好,在对redis进行操作前
是 每次都使用 jedisPool.getResource() 获取连接池的一个jedis对象 使用完成后 再jedis.close()
还是 在项目启动时 获取一个jedis 对象 然后全局使用
我觉得第一种方式不可取,频繁的getResource和close会对系统有很高的开销
如果是第二种方式,那么再新加proxy的时候,客户端是不知道的,不重启客户端就不能识别到新的proxy
我的问题是 应该如何使用 jedisPool.getResource()
看官网上写的在以下情况,codis-ha会退出,
从 codis-dashboard 获取集群状态失败时;
向 codis-dashboard 发送主从切换指令失败时;
问题(1)如果一个proxy和一个master死掉,那么无法主从切换成功,codis-ha会退出?
问题(2)能否针对上面的2种退出场景给一些案例
谢谢
客户端开启多个线程获取多个pool时出现异常,JedisResourcePool jedisPool = RoundRobinJedisPool.create().curatorClient("localhost:2181", 30000)
.zkProxyDir("/zk/codis/db_kok/proxy").build();
异常信息:org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:198) [curator-client-2.5.0.jar:na]
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88) [curator-client-2.5.0.jar:na]
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115) [curator-client-2.5.0.jar:na]
at org.apache.curator.utils.EnsurePath$InitialHelper$1.call(EnsurePath.java:148) [curator-client-2.5.0.jar:na]
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) [curator-client-2.5.0.jar:na]
at org.apache.curator.utils.EnsurePath$InitialHelper.ensure(EnsurePath.java:140) [curator-client-2.5.0.jar:na]
at org.apache.curator.utils.EnsurePath.ensure(EnsurePath.java:99) [curator-client-2.5.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.rebuild(PathChildrenCache.java:320) [curator-recipes-2.5.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:297) [curator-recipes-2.5.0.jar:na]
at com.wandoulabs.jodis.RoundRobinJedisPool.(RoundRobinJedisPool.java:239) [jodis-0.2.2.jar:na]
at com.wandoulabs.jodis.RoundRobinJedisPool.(RoundRobinJedisPool.java:64) [jodis-0.2.2.jar:na]
at com.wandoulabs.jodis.RoundRobinJedisPool$Builder.build(RoundRobinJedisPool.java:479) [jodis-0.2.2.jar:na]
求解决办法
参见Tutorial 中文 操作,版本为 release3.1, 所有codis服务一台,3个zookeeper进程组成伪集群在另一台, 出现了与#10一样的问题
zookeeper 路径 /codis3/codis-demo/proxy/yyyy
信息:
{
"id": 1,
"token": "abbddfe14ddce61f446c85503b6bb46b",
"start_time": "2017-01-19 10:19:32.607652349 +0800 HKT",
"admin_addr": "172.16.100.50:11080",
"proto_type": "tcp4",
"proxy_addr": "172.16.100.50:19000",
"jodis_path": "/jodis/codis-demo/proxy-abbddfe14ddce61f446c85503b6bb46b",
"product_name": "codis-demo",
"pid": 11950,
"pwd": "/software/src/github.com/CodisLabs/codis",
"sys": "Linux localhost.localdomain 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux",
"hostname": "localhost.localdomain",
"datacenter": ""
}
通过proxy端口进行压测正常,登录codis-server 查看也正常,就是使用JAVA 调用时报错
你好,请问Codis3.2支持多库,这个databases参数设置多大比较合理呢?库开多了会不会有性能问题呢?
我启用了2个proxy(分别是proxy1、proxy2),用jodis并发测试。在测试过程中,我手动kill掉proxy2,这个时候jodis客户端报:
redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
感觉jodis实现负载均衡没有起到作用,还是我使用不正确?
我封装了jodis的dubbo服务,然后我们做压力测试的时候(500并发),之前还挺高的,能达到2wtps,但是后来我打算在出现异常的情况下打log日志,因此按理说不报错不会有日志记录,但是这样tps下降了1w,程序也没有报错,(我把dubbo的threads设置为了800,jodis的maxTotal设置为了1000,zkSessionTimeout设置为了30000),最后dump线程发现很多waiting状态,不知这是什么原因导致的?
/**
* This methods assumes there are required bytes to be read. If we cannot read anymore bytes an
* exception is thrown to quickly ascertain that the stream was smaller than expected.
*/
private void ensureFill() throws JedisConnectionException {
if (count >= limit) {
try {
limit = in.read(buf);
count = 0;
if (limit == -1) {
throw new JedisConnectionException("Unexpected end of stream.");
}
} catch (IOException e) {
throw new JedisConnectionException(e);
}
}
}
}
JedisResourcePool jedisPool = RoundRobinJedisPool.create() .curatorClient("zkserver:2181", 30000).zkProxyDir("/zk/codis/db_xxx/proxy").build();
使用上述方式连接zkserver(我是应用启动的时候才获取一次zk),假如zkserver挂掉的话,之后proxy再挂掉一个,实际使用中之前获取的jedisPool仍旧有挂掉的proxy
后来使用zkserver1:2181,zkserver2:2181,zkserver3:2181 这种模式再试,当其中的一个zk挂掉,其它的是可以正常使用的,zk的可用性方面是否使用zkserver1:2181,zkserver2:2181,zkserver3:2181这种方式?
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:201)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:141)
at redis.clients.jedis.Protocol.read(Protocol.java:205)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:196)
at redis.clients.jedis.Jedis.set(Jedis.java:69)
at com.nonobank.cache.cachetools.CodisCacheClient.set(CodisCacheClient.java:27)
at com.nonobank.cache.cachetools.CodisCacheClient.main(CodisCacheClient.java:48)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195)
... 8 more
JedisResourcePool jedisPool = RoundRobinJedisPool.create()
.curatorClient("ip1:2181,ip2,ip3:2181",
30000).zkProxyDir("/zk/codis/db_test/proxy").poolConfig(jedisPoolConfig).build();
项目启动后 jedisPool 就初始化完成了。在项目运行过程中,如果为codis添加了proxy,这个jedisPool 会自己更新吗。
public static void main(String[] args) {
JedisResourcePool jedisPool = RoundRobinJedisPool.create()
.curatorClient("10.10.1.200:2181", 3000).zkProxyDir("/jodis/codis-demo").build();
try(Jedis jedis = jedisPool.getResource()) {
jedis.set("foo", "bar");
String value = jedis.get("foo");
System.out.println(value);
}
}
按照的文档上的代码执行
每次执行都报如下错误
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:198)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:141)
at redis.clients.jedis.Protocol.read(Protocol.java:205)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:196)
at redis.clients.jedis.Jedis.set(Jedis.java:69)
at jodis.JodisTest.main(JodisTest.java:16)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
查看proxy的日志发现
slots.go:91: [WARN] slot-0289 is not ready: hkey = foo
目前配置的是4个group,每个group一个主备
找了很多,网上说buffer中积压的数据达到阀值,可是client-output-buffer-limit中相关设置为0,这是什么原因 @spinlock
JedisResourcePool jedisPool = RoundRobinJedisPool.create()
.curatorClient("127.0.0.1:19000", 30000).zkProxyDir("/Users/lili/Documents/jodis").build();
try (Jedis jedis = jedisPool.getResource()) {
jedis.set("foo", "bar");
String value = jedis.get("foo");
System.out.println(value);
}
提示如下错误
[main-SendThread(127.0.0.1:19000)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:19000. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:19000)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:19000, initiating session
[main] ERROR org.apache.curator.ConnectionState - Connection timed out for connection string (127.0.0.1:19000) and timeout (15000) / elapsed (15086)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:195)
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87)
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
at io.codis.jodis.TestMain.main(TestMain.java:11)
[main-SendThread(127.0.0.1:19000)] WARN org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 30002ms for sessionid 0x0
但是我用命令行操作
./redis-cli -h 127.0.0.1 -p 19000
127.0.0.1:19000> get ju
"xian"
127.0.0.1:19000> get ju
"xian"
127.0.0.1:19000> get ju
"xian"
是没有问题
我的zookeeper是用的是3.4.10版本
libs目录里commons-pool2-2.3 curator-client-2.9.1 curator-framework-2.9.1 curator-recipes-2.9.1 guava-16.0.1 jackson-annotations-2.8.8 jackson-core-2.8.8 jackson-databind-2.8.8 jedis-2.8.0 jsr305-1.3.9 slf4j-api-1.7.25 slf4j-simple-1.7.25 zookeeper-3.4.10 的jar包
请牛人看看
参照作者给的例子,springboot 项目 接入 jodis,项目启用了spring session,启动的时候报错:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'enableRedisKeyspaceNotificationsInitializer' defined in class path resource [org/springframework/boot/autoconfigure/session/RedisSessionConfiguration$SpringBootRedisHttpSessionConfiguration.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: Unable to configure Redis to keyspace notifications. See http://docs.spring.io/spring-session/docs/current/reference/html5/#api-redisoperationssessionrepository-sessiondestroyedevent
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1589)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:554)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:759)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151)
at com.jinbaozheng.pufa.film.PufaApp.main(PufaApp.java:29)
Caused by: java.lang.IllegalStateException: Unable to configure Redis to keyspace notifications. See http://docs.spring.io/spring-session/docs/current/reference/html5/#api-redisoperationssessionrepository-sessiondestroyedevent
at org.springframework.session.data.redis.config.ConfigureNotifyKeyspaceEventsAction.getNotifyOptions(ConfigureNotifyKeyspaceEventsAction.java:81)
at org.springframework.session.data.redis.config.ConfigureNotifyKeyspaceEventsAction.configure(ConfigureNotifyKeyspaceEventsAction.java:55)
at org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration$EnableRedisKeyspaceNotificationsInitializer.afterPropertiesSet(RedisHttpSessionConfiguration.java:251)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1648)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1585)
... 16 common frames omitted
Caused by: org.springframework.dao.InvalidDataAccessApiUsageException: ERR handle request, command 'CONFIG' is not allowed; nested exception is redis.clients.jedis.exceptions.JedisDataException: ERR handle request, command 'CONFIG' is not allowed
at org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert(JedisExceptionConverter.java:64)
at org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert(JedisExceptionConverter.java:41)
at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:37)
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:37)
at org.springframework.data.redis.connection.jedis.JedisConnection.convertJedisAccessException(JedisConnection.java:242)
at org.springframework.data.redis.connection.jedis.JedisConnection.getConfig(JedisConnection.java:603)
at org.springframework.session.data.redis.config.ConfigureNotifyKeyspaceEventsAction.getNotifyOptions(ConfigureNotifyKeyspaceEventsAction.java:74)
... 20 common frames omitted
Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR handle request, command 'CONFIG' is not allowed
at redis.clients.jedis.Protocol.processError(Protocol.java:127)
at redis.clients.jedis.Protocol.process(Protocol.java:161)
at redis.clients.jedis.Protocol.read(Protocol.java:215)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
at redis.clients.jedis.Connection.getBinaryMultiBulkReply(Connection.java:276)
at redis.clients.jedis.Connection.getMultiBulkReply(Connection.java:269)
at redis.clients.jedis.Jedis.configGet(Jedis.java:2630)
at org.springframework.data.redis.connection.jedis.JedisConnection.getConfig(JedisConnection.java:601)
... 21 common frames omitted
大概意思是 不支持 config命令, 请问作者 有什么好的解决方案没有
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool at redis.clients.util.Pool.getResource(Pool.java:53) at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226) at io.codis.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:214) at com.appinterface.codis.CodisUtil.main(CodisUtil.java:13) Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.ConnectException: Connection refused: connect at redis.clients.jedis.Connection.connect(Connection.java:207) at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93) at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767) at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:868) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at redis.clients.util.Pool.getResource(Pool.java:49) ... 3 more Caused by: java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at redis.clients.jedis.Connection.connect(Connection.java:184) ... 10 more
operating system centos6.8
jodis version 0.4.1
firewall is close status;
Bothering me for a few days, asking for help
Jodis可以配连接池,我理解是Jodis到proxy的连接保持,那proxy到server之间有连接池吗(没看过proxy源码)?如果有,是不是jodis到proxy保持10个连接,那么proxy到各个server都有10个连接?如果空闲一个连接release掉,是不是proxy到各个server节点的连接都同步释放掉一个?
问题:
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:50)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:86)
at com.wandoulabs.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:276)
at com.datatang.api.database.redisjava.JodisTemplate.hgetall(JodisTemplate.java:101)
at com.datatang.api.accesscontrol.server.acount.updatedata.UpdateAccountData2Mongodb$StartUpdate.run(UpdateAccountData2Mongodb.java:61)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:449)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at redis.clients.util.Pool.getResource(Pool.java:48)
... 11 more
注释:方法里已经关闭redis连接了。
Jedis redis = pool.getResource();
Map map = redis.hgetAll(key);
redis.close();
等把pool里设置的连接数消耗完,就报上边的错误了。但是为什么redis没有返回pool里呢。
连接池使用了jodis。
codis版本2.0.7
下面是 jodis 客服端代码:
JedisResourcePool jedisPool = RoundRobinJedisPool.create()
.curatorClient("172.16.100.40:2181", 30000).zkProxyDir("/zk/codis/TEST/proxy").build();
try (Jedis jedis = jedisPool.getResource()) {
jedis.set("foo", "bar");
String value = jedis.get("foo");
System.out.println(value);
}
下面是异常:
redis.clients.jedis.exceptions.JedisException: Proxy list empty
at io.codis.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:199)
我用spring-DM 集成jodis之后,本地代码测试正常,服务器报错。开始以为环境问题,我用测试例子打了个执行jar包,在服务器执行没问题。环境上应该没有问题的。。可以帮看一下我的错误,给我一个查找错误的方向吗? 报错如下
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /zk/codis/db_test_codis_cluster/proxy/foo
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) [null:na]
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) [null:na]
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) [null:na]
at org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:226) [null:na]
at org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:215) [null:na]
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) [null:na]
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:212) [null:na]
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:205) [null:na]
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:168) [null:na]
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:39) [null:na]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.createContainers(CuratorFrameworkImpl.java:193) [null:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.ensurePath(PathChildrenCache.java:616) [null:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.rebuild(PathChildrenCache.java:322) [null:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:299) [null:na]
at com.wandoulabs.jodis.RoundRobinJedisPool.(RoundRobinJedisPool.java:242) [null:na]
... 56 common frames omitted
客户端连接Codis时报错。
JedisResourcePool jedispool= RoundRobinJedisPool.create().curatorClient("124.202.133.28:2181", 30000).zkProxyDir("/zk/codis/db_test/proxy").build();
异常信息
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:195)
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87)
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:487)
at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:192)
at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:186)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:183)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:168)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:39)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.createContainers(CuratorFrameworkImpl.java:193)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.ensurePath(PathChildrenCache.java:616)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.rebuild(PathChildrenCache.java:322)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:299)
at com.wandoulabs.jodis.RoundRobinJedisPool.(RoundRobinJedisPool.java:239)
at com.wandoulabs.jodis.RoundRobinJedisPool.(RoundRobinJedisPool.java:64)
at com.wandoulabs.jodis.RoundRobinJedisPool$Builder.build(RoundRobinJedisPool.java:479)
at redistool.main.RedisTool.main(RedisTool.java:201)
您给了个例子:
JedisResourcePool jedisPool = RoundRobinJedisPool.create().curatorClient("192.168.80.81:19000", 6000).zkProxyDir("").build();
Jedis jedis = jedisPool.getResource();
spring中用的时候,怎么配置?
可以结合到spring-data-redis的RedisTemplate中吗?
zookeeper里的信息是:、
[zk: localhost:2181(CONNECTED) 51] ls /jodis/codis-demo
[proxy-2ab1df0bb5a20535a65122033bad07d4]
[zk: localhost:2181(CONNECTED) 52] get /jodis/codis-demo/proxy-2ab1df0bb5a20535a65122033bad07d4
{
"addr": "192.168.118.128:19000",
"admin": "192.168.118.128:11080",
"start": "2016-12-29 11:14:08.907327194 +0800 CST",
"state": "online",
"token": "2ab1df0bb5a20535a65122033bad07d4"
}
为什么客户端启动时候会提示
parse /jodis/codis-demo/proxy-2ab1df0bb5a20535a65122033bad07d4 failed
你好,jodis你们有提供JDK1.6的版本么?如果要弄1.6版本的话,重新编译jodis,JDK要用1.6的么?那rt.jar的包,缺少的文件要怎么弄呢?
for (;;) {
int current = nextIdx.get();
int next = current >= pools.size() - 1 ? 0 : current + 1;
if (nextIdx.compareAndSet(current, next)) {
return pools.get(next).pool.getResource();
}
}
这段源码存在原子性问题
我使用jodis后无法自动回收资源。使用getResource并进行业务处理,多次处理之后出现异常,无法获取jedis资源,以下是异常信息
redis.clients.jedis.exceptions.JedisException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:51)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:99)
at io.codis.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:205)
at com.yym.disney.codis.CodisPool.getResource(CodisPool.java:34)
at com.yym.disney.codis.CodisBase.getJedis(CodisBase.java:31)
at com.yym.disney.codis.CodisOther.expireKey(CodisOther.java:77)
at com.yym.disney.common.servlet.RedisSessionRequest.getSession(RedisSessionRequest.java:40)
at org.springframework.web.servlet.support.SessionFlashMapManager.retrieveFlashMaps(SessionFlashMapManager.java:45)
at org.springframework.web.servlet.support.AbstractFlashMapManager.retrieveAndUpdate(AbstractFlashMapManager.java:87)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:848)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at com.yym.disney.common.servlet.RedisSessionFilter.doFilter(RedisSessionFilter.java:44)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:449)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at redis.clients.util.Pool.getResource(Pool.java:49)
... 42 more
Exception in thread "main" redis.clients.jedis.exceptions.JedisException: Proxy list empty
at io.codis.jodis.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:207)
at com.enjoyf.platform.util.redis.JodisManager.set(JodisManager.java:52)
at com.enjoyf.platform.util.redis.JodisManager.main(JodisManager.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
proxy.toml相关设置如下:
jodis_name = ""
jodis_addr = "web001.dev:2181,web002.dev:2181"
jodis_timeout = "20s"
jodis_compatible = false
jodis链接的代码如下:
pool = RoundRobinJedisPool.create().curatorClient("web001.dev:2181,web002.dev:2181", 30000).zkProxyDir("/jodis/codis-demo").build();
zookeeper中proxy有数据
[zk: localhost:2181(CONNECTED) 0] ls /codis3/codis-demo/proxy
[proxy-9cec7123adc59b0306635d4441a99d53]
zookeeper中/jodis/codis-demo为空
JedisResourcePool jedisPool = new RoundRobinJedisPool("zkserver:2181", 30000, "/zk/codis/db_xxx/proxy", new JedisPoolConfig());
try (Jedis jedis = jedisPool.getResource()) {
jedis.set("foo", "bar");
String value = jedis.get("foo");
}
在你们给出的上述示例代码中,RoundRobinJedisPool构造函数没有传timeout参数,所以使用了默认的JEDIS_POOL_TIMEOUT_UNSET,也就是-1。这样JedisFactory产生的Jedis的timeout也是-1,在运行后会产生Socket错误,提示timeout不能为-1。
使用项目springcloud项目,用到了jodis,发现都引用了guava,jodis引用的19.0版的,spring是21.0版的,导致项目报错,希望可以优化下
升级了版本之后出现以下异常:
java.lang.NoSuchMethodError: redis.clients.jedis.JedisPool.<init>(Lorg/apache/commons/pool2/impl/GenericObjectPoolConfig;Ljava/lang/String;IIILjava/lang/String;ILjava/lang/String;)V at com.wandoulabs.jodis.RoundRobinJedisPool.resetPools(RoundRobinJedisPool.java:267) at com.wandoulabs.jodis.RoundRobinJedisPool.access$200(RoundRobinJedisPool.java:64) at com.wandoulabs.jodis.RoundRobinJedisPool$1.childEvent(RoundRobinJedisPool.java:230) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:522) at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:516)
如果codis 的 proxy突然down了一台,重启之后,jodis不会重新通过zk发现这个proxy?
1、jodis支持redis设置密码吗
2、codis可以使用密码访问redis, jodis连的codis,怎么就是验证密码失败呢
for (ChildData childData : watcher.getCurrentData()) {
try {
CodisProxyInfo proxyInfo = MAPPER.readValue(childData.getData(),
CodisProxyInfo.class);
if (!CODIS_PROXY_STATE_ONLINE.equals(proxyInfo.getState())) {
continue;
}
String addr = proxyInfo.getAddr();
PooledObject pool = addr2Pool.remove(addr);
if (pool == null) {
LOG.info("Add new proxy: " + addr);
String[] hostAndPort = addr.split(":");
String host = hostAndPort[0];
int port = Integer.parseInt(hostAndPort[1]);
pool = new PooledObject(addr,
new JedisPool(poolConfig, host, port, connectionTimeoutMs, soTimeoutMs,
password, database, clientName, false, null, null, null));
}
builder.add(pool);
} catch (Throwable t) {
LOG.warn("parse " + childData.getPath() + " failed", t);
}
}
这里catch Throwable 实在是太大了。
Error继承了Throwable。如果出现jedis的包依赖冲突,引用了低版本的jedis,JedisPool构造函数不同导致NoSuchMethodError。结果被catch住了,变成了一个warn日志,应用还是正常启动了。T.T
建议只catch Exception吧。Error直接抛出来好了。
设置retry重试策略后,“new RetryNTimes(0, 1000)“。。仍然重试连接zookeep了三次,导致应用不能在较短时间内启动。【除了延长启动时间,希望有其他方式处理】。thx
用了jodis,配置JedisPoolConfig后总是超时。原来用的jedis,JedisPoolConfig的配置没有变,用jedis不超时。
发现使用jdk7会报错,用了jdk8就可以了,看了下好像是用到的guava的版本是23,guava依赖jdk1.8?
在线上生产环境中出现以下Exception:
ERROR o.a.c.framework.recipes.cache.PathChildrenCache -
java.lang.NullPointerException: null
at io.codis.jodis.RoundRobinJedisPool$1.logEvent(RoundRobinJedisPool.java:135) ~[io.codis.jodis.jodis-0.3.0.jar:na]
at io.codis.jodis.RoundRobinJedisPool$1.childEvent(RoundRobinJedisPool.java:144) ~[io.codis.jodis.jodis-0.3.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:518) [org.apache.curator.curator-recipes-2.9.1.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:512) [org.apache.curator.curator-recipes-2.9.1.jar:na]
at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) [org.apache.curator.curator-framework-2.9.1.jar:na]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) [com.google.guava.guava-18.0.jar:na]
at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) [org.apache.curator.curator-framework-2.9.1.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:509) [org.apache.curator.curator-recipes-2.9.1.jar:na]
at org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35) [org.apache.curator.curator-recipes-2.9.1.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache$9.run(PathChildrenCache.java:766) [org.apache.curator.curator-recipes-2.9.1.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
是因为获取不到zookeeper信息造成的?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.