javahongxi/whatsmars

Java生态研究(Spring Boot + Redis + Dubbo + RocketMQ + Elasticsearch)🔥🔥🔥🔥🔥

douyin: wuhan.hongxi

whatsmars's Issues

xx

“NoSuchMethodErrors” due to multiple versions of org.apache.zookeeper:zookeeper

Issue description:

There are multiple versions of org.apache.zookeeper:zookeeper in whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer. As shown in the following dependency tree, according to Maven's "nearest wins" strategy, only org.apache.zookeeper:zookeeper:3.4.8 can be loaded, org.apache.zookeeper:zookeeper:3.6.0 will be shadowed.

However, several methods defined in shadowed version org.apache.zookeeper:zookeeper:3.6.0 are referenced by client project via org.apache.curator:curator-recipes:5.1.0 but missing in the actually loaded version org.apache.zookeeper:zookeeper:3.4.8.

For instance, the following missing method(defined in org.apache.zookeeper:zookeeper:3.6.0) are actually referenced by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer, which will introduce a runtime error(i.e., "NoSuchMethodError") into whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer.

1. org.apache.zookeeper.ZooKeeper: void removeAllWatches(java.lang.String,org.apache.zookeeper.Watcher$WatcherType,boolean) is invoked by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer via the following path:

Invocation path------
<org.hongxi.whatsmars.dubbo.demo.consumer.AsyncConsumer2: void main(java.lang.String[])> whatsmars-master/whatsmars-dubbo/whatsmars-dubbo-consumer/target/classes
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: int read()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: void advance()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream openStream()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream sliceStream(java.io.InputStream)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.Closer: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.framework.recipes.watch.PersistentWatcher: void close()> Respository/org/apache/curator/curator-recipes/5.1.0/curator-recipes-5.1.0.jar
<org.apache.curator.framework.imps.RemoveWatchesBuilderImpl: java.lang.Object forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.RemoveWatchesBuilderImpl: java.lang.Void forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.RemoveWatchesBuilderImpl: void pathInForeground(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.zookeeper.ZooKeeper: void removeAllWatches(java.lang.String,org.apache.zookeeper.Watcher$WatcherType,boolean)>

2. org.apache.zookeeper.ZooKeeper: void removeWatches(java.lang.String,org.apache.zookeeper.Watcher,org.apache.zookeeper.Watcher$WatcherType,boolean) is invoked by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer via the following path:

Invocation path------
<org.hongxi.whatsmars.dubbo.demo.consumer.AsyncConsumer2: void main(java.lang.String[])> whatsmars-master/whatsmars-dubbo/whatsmars-dubbo-consumer/target/classes
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: int read()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: void advance()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream openStream()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream sliceStream(java.io.InputStream)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.Closer: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.framework.recipes.watch.PersistentWatcher: void close()> Respository/org/apache/curator/curator-recipes/5.1.0/curator-recipes-5.1.0.jar
<org.apache.curator.framework.imps.RemoveWatchesBuilderImpl: java.lang.Object forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.RemoveWatchesBuilderImpl: java.lang.Void forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.RemoveWatchesBuilderImpl: void pathInForeground(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.zookeeper.ZooKeeper: void removeWatches(java.lang.String,org.apache.zookeeper.Watcher,org.apache.zookeeper.Watcher$WatcherType,boolean)>

3. org.apache.zookeeper.ZooKeeper: boolean close(int) is invoked by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer via the following path:

Invocation path------
<org.hongxi.whatsmars.dubbo.demo.consumer.AsyncConsumer2: void main(java.lang.String[])> whatsmars-master/whatsmars-dubbo/whatsmars-dubbo-consumer/target/classes
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: int read()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: void advance()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream openStream()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream sliceStream(java.io.InputStream)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.Closer: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.ConnectionState: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.ConnectionState: void close(int)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.HandleHolder: void closeAndClear(int)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.HandleHolder: void internalClose(int)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.zookeeper.ZooKeeper: boolean close(int)>

4. org.apache.zookeeper.admin.ZooKeeperAdmin: void reconfigure(java.util.List,java.util.List,java.util.List,long,org.apache.zookeeper.AsyncCallback$DataCallback,java.lang.Object) is invoked by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer via the following path:

Invocation path------
<org.hongxi.whatsmars.dubbo.demo.consumer.AsyncConsumer2: void main(java.lang.String[])> whatsmars-master/whatsmars-dubbo/whatsmars-dubbo-consumer/target/classes
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: int read()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: void advance()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream openStream()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream sliceStream(java.io.InputStream)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.Closer: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2$3: void close()> Respository/org/apache/curator/curator-recipes/5.1.0/curator-recipes-5.1.0.jar
<org.apache.curator.framework.imps.DeleteBuilderImpl: java.lang.Object forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.DeleteBuilderImpl: java.lang.Void forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.CuratorFrameworkImpl: void processBackgroundOperation(org.apache.curator.framework.imps.OperationAndData,org.apache.curator.framework.api.CuratorEvent)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.CuratorFrameworkImpl: void performBackgroundOperation(org.apache.curator.framework.imps.OperationAndData)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.OperationAndData: void callPerformBackgroundOperation()> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.ReconfigBuilderImpl: void performBackgroundOperation(org.apache.curator.framework.imps.OperationAndData)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.zookeeper.admin.ZooKeeperAdmin: void reconfigure(java.util.List,java.util.List,java.util.List,long,org.apache.zookeeper.AsyncCallback$DataCallback,java.lang.Object)>

5. org.apache.zookeeper.admin.ZooKeeperAdmin: void init(java.lang.String,int,org.apache.zookeeper.Watcher,boolean) is invoked by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer via the following path:

Invocation path------
<org.hongxi.whatsmars.dubbo.demo.consumer.AsyncConsumer2: void main(java.lang.String[])> whatsmars-master/whatsmars-dubbo/whatsmars-dubbo-consumer/target/classes
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: int read()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: void advance()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream openStream()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream sliceStream(java.io.InputStream)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.Closer: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.ConnectionState: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.ConnectionState: void close(int)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.HandleHolder: void closeAndClear(int)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.HandleHolder: void internalClose(int)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.HandleHolder$1: org.apache.zookeeper.ZooKeeper getZooKeeper()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.utils.DefaultZookeeperFactory: org.apache.zookeeper.ZooKeeper newZooKeeper(java.lang.String,int,org.apache.zookeeper.Watcher,boolean)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.zookeeper.admin.ZooKeeperAdmin: void init(java.lang.String,int,org.apache.zookeeper.Watcher,boolean)>

6. org.apache.zookeeper.ZooKeeper: void addWatch(java.lang.String,org.apache.zookeeper.AddWatchMode,org.apache.zookeeper.AsyncCallback$VoidCallback,java.lang.Object) is invoked by whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer via the following path:

Invocation path------
<org.hongxi.whatsmars.dubbo.demo.consumer.AsyncConsumer2: void main(java.lang.String[])> whatsmars-master/whatsmars-dubbo/whatsmars-dubbo-consumer/target/classes
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: int read()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.MultiInputStream: void advance()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream openStream()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.ByteSource$SlicedByteSource: java.io.InputStream sliceStream(java.io.InputStream)> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.shaded.com.google.common.io.Closer: void close()> Respository/org/apache/curator/curator-client/5.1.0/curator-client-5.1.0.jar
<org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2$3: void close()> Respository/org/apache/curator/curator-recipes/5.1.0/curator-recipes-5.1.0.jar
<org.apache.curator.framework.imps.DeleteBuilderImpl: java.lang.Object forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.DeleteBuilderImpl: java.lang.Void forPath(java.lang.String)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.CuratorFrameworkImpl: void processBackgroundOperation(org.apache.curator.framework.imps.OperationAndData,org.apache.curator.framework.api.CuratorEvent)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.CuratorFrameworkImpl: void performBackgroundOperation(org.apache.curator.framework.imps.OperationAndData)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.OperationAndData: void callPerformBackgroundOperation()> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.curator.framework.imps.AddWatchBuilderImpl: void performBackgroundOperation(org.apache.curator.framework.imps.OperationAndData)> Respository/org/apache/curator/curator-framework/5.1.0/curator-framework-5.1.0.jar
<org.apache.zookeeper.ZooKeeper: void addWatch(java.lang.String,org.apache.zookeeper.AddWatchMode,org.apache.zookeeper.AsyncCallback$VoidCallback,java.lang.Object)>

Suggested fixing solutions:

1. Use configuration <dependencyManagement> to unify the version of library org.apache.zookeeper:zookeeper to be 3.6.0 in the pom file.
2. Declare a direct dependency org.apache.zookeeper:zookeeper:3.6.0 in the whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer's pom file.
3. Use <exclude> to exclude org.apache.zookeeper:zookeeper:3.4.8 in the whatsmars/whatsmars-dubbo/whatsmars-dubbo-consumer's pom file

Please let me know which solution do you prefer? I can submit a PR to fix it.

Thank you very much for your attention.
Best regards,

Dependency tree----

[INFO] org.hongxi:whatsmars-dubbo-consumer:jar:JAWS.SR1
[INFO] +- org.hongxi:whatsmars-dubbo-api:jar:JAWS.SR1:compile
[INFO] |  \- (org.projectlombok:lombok:jar:1.18.18:compile - omitted for duplicate)
[INFO] +- org.apache.dubbo:dubbo:jar:2.7.3:compile
[INFO] |  +- org.springframework:spring-context:jar:5.3.4:compile (version managed from 4.3.16.RELEASE)
[INFO] |  |  +- org.springframework:spring-aop:jar:5.3.4:compile
[INFO] |  |  |  +- (org.springframework:spring-beans:jar:5.3.4:compile - omitted for duplicate)
[INFO] |  |  |  \- (org.springframework:spring-core:jar:5.3.4:compile - omitted for duplicate)
[INFO] |  |  +- org.springframework:spring-beans:jar:5.3.4:compile
[INFO] |  |  |  \- (org.springframework:spring-core:jar:5.3.4:compile - omitted for duplicate)
[INFO] |  |  +- org.springframework:spring-core:jar:5.3.4:compile
[INFO] |  |  |  \- org.springframework:spring-jcl:jar:5.3.4:compile
[INFO] |  |  \- org.springframework:spring-expression:jar:5.3.4:compile
[INFO] |  |     \- (org.springframework:spring-core:jar:5.3.4:compile - omitted for duplicate)
[INFO] |  +- org.javassist:javassist:jar:3.24.1-GA:compile (version managed from 3.20.0-GA)
[INFO] |  +- (io.netty:netty-all:jar:4.1.59.Final:compile - version managed from 4.1.25.Final; omitted for duplicate)
[INFO] |  +- com.google.code.gson:gson:jar:2.8.6:compile (version managed from 2.8.5)
[INFO] |  \- org.yaml:snakeyaml:jar:1.27:compile (version managed from 1.20)
[INFO] +- com.101tec:zkclient:jar:0.9:compile
[INFO] |  \- org.apache.zookeeper:zookeeper:jar:3.4.8:compile
[INFO] |     +- jline:jline:jar:0.9.94:compile
[INFO] |     \- io.netty:netty:jar:3.7.0.Final:compile
[INFO] +- org.apache.curator:curator-recipes:jar:5.1.0:compile
[INFO] |  \- org.apache.curator:curator-framework:jar:5.1.0:compile
[INFO] |     \- org.apache.curator:curator-client:jar:5.1.0:compile
[INFO] |        +- (org.apache.zookeeper:zookeeper:jar:3.6.0:compile - omitted for conflict with 3.4.8)
[INFO] |        +- com.google.guava:guava:jar:20.0:compile (version managed from 27.0.1-jre)
[INFO] |        \- (org.slf4j:slf4j-api:jar:1.7.30:compile - version managed from 1.7.25; omitted for duplicate)
[INFO] +- com.alibaba:fastjson:jar:1.2.70:compile
[INFO] +- ch.qos.logback:logback-classic:jar:1.2.3:compile
[INFO] |  +- ch.qos.logback:logback-core:jar:1.2.3:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.30:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.11:compile
[INFO] +- io.netty:netty-all:jar:4.1.59.Final:compile
[INFO] +- org.projectlombok:lombok:jar:1.18.18:compile
[INFO] +- junit:junit:jar:4.13.2:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:2.2:test (version managed from 1.3)
[INFO] |     \- org.hamcrest:hamcrest:jar:2.2:test
[INFO] +- org.mockito:mockito-core:jar:3.6.28:test
[INFO] |  +- net.bytebuddy:byte-buddy:jar:1.10.20:test (version managed from 1.10.18)
[INFO] |  +- net.bytebuddy:byte-buddy-agent:jar:1.10.20:test (version managed from 1.10.18)
[INFO] |  \- org.objenesis:objenesis:jar:3.1:test
[INFO] +- org.hamcrest:hamcrest-library:jar:2.2:test
[INFO] |  \- (org.hamcrest:hamcrest-core:jar:2.2:test - version managed from 1.3; omitted for duplicate)
[INFO] \- org.assertj:assertj-core:jar:3.18.1:test

wiki里面的有个rocketmq消息消费失败的链接可以改成原文地址

https://www.jianshu.com/p/925a628f3f80

Wiki中错别字

请问您遇到过使用mongodb存储文件时在读取文件时速度特别慢的问题吗？

您好， 该项目没有上传到maven吗，依赖无法下载

Cryptographic API misuse detected

Hi, I am currently looking into projects on github which are parametrically misusing cryptographic APIs for my research and I came across a few instances in your project where I found such misuses. These misuses have been highlighted in research papers such as

• CDRep: Automatic Repair of Cryptographic Misuses in Android Applications

In your source code file DESUtils.java there are two functions encrypt(byte[], byte[]) and decrypt(byte[], byte[]) both of which have misuses:

• At line 56 and 33

Cipher cipher = Cipher.getInstance(PADDING);

First parameter is with value "DES/ECB/PKCS5Padding" which should be any of {AES, Blowfish, DESede, PBEWithHmacSHA224AndAES_128, PBEWithHmacSHA256AndAES_128, PBEWithHmacSHA384AndAES_128, PBEWithHmacSHA512AndAES_128, PBEWithHmacSHA224AndAES_256, PBEWithHmacSHA256AndAES_256, PBEWithHmacSHA384AndAES_256, PBEWithHmacSHA512AndAES_256, RSA}.

• Then in file AESUtils.java there are misuses seen in three functions encrypt(byte[],byte[]), decrypt(byte[],byte[]) and genarateRandomKey()

At line 38 and 64

Cipher cipher = Cipher.getInstance(PADDING);

First parameter (with value "AES/ECB/PKCS5Padding") should be any of AES/{CBC, GCM, PCBC, CTR, CTS, CFB, OFB}
At line 37 and 61

SecretKeySpec seckey = new SecretKeySpec(enCodeFormat,"AES");

First parameter was not properly randomized, it should be randomized using java.security.SecureRandom JCA API.

At line 115

keygen = KeyGenerator.getInstance(PADDING);

First parameter (with value "AES/ECB/PKCS5Padding") should be any of {AES, Blowfish, DESede, HmacSHA224, HmacSHA256, HmacSHA384, HmacSHA512}

I believe fixing these issues would help your product be more secure.