brianmhess/cassandra-count

Count rows in Cassandra Table

cassandra-count's Issues

An OperationTimedOutException occurred. Try increasing -numSplits or reducing -splitSize

Query a large table(more than 70,000,000 rows) always return error.
Use -splitSize 1 can't help

cassandra-count should error out on C* 2.0, or fix the dependency on size_estimates table

It appears that if you try to use cassandra-count against a C* 2.0 table, because 2.0 doesn't have system.size_estimates table yet, it will error out with some obscure error messages. We should either remove the dependency on this system table information, or catch this error and prevent cassandra-count from running against a C* cluster lower than version 2.1.

Tried to use this tool to count rows in latest DSE Cassandra and getting incorrect result - 0 rows for any table

Cassandra 3.11.0.1900 | DSE 5.1.5
dbaadmin@cqlsh> select count(*) from devsp1_fcast.cassandra_migration_version limit 100000000;

count

 5

this proc DEBUG output:
[cassandra@dld-awurxcass01 ~]$ java -jar cassandra-count -host dld-awurxcass01 -user dbaadmin -pw ******* -keyspace devsp1_fcast -table cassandra_migration_version -consistencyLevel LOCAL_QUORUM -debug 2
Version: 0.0.5
Splitting by size: 2097152
Splitting (-1096338938064404326,-1039471400470673160] into 1 splits
(-1096338938064404326, -1039471400470673160]
Splitting (-140507488250420230,-123284837239520266] into 1 splits
(-140507488250420230, -123284837239520266]
Splitting (-1683629337682414,48047111993801791] into 1 splits
(-1683629337682414, 48047111993801791]
Splitting (-276207253566777732,-260146363320785296] into 1 splits
(-276207253566777732, -260146363320785296]
Splitting (-2874160338427404896,-2774609892308133676] into 1 splits
(-2874160338427404896, -2774609892308133676]
Splitting (-3282038318922351216,-3196889832701303141] into 1 splits
(-3282038318922351216, -3196889832701303141]
Splitting (-4285554009235794871,-4241390097831102389] into 1 splits
(-4285554009235794871, -4241390097831102389]
Splitting (-4448215975809191401,-4439018156743759650] into 1 splits
(-4448215975809191401, -4439018156743759650]
Splitting (-4812944815252101189,-4810635237789321971] into 1 splits
(-4812944815252101189, -4810635237789321971]
Splitting (-5193012636959512121,-4996436333230203806] into 1 splits
(-5193012636959512121, -4996436333230203806]
Splitting (-6270244668766646436,-6233830326431550289] into 1 splits
(-6270244668766646436, -6233830326431550289]
Splitting (-637601314034166764,-601583909945053562] into 1 splits
(-637601314034166764, -601583909945053562]
Splitting (-6836195053147453644,-6829279523672589051] into 1 splits
(-6836195053147453644, -6829279523672589051]
Splitting (-7347039922211848666,-6946364529035290199] into 1 splits
(-7347039922211848666, -6946364529035290199]
Splitting (-8116895881643764758,-8010067108330285253] into 1 splits
(-8116895881643764758, -8010067108330285253]
Splitting (-8935561605358643441,-8882565820920956969] into 1 splits
(-8935561605358643441, -8882565820920956969]
Splitting (1969544382596904694,2053373484337786146] into 1 splits
(1969544382596904694, 2053373484337786146]
Splitting (2380245687854922619,2457575609527932303] into 1 splits
(2380245687854922619, 2457575609527932303]
Splitting (3105037024683861550,3216999442406897807] into 1 splits
(3105037024683861550, 3216999442406897807]
Splitting (3477073448056510741,3494338976265345816] into 1 splits
(3477073448056510741, 3494338976265345816]
Splitting (3606271877448556197,3620652125658179147] into 1 splits
(3606271877448556197, 3620652125658179147]
Splitting (3871151482211312118,3959439699880611217] into 1 splits
(3871151482211312118, 3959439699880611217]
Splitting (4524068025875944671,4534382880239323224] into 1 splits
(4524068025875944671, 4534382880239323224]
Splitting (4974281547384513255,5000215653813396878] into 1 splits
(4974281547384513255, 5000215653813396878]
Splitting (5585650408328373773,5596506786177330462] into 1 splits
(5585650408328373773, 5596506786177330462]
Splitting (6588137736033412028,6657376467345790293] into 1 splits
(6588137736033412028, 6657376467345790293]
Splitting (6657376467345790293,6688847055777766078] into 1 splits
(6657376467345790293, 6688847055777766078]
Splitting (7820870107791498531,7905990160887596734] into 1 splits
(7820870107791498531, 7905990160887596734]
Splitting (8459724375824154567,8532970556766479266] into 1 splits
(8459724375824154567, 8532970556766479266]
Splitting (8686620659637275012,8698758863479850310] into 1 splits
(8686620659637275012, 8698758863479850310]
Splitting (8698758863479850310,8752063909078361054] into 1 splits
(8698758863479850310, 8752063909078361054]
Splitting (8966242228595437071,8977428669139837736] into 1 splits
(8966242228595437071, 8977428669139837736]
Total ranges: 32
Query: SELECT COUNT(*) FROM devsp1_fcast.cassandra_migration_version WHERE Token(version) > ? AND Token(version) <= ?
Executing: -1096338938064404326 -1039471400470673160
Executing: -140507488250420230 -123284837239520266
Executing: -1683629337682414 48047111993801791
Executing: -276207253566777732 -260146363320785296
Executing: -2874160338427404896 -2774609892308133676
Executing: -3282038318922351216 -3196889832701303141
Executing: -4285554009235794871 -4241390097831102389
Executing: -4448215975809191401 -4439018156743759650
Executing: -4812944815252101189 -4810635237789321971
Executing: -5193012636959512121 -4996436333230203806
Executing: -6270244668766646436 -6233830326431550289
Executing: -637601314034166764 -601583909945053562
Executing: -6836195053147453644 -6829279523672589051
Executing: -7347039922211848666 -6946364529035290199
Executing: -8116895881643764758 -8010067108330285253
Executing: -8935561605358643441 -8882565820920956969
Executing: 1969544382596904694 2053373484337786146
Executing: 2380245687854922619 2457575609527932303
Executing: 3105037024683861550 3216999442406897807
Executing: 3477073448056510741 3494338976265345816
Executing: 3606271877448556197 3620652125658179147
Executing: 3871151482211312118 3959439699880611217
Executing: 4524068025875944671 4534382880239323224
Executing: 4974281547384513255 5000215653813396878
Executing: 5585650408328373773 5596506786177330462
Executing: 6588137736033412028 6657376467345790293
Executing: 6657376467345790293 6688847055777766078
Executing: 7820870107791498531 7905990160887596734
Executing: 8459724375824154567 8532970556766479266
Executing: 8686620659637275012 8698758863479850310
Executing: 8698758863479850310 8752063909078361054
Executing: 8966242228595437071 8977428669139837736
devsp1_fcast.cassandra_migration_version: 0

Exception in thread "main" java.lang.NullPointerException

我的测试表里面没有一条数据，我执行
./cassandra-count -host xxx -user xxx -pw xxx -keyspace xxx -table xxx -debug 2
就会报错，请问是什么原因啊？
Version: 0.0.6
Splitting into 256 splits
Splitting 257 ranges each into 0 splits
Total ranges: 257
Exception in thread "main" java.lang.NullPointerException
at hessian.CqlCount.prepareStatement(CqlCount.java:433)
at hessian.CqlCount.run(CqlCount.java:470)
at hessian.CqlCount.main(CqlCount.java:529)

Enhancement: detect incorrect keyspace replication setting and warn about it

I was helping a user to try out cassandra-count and they ran into some interesting timeout errors like the following:

$ ./cassandra-count -host 127.0.0.1 -keyspace testks -table testcf -numFutures 10 -consistencyLevel ALL
An OperationTimedOutException occurred. Try increasing -numSplits or reducing -splitSize

Turned out their RF setting in testks keyspace is wrong, as this is a single node cluster, and they use SimpleStrategy with RF=3. Since this is an error condition that's often encountered by green user, can we add some logic to check RF setting for the target keyspace and complain about it when it detects an impossible condition (such as trying to get 3 replicas on a single-node cluster)?