instaclustr/cassandra-ttl-remover

Tool for rewriting SSTables to not contain TTLs

cassandra-ttl-remover's Issues

Support 3.11.x

2 PR's necessary, one for each C* branch supporting 3.0 and 3.11.

TTLRemover do not compile with Cassandra 2.2

Hello,

Very useful and instructive piece of software, thank you to share.

Anyway, I've followed the instructions on the Readme, but it didn't compile:

    [javac] /home/ferran/TTLRemover/cassandra-2.2-src/src/java/org/apache/cassandra/noTTL/NoTTLReader.java:1958: error: cannot find symbol
    [javac]         return sstableMetadata.replayPosition;
    [javac]                               ^
    [javac]   symbol:   variable replayPosition
    [javac]   location: variable sstableMetadata of type StatsMetadata

And compiler is right, this variable do not exist... on 2.2 (but it exists on 1.6)

To quickly solve the issue I've just deleted the code calling this (noTTL/NoTTLReader.java:1958):

    public ReplayPosition getReplayPosition()
    {
        return sstableMetadata.replayPosition;
    }

And it works!

TTL Remover not working in Cassandra 4.

Run.sh for Cassandra 4:

For Cassandra 3 and 4.0

CLASSPATH=$CLASSPATH./impl/target/ttl-remover.jar:./cassandra-4/target/ttl-remover-cassandra-4.jar

change versions of jars on classpath to target 3 or 4

change --cassandra-version if necessary

java -javaagent:/opt/cassandra-ttl-remover/buddy-agent/target/byte-buddy-agent.jar
-cp "/opt/cassandra-ttl-remover/impl/target/ttl-remover.jar:/opt/cassandra-ttl-remover/cassandra-4/target/ttl-remover-cassandra-4.jar"
$JVM_OPTS
com.instaclustr.cassandra.ttl.cli.TTLRemoverCLI
--cassandra-version=4
--sstables
/var/lib/cassandra/data/cycling
--output-path
/var/lib/cassandra/data/cycling/stripped
--cql
'CREATE TABLE IF NOT EXISTS test.test (id uuid, name text, surname text, PRIMARY KEY (id)) WITH default_time_to_live = 10;'

=============================================================================
Connected to Test Cluster at 127.0.0.1:9042
[cqlsh 6.0.0 | Cassandra 4.0.7 | CQL spec 3.4.5 | Native protocol v5]
Use HELP for help.
cqlsh> show version;
[cqlsh 6.0.0 | Cassandra 4.0.7 | CQL spec 3.4.5 | Native protocol v5]
cqlsh>

Below is the error Log while running run.sh.

root@:/opt/cassandra-ttl-remover# sh run.sh
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.util.ServiceConfigurationError: com.instaclustr.cassandra.ttl.SSTableTTLRemover: com.instaclustr.cassandra.ttl.Cassandra4TTLRemover Unable to get public no-arg constructor
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:582)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:673)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1233)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1265)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1300)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1385)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at com.instaclustr.cassandra.ttl.cli.TTLRemoverCLI.getTTLRemover(TTLRemoverCLI.java:134)
at com.instaclustr.cassandra.ttl.cli.TTLRemoverCLI.run(TTLRemoverCLI.java:100)
at picocli.CommandLine.executeUserObject(CommandLine.java:1919)
at picocli.CommandLine.access$1100(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
at picocli.CommandLine.execute(CommandLine.java:2058)
at com.instaclustr.cassandra.ttl.cli.TTLRemoverCLI.execute(TTLRemoverCLI.java:119)
at com.instaclustr.cassandra.ttl.cli.TTLRemoverCLI.main(TTLRemoverCLI.java:77)
at com.instaclustr.cassandra.ttl.cli.TTLRemoverCLI.main(TTLRemoverCLI.java:73)
Caused by: java.lang.NoClassDefFoundError: org/apache/cassandra/db/lifecycle/ILifecycleTransaction
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
at java.base/java.lang.Class.getConstructor0(Class.java:3342)
at java.base/java.lang.Class.getConstructor(Class.java:2151)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:660)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:657)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:668)
... 23 more
Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.db.lifecycle.ILifecycleTransaction
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)

=================================================================
echo $JAVA_HOME
/opt/zulu11.60.19-ca-jdk11.0.17-linux_x64
root@# echo $CASSANDRA_HOME

root@:/opt/cassandra-ttl-remover# echo $CLASSPATH
/opt/cassandra-ttl-remover-1.1.2/impl/target/ttl-remover.jar:/opt/cassandra-ttl-remover-1.1.2/cassandra-4/target/ttl-remover-cassandra-4.jar

Error in TTL Remover Script

Hi Team ,

There is some typo in TTL remover script .
"$JAVA" $JAVA_AGENT -cp "$CLASSPATH" $JVM_OPTS -Dstorage-config="$CASSANDRA_CONF"
-Dcassandra.storagedir="$cassandra_storagedir"
-Dlogback.configurationFile=logback-tools.xml
org.apach

It should be
"$JAVA" $JAVA_AGENT -cp "$CLASSPATH" $JVM_OPTS -Dstorage-config="$CASSANDRA_CONF"
-Dcassandra.storagedir="/opt/cassandra/"
-Dlogback.configurationFile=logback-tools.xml
org.apache.cassandra.noTTL.TTLRemover

Also after making changes , its giving the following error .
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.cli.ParseException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more

Exception in thread "main" org.apache.cassandra.db.KeyspaceNotDefinedException: Keyspace system_schema does not exist

Hi Team ,

While executing TTL remover , its giving below given error .

./TTLRemover /opt/cassandra/data/feb_data/table_1-9fbb0adc48f711e8a8b7e55aaace1278/ mc-56-big-Data.db -p /opt/cassandra/data/feb_ttl_drop/

=======
error :

Exception in thread "main" org.apache.cassandra.db.KeyspaceNotDefinedException: Keyspace system_schema does not exist
at org.apache.cassandra.thrift.ThriftValidation.validateKeyspace(ThriftValidation.java:84)
at org.apache.cassandra.thrift.ThriftValidation.validateColumnFamily(ThriftValidation.java:108)
at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:893)
at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:888)
at org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:515)
at org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:224)
at org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:268)
at org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:276)
at org.apache.cassandra.schema.SchemaKeyspace.query(SchemaKeyspace.java:1239)
at org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:861)
at org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:853)
at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:136)
at org.apache.cassandra.noTTL.TTLRemover.main(TTLRemover.java:160)