Migrate Kairos and Cassandra out of DC/OS

[astronomer]

Migrate Kairos and Cassandra out of DC/OS

[schnie]

https://github.com/kairosdb/kairosdb/releases

Version 1.2.0 switches from Thrift to CQL. I believe our current version is 1.1.3, the prior version. I tried this a while back and got thrift related errors on a compose ScyllaDB instead of Cassandra.

Thinking that if we update to the latest version, we'll be able to connect KairosDB to a hosted ScyllaDB. Compose support engineer seemed to think it would work too.

@thisisamurray let me know when you get to this, and we can put a migration plan together.

[thisisamurray]

Kairos to Scylla Migration Plan

• Update https://github.com/astronomerio/docker-openjdk8-kairosdb to use latest stable Kairos version.
• Startup Scylladb on Compose
• Start new Kairos DB that is pointing to Scylla
• Verify that it is functioning properly
• Update logstash.ingestion-tracking to point to New Kairos
• Migrate Old Kairos' data to New Kairos

Notes

• https://www.scylladb.com/2017/05/24/9-steps-building-highly-available-time-series-solution-scylla-kairosdb/

[thisisamurray]

@ryw @schnie

Progress Report:

Running into issues w/ remote connection into new Scylla, opened a ticket with Compose to help me gain access. Once I have access, will be able to create the necessary tables for Kairos, then start up the new one for us to point data towards.

Hopefully downhill from there.

[thisisamurray]

@ryw @schnie

Still have not been able to resolve connection issues, even with schnie and cwurtz also looking into it. Have only received one response to my two open Compose support tickets, with the one being a half response. Hopefully Compose can clear up any hurdles remaining.

[thisisamurray]

Month later, two new scylla instances on compose... still cannot connect successfully. Will hit it again next week, maybe try a different OS 🤷‍♂️

[thisisamurray]

Got a successful connecting by sipping up ubuntu in docker. Proceeding to get a version of Kairos connected to it

[thisisamurray]

Okay, new issue. Cannot get Kairos on ECS to connect to Scylladb on Compose. Seems that, in order to support the auth over the external network, that Kairos will need to use an Address Translation Map. This is supported by Compose, but doesn't seem to be native to Kairos... we may need to fork Kairos to add that java functionality. @schnie Got any insight to whether or not it's worth perusing me learning enough java to get this project forked and updated, or should I look for a different solution (either getting Cassandra to run on ECS or new timeseries engine)?

[schnie]

@thisisamurray catching up here. What's the difference between the Kairos you got running and connected 4 days ago, compared to the one running on ECS?

[ryw]

wrong repo