Updating Cassandra version on a live cluster

[dynamike]

Updating Cassandra version on a live cluster

[jasobrown]

Our process at Netflix is two steps. First we do a 'binary update' on each node, like this:

• take the node out of gossip and thrift (there nodetool commands for this)
• install the new Priam and/or c* - fix up any symlinks or whatnot
• terminate c*, then terminate Priam
• start Priam (should start c*, which will bring itself back into thrift, gossip, etc)

Then continue for the rest of the nodes in the cluster.
We then update the ASG with an updated AMI onto which we've baked the update c* and Priam codes bases (we package as rpms). This way if a node dies, a new node will come up with the correct/new versions or c*/Priam. At some future point, we will do a 'rolling AMI' upgrade to make sure the new AMI goes out, as well, as this will conatin updates for our internal monitoring tools, etc.

Hope this helps,

-Jason

[dynamike]

Makes sense to me. Thanks!