problem deploying cassandra-mesos with marathon

[mesosphere-backup]

problem deploying cassandra-mesos with marathon

[BenWhitehead]

@kikicarbonell Can you also attache cassandra-mesos.log from the sandbox that contains the actual logs from the scheduler process.

[kikicarbonell]

@BenWhitehead please, can you read all logs from: http://stackoverflow.com/questions/32830901/cassandra-mesos-health-check-fail-when-deploy-with-marathon

[BenWhitehead]

Hi @kikicarbonell The ports used by cassandra can be found here in the official documentation. We don't start OpsCenter so you can disregard those ports.

[kikicarbonell]

@BenWhitehead well I was opening port: 7000,7001,9042,9160,7199,22 but not work? if possible that made some problem around mesos framework?

[BenWhitehead]

Possibly, I thought you were saying that the farmwork tried to start the tasks but cassandra wasn't able to ever form the ring.

I'll try and outline all of the network connections I know about in play here.

1. Cassandra-Mesos framework scheduler need to communicate with zk 2181 to persist it's state, locate the leading mesos-master. Needs to be able to communicate with the mesos master at port 5050.
2. The mesos-master then needs to be able to communicate back with the scheduler on a random port selected by libmesos at runtime (it's in the random port range the kernel reserves).
3. marathon will need to be able to communicate with the scheduler on $PORT0 in order to issue health checks against the http api.
4. The Custom Executor needs to be able to connect to 5051 to communicate with the mesos-agent
5. The mesos-agent will need to be able to communicate back with the executor on a random port selected by libmesos at runtime (it's in the random port range the kernel reserves).
6. Each Server task will bind against 7000,7001,7199,9042,9160. Port 7000 is used by cassandra for it's internode gossip communication.
7. Each Custom Executor will need to be able to communicate with localhost:7199 in order to connect to the JMX MBeans for nodetool commands (health checks, cluster jobs, shutdown, etc.)
8. Each Custom Executor will need to be able to communicate with the random scheduler port bound in note 2 above. (this is for Framework Messages sent from the custom executor to the scheduler with information about health check and a couple other things that don't necessitate a task.)
9. Then each client will need to be able to access 9042/9160 of each node running a server so that it can interact with cassandra itself.

Hopefully this can help you track down the packet breakdown. I'd be interested in knowing what ends up working for you.

[kikicarbonell]

@BenWhitehead thks for your detailed explanation about process and port used on
cassandra-mesos fremework, after this I suppouse that the problem will be
the random port used by mesos (step 5 in your description), then I create a
rule on iptable to allow tcp traffic betwen al mesos nodes, it' s a
temporal solution but it work.

The iptable command to do it:

iptable -A INPUT -p tcp -s <hostname> -j ACCEPT;

[triclambert]

This repository is deprecated. Please use the latest supported framework here: https://github.com/mesosphere/dcos-cassandra-service, and provide feedback for us here: https://dcosjira.atlassian.net/browse/CASSANDRA.