joshuad2/rabbitmqclustermgr

RabbitMQ configuration management database, provisioner, and heuristic analysis of traffic.

RabbitMQClusterMgr

RabbitMQClusterMgr - RabbitMQ configuration management database, provisioner, and heuristic analysis of traffic.

This project is a result of many discussions around the use and provisioning of RabbitMQ which has become pervasive in the past year.  The issue is that when you initially setup RabbitMQ and expose it to the different needs of the business it typically will fail.  As the use increases, the setup out of the box does not suffice.  This is where the RabbitMQClusterMgr comes into play.  The plan is for it to be a small clustered application that will work in a Cloud Foundry, Docker Swarm, Cloud (AWS or GCE) or bare metal environment.

Note that this code base uses ideas and code snippets from the RabbidManagement project https://github.com/Berico-Technologies/RabbidManagement

This project is a REST service that provides a set of RESTful endpoints protected by OAuth 2. The REST service is based on the Building a RESTful Web Service getting started guide. This project incorporates the new Java-based configuration support, now available in Spring Security OAuth 2.0.

This project builds virtual RabbitMQ clusters in an internal database through a provided resource based REST interface. This set of clusters is a "Project" which has clusters and then nodes. The purpose of this is to have one place where a RabbitMQ Federation or just one RabbitMQ cluster configuration and performance statistics can be stored/analyzed. By doing this it enables the provisioning of the cluster through Docker containers that can be performed by via a statistical model based upon a tuned model described by the user or through the default model. The purpose of the RabbitMQClusterMgr is to sit outside the PAAS or IAAS layer and predict the workload and use the scheduling API's to

• BOSH release
• Docker Swarm
• Google Kubernetes on Amazon Web Services or Google Compute Engine
• Apache Mesos on Amazon Web Services or Google Compute Engine
• Apache Brooklyn on Amazon or Google Compute Engine

Also, this application will work directly to handle no-downtime maintenance of RabbitMQ clusters on either Kubernetes, Mesos, or Brooklyn via their scheduling plugins/services.

The purpose of the RabbitMQClusterMgr service is that once the project is provisioned, it will continually collect heuristic information about the cluster.  This would be things like 

• Drive space and memory used by queues
• Memory used by the system overall
• Number of Exchanges created
• Use of Federation
• Use of Shovel

Using the original template by which the project (and it's clusters) are created the RabbitMQClusterMgr will have the capability to automatically create the new cluster in the environments and then transition the load to the new cluster(s).  An HAProxy instance will be used to to create the no-downtime upgrade/deployment.  In the future it should support hardware based load balancers.

Dependencies

• Apache Geode
• Spring Boot
• Spring Data GemFire