chimera-deploy/chimera

Chimera automates canary deployments for containerized microservices running on AWS Fargate, orchestrated by AWS ECS, and communicating via AWS App Mesh.

Chimera

Chimera automates canary deployments for containerized microservices that communicate synchronously on AWS. It reduces the risk of introducing bugs that propagate through your microservice infrastructure by gradually shifting traffic to a canary version of your service while automating simple threshold canary analysis.

Currently, Chimera supports canary deployments for microservices that run on ECS/Fargate and communicate via App Mesh. For monitoring metrics, Chimera uses a Prometheus-configured CloudWatch agent that Chimera must set up before it can perform canary deployments.

For more information, please see Chimera's official webpage.

Installation and Setup

Chimera is deployed with Docker-Compose. Please ensure that both Docker and Docker-Compose are running on your machine.

To install Chimera, clone this repository. You can use the following terminal command:

git clone https://github.com/chimera-deploy/Chimera.git && cd Chimera

Once you are inside the Chimera directory, create an .env file and add the following:

AWS_ACCESS_KEY_ID=<your aws access key id>
AWS_SECRET_ACCESS_KEY=<your aws secret access key>

Save that file and close it. Once that is complete, you only need to execute the following command:

docker-compose up

Chimera will now be ready to set up the Prometheus-configured CloudWatch agent in your ECS cluster. You can access Chimera's UI at port 3000. Once there, have Chimera perform the automatic deployment of the CloudWatch agent unless you've already had Chimera do this before. After the CloudWatch agent is set up, your teams can perform canary deployments on your microservices.

To remove Chimera from your machine, simply execute:

docker-compose down

Team

Ohio, USA

• Josh Leath (LinkedIn, GitHub)

Pennsylvania, USA

• Trevor Kelly (LinkedIn, GitHub)

Massachusettes, USA

• Will Rossen (LinkedIn, GitHub)
• Wes Anderson (LinkedIn, GitHub)

Show your support

Give a ⭐️ if you liked this project!

---

chimera's Issues

Generalize CW Agent so that it can be deployed on either public or private subnets

remove config references from getECSService

update `update` function in ECSService.js - change parameter to take value rather than config

Get info for Task Definitions

Determine What User Details are Needed for AppMeshClientConfig

If Chimera is running in the cloud, each user will have to provide their AWS credentials. These are used as a parameter to create the client used by the SDK.

https://docs.aws.amazon.com/en_us/AWSJavaScriptSDK/v3/latest/clients/client-app-mesh/classes/appmeshclient.html

Get Info for ECS Services

Taskdef of canary requires an update on the log prefix

Move functions that create the CloudWatch Agent into a separate module

remove AWS SDK references from chimera file

Right now we have the AWS sdk helper functions returning the response from the AWS client, we should have those functions return the part of the response we actually need for the main deployment file.

For example:

const vnResponse = await createVirtualNode(/* args */);
this.virtualNode = vnResponse.virtualNode;

should just be:

const virtualNode = await createVirtualNode(/* args */);

Pull Info for Virtual Service

taskDefinition is hardcoded in `/pullInfo/tasks.js`

it is called registerTaskDefinition with the originalTaskName as an argument

Write `healthCheck` function

Get info for Virtual Routes

remove config references from createVirtualNode

remove all possible reverences to the config file from createVirtualNode.js using functions from the pullInfo folder

Remove `currentCloudmapInstanceCount` function

Remove Docker labels indicating Envoy's Prometheus metrics endpoint from movieselector toy example

These will have to be added in by Chimera as we're assuming they do not already have a Prometheus service. We'll have to remove this from the initial Gateway Task Definition as well (not just the initial microservices) because Chimera will have to add it.

clean up `createCW` in TaskDefinition.js

only one metric declaration needed
clear unused metric selectors

add route update interval and shift weight to config file

remove config file from updateRoutes

Create a new Security Group for the Cloud Watch Agent on ECS

Currently using the existing security groups on other ECS services. Need to create one for the Cloud Watch Agent and ensure it has all the necessary ingress rules.

Fix bug in first line of `buildCanary`

Get Info for Virtual Gateways

Fix HealthCheck by using Task ID

Merge various functions that interact with AWS SDK into logical modules

Update naming conventions

let the user determine the new names for tasks, nodes, services, ect.

clean up `update` function ins ECSService.js

Remove unneeded files

move AWS queries into the files where they are used

Fix bug in config: ImageURI -> ImageURL

Fix bug in `rollback` function: `ECSService.destroy` needs to be passed `clusterName` not `config`

Get Info for Virtual Routers

Have Chimera deploy roles and ecs resources for Prometheus-configured CloudWatch agent

Need to ensure that all envoy environments have APPMESH_METRIC_EXTENSION_VERSION = 1

clean up `register` method in TaskDefinition.js

add log after build canary is done

Have Chimera add Docker labels indicating Envoy's Prometheus metrics endpoint to microservices upon canary deployment

Get Info for Virtual Nodes

Fix Unhandled Promise Reject in unhappy path

Unhandled promise rejection triggered by line 55 in CloudWatch.js when trying to deploy unhappy path.

created virtual node
registered task definition
created ECS service
waiting for cloudmap
finished deploying canary
attempting to shift traffic
values downstream/ingress 500: []
values downstream/ingress 500: [ 0, 0 ]
values downstream/ingress 500: [ 5, 1, 0, 0 ]
(node:14151) UnhandledPromiseRejectionWarning: Error: the canary is not healthy
    at getHealthCheck (/home/wor101/capstone/strange-chimera/prototype/autoCanarySDK/services/CloudWatch.js:55:11)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:14151) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 593)
(node:14151) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

remove config references from registerTaskDefinition

Remove APPMESH_METRIC_EXTENSION_VERSION from movieselector toy example microservices & gateway

Remove CloudWatch IAM Roles from base infrastructure of movieselector toy example

take out adding of log prefix in task definition

Have Chimera add Docker labels indicating Envoy's Prometheus metrics endpoint to existing Gateway Task Definition

Will also require that Gateway ECS Service references updated Task Definition

Pull info for App mesh

Remove unneeded preliminary update of virtual gateway

The Chimera.setup method only needs to create new IAM roles and the CloudWatch agent.

clean up `getMetricData` function in CloudWatch.js

do a check in `shiftTraffic` to make sure newVersionWeight doesn't pass 100

Handle Creation of Virtual Nodes that Use DNS and not AWS CloudMap

serviceDiscovery:
{ dns:
{
hostname: ${serviceDiscoveryName}.apps.local,
name: virtualNodeName
}
},

Is the 'name' property required?

Clean up shiftTraffic function

separate logic and maybe move into a new file

clean up `getHealthCheck` function

determine fail count logic