ratnakarreddyg/aws-jenkins-study

:school: Case study using AWS to setup Jenkins

aws-jenkins-study

File me tickets or star the repo.

Table of Contents

• aws-jenkins-study
• Scenarios
  • Scenario-101: Docker Single-Node Jenkins Deployment I
  • Scenario-102: Docker Single-Node Jenkins Deployment II
  • Scenario-201: VM Single-Node Jenkins Deployment I
  • Scenario-202: VM Single-Node Jenkins Deployment II
  • Scenario-203: VM Single-Node Jenkins Deployment III
  • Scenario-301: VM 2-Nodes Jenkins Deployment I
  • Scenario-302: VM 2-Nodes Jenkins Deployment II
  • Scenario-401: ECS Jenkins Deployment I
  • Scenario-402: ECS Jenkins Deployment II
  • Scenario-403: ECS Jenkins Deployment III
  • Scenario-501: Large Scale Jenkins Deployment I
• Highlights
• Contributors: Give People Credits
• License

Case study using AWS TechStack to setup Jenkins env

Scenarios

Scenario-101: Docker Single-Node Jenkins Deployment I

• Objective: Deploy Docker container in AWS
• Requirements:

1. Start an EC2 instance by cloudformation
2. Provision the instance as docker daemon
3. Setup Jenkins container inside the instance

• Main Tech: Cloudformation, Docker
• See more: Scenario-101

Scenario-102: Docker Single-Node Jenkins Deployment II

• Objective: Customize Jenkins docker deployment in AWS
• Requirements:

1. Finish Scenario-101, create a jenkins user by code.
2. Anonymous user can't open the jenkins. Only login user can.
3. When Jenkins is down, get alerts
4. Make sure Jenkins GUI changes can be seamlessly tracked in git repo.

• Main Tech: Cloudformation, Docker
• See more: Scenario-102
• TODO

Scenario-201: VM Single-Node Jenkins Deployment I

• Objective: We need a live Jenkins env in public Cloud. Fast and easy.
• Requirements:

1. Use cloudformation to start an EC2 instance
2. Start Jenkins inside the EC2 instance

• Main Tech: Cloudformation, Chef
• See more: Scenario-201

Scenario-202: VM Single-Node Jenkins Deployment II

• Objective: Customize Jenkins docker deployment in AWS
• Requirements:

1. Finish Scenario-201, create a jenkins user by code.
2. Create a dedicated VPC for the jenkins. And allow selective source IP to access.
3. Anonymous user can't open the jenkins. Only login user can.
4. Make sure Jenkins GUI changes can be seamlessly tracked in git repo.

• Main Tech: Cloudformation, Chef, VPC, Slack
• See more: Scenario-202

Scenario-203: VM Single-Node Jenkins Deployment III

• Objective: Customize Jenkins docker deployment in AWS
• Requirements:

1. Finish Scenario-202
2. Enable CloudWatch monitoring for Jenkins
3. When Jenkins is down, get slack notification
4. In case Jenkins master crash, create an auto-scaling group. And configure the instance count as 1

• Main Tech: Cloudformation, Chef, VPC, CloudWatch, Slack
• See more: Scenario-203
• TODO

Scenario-301: VM 2-Nodes Jenkins Deployment I

• Objective: Avoid SPOF by adding 2 Jenkins instance
• Requirements:

1. Start 2 Jenkins instance behind one ALB
2. Enable auto-scaling with minimum instance 2 and max 4.

• Main Tech: Cloudformation, Chef, VPC, CloudWatch, Jenkins Slack Integration, ALB
• See more: Scenario-301
• TODO
• TODO: how the 2 Jenkins instance gonna to coordinate with each other?

Scenario-302: VM 2-Nodes Jenkins Deployment II

• Objective: Jenkins cluster deployment
• Requirements:

1. Start 1 jenkins master and 1 jenkins slave
2. Enable auto-scaling for Jenkins master. With instance count 1
3. Enable auto-scaling for Jenkins slaves. With instance count range from 1 to 3
4. Customized VPC to allow limited network access

• Main Tech: Cloudformation, Chef, VPC, CloudWatch, EBS, Jenkins Slack Integration, ALB
• See more: Scenario-302
• TODO

Scenario-401: ECS Jenkins Deployment I

• Objective: Get exposed to docker orchestration service.
• Requirements:

1. Start ECS with 1 node
2. Install a single Jenkins instance

• Main Tech: Cloudformation, ECS, EBS
• See more: Scenario-401
• TODO

Scenario-402: ECS Jenkins Deployment II

• Objective: Deploy a 2-nodes Jenkins cluster
• Requirements:

1. Start ECS with 2 node
2. Start Jenkins service with 2 instances in ECS. One for master, one for slave.
3. Enable ALB for Jenkins master

• Main Tech: Cloudformation, ECS, ELB, CloudWatch, ALB
• See more: Scenario-402
• TODO
• TODO: How to avoid Jenkins SPOF, theoretically speaking?

Scenario-403: ECS Jenkins Deployment III

• Objective: Deploy Jenkins 1 Master 3 Slaves with 2 nodes in ECS2
• Requirements:

1. Start ECS with 2 node
2. Start Jenkins service within ECS. 1 Master and 3 Slaves
3. Enable ALB for Jenkins master

• Main Tech: Cloudformation, ECS, ELB, CloudWatch, ALB
• See more: Scenario-403
• TODO
• TODO: How to avoid Jenkins SPOF, theoretically speaking?

Scenario-501: Large Scale Jenkins Deployment I

• Objective: Suppose you have 1K developers to use your Jenkins. Improve scalability, availability, security, etc.
• Requirements:

1. Scalability: multiple Jenkins master instances
2. Availability: Jenkins slave; Jenkins Master
2. Security: VPC, Jenkins authentication integration

• Main Tech: Cloudformation, ECS, EBS, ALB
• See more: Scenario-501
• TODO

Highlights

• Principles:

1. Fully automated
2. Improve HA

• Really like container deployments over VM deployments.

• Jenkins community don't have HA solution for Jenkins master yet.

• TODO: speed up the whole setup process

• TODO: reduce the deployment failure rate

• More Resources:

https://github.com/awslabs/startup-kit-templates
https://github.com/awslabs

Contributors: Give People Credits

Below are folks who have contributed via GitHub!

License

• Code is licensed under MIT License.