andregribeiro/pod-chaos-monkey

A very bad monkey delete pods randomly on a k8s cluster

Pod Chaos Monkey

An awesome challenge to play with chaos engineering on k8s clusters!
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. License
6. Contact

About The Project

This project provides:

• a script (chaos-monkey-script.sh) which will be added on a Dockerfile that will be responsible for killing pods.
• a Dockerfile to build an image to run inside a kubernetes cluster.
• a job manifest to deploy the image created as the "chaos owner" in a specific namespace (Ex:"workloads")
• a deployment manifest as the "chaos test target" in a specific namespace "workloads" to test the job

(back to top)

Getting Started

Here is where all the information to get this running and understand this challenge will be presented.

Prerequisites

• Kubernetes cluster up and running.

Here is some options to get you started: [Local kubernetes cluster] https://docs.docker.com/get-docker/ (Enable Kubernetes feature) [Local kubernetes cluster] https://minikube.sigs.k8s.io/docs/start/ [Online kubernetes cluster] https://killercoda.com/kubernetes/scenario/a-playground

Usage

1. Create a docker image

    docker build . -t chaos-monkey:0.0.1

2. Deploy the "chaos test target"

   kubectl apply -f k8s/target-deployment.yaml

3. Deploy the "chaos owner" and all dependencies

    kubectl apply -f k8s/chaos-monkey.yaml

(back to top)

Example

Use this space to show useful examples of how a project can be used. Additional screenshots, code examples and demos work well in this space. You may also link to more resources.

1. Create a docker image

[Extra] Scan image to check vulnerabilities. I used the Snyk plugin that Docker Desktop provided.

2. Deploy the "chaos test target". This deployment manifest has a specific label chaosmonkeytarget: "yes" as a security strategy to know which pods are available to be a target for our "chaos-monkey".

3. Deploy the "chaos owner" and all dependencies In this manifest we have all dependencies to provide our chaos-monkey permitions to kill some pods:

• Create a namespace where our chaos-monkey will "live" -> chaosmonkey
• Create a ServiceAccount (monkey-kill) to chaosmonkey use with specfic permissions.
• Create a ClusterRole (modify-pods) to give permitions of get, list, delete and watch for only resources of type "pod".
• Create a RoleBinding (modify-pods-monkeykill-sa) to give permitions to the ServiceAccount monkey-kill be able to use the ClusterRole modify-pods in "workloads" namespace only.
• Create a job to run a chaosmonkey pod that will delete some pods with spefic arguments:
  • "NAMESPACE": Here we can specify a specific namespace.
  • "NR_POD_TO_KILL": Here we can choose how many pods we want to kill.
  • "TIME_INTERVAL" : Here we chan choose the time interval between each kill.

Chaosmonkey job

Chaosmonkey in action with k8s events

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Contact

André Ribeiro - @My Linkedin

Project Link: https://github.com/andregribeiro/pod-chaos-monkey/

(back to top)