This README provides a comprehensive guide on setting up the Dev Portfolio project, showcasing robust DevOps practices. The project involves provisioning an AWS EKS cluster using Terraform, implementing a CI/CD pipeline with GitHub Actions, Docker, Kubernetes, and Argo CD, and ensuring security best practices are followed.
- Overview
- Setup Overview
- Prerequisites
- Setup Instructions
- Workflow Steps
- Monitoring and Visualization
- Cost Management
The Dev Portfolio project focuses on creating a portfolio website. This README guides you through setting up the project's infrastructure, implementing a robust CI/CD pipeline, and ensuring security measures are in place.
This website is Designed/Created by Adrian Hajdin.
![dev-portfolio website](https://private-user-images.githubusercontent.com/32189783/262873415-9b840485-5588-4e4d-92e5-742801a5a149.gif?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ0MzMwMjcsIm5iZiI6MTcyNDQzMjcyNywicGF0aCI6Ii8zMjE4OTc4My8yNjI4NzM0MTUtOWI4NDA0ODUtNTU4OC00ZTRkLTkyZTUtNzQyODAxYTVhMTQ5LmdpZj9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDE3MDUyN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk4ZDFhZmNmZjJhOGZjMzEzMDllMDA0MzBlZmQxNDFiOGIwZjFlY2Q4YzYyYWIwMmYzMmZjZjllOGJhNWEzMGYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.wGNcvDDKM5DHoJnHhTyidlp_xorKD7HcnDibZ4Fe9gc)
The Dev Portfolio project is built using a variety of tools and technologies to ensure robust development, deployment, and monitoring practices. Here's a list of the key technologies and tools used:
This selection of tools and technologies collectively highlights a modern approach to DevOps practices, encompassing development, deployment, security, and monitoring.
project.dev-portfolio_4.mp4
Before you begin, ensure you have the following:
- AWS account
- Terraform installed
- Docker Hub account
- GitHub account
- Slack account (for notifications)
- Clone the project repository:
git clone https://github.com/Saurabhkr952/dev-portfolio.git
- Navigate to the
terraform
directory:cd dev-portfolio/terraform
- Modify the Terraform configuration files to match your environment.
- Run
terraform init
andterraform apply
to provision the EKS cluster.
To ensure the security of the Kubernetes cluster for the Dev Portfolio project, the following measures have been implemented:
- Distroless Image Usage: Utilized distroless images, containing only essential packages, to build Docker images, reducing vulnerability exposure.
- Container Runtime Security: Ensured container runtime security by running containers as non-root users.
- Docker Image Signing: Implemented Docker image signing using COSIGN to guarantee the integrity and authenticity of container images.
- Image Scanning: Integrated Trivy into the CI/CD pipeline for thorough image and container scanning, enhancing security posture.
- Security Scanning: Leveraged Kubescape for comprehensive security scanning based on NSA and MITRE guidelines.
- CIS Benchmark Adherence: Adhered to Kubernetes CIS benchmarks for robust security standards and best practices.
- Secrets Management: Safeguarded Kubernetes secrets using sealed secrets, ensuring encrypted and tamper-proof storage.
- Network Policies: Implemented network policies to restrict inter-pod communication at the OSI layer 3/4, fortifying the cluster's security posture. (Work in Progress)
- RBAC Implementation: Managed User & Permissions within Kubernetes using RBAC, ensuring granular control over access and actions. (Work in Progress)
- mTLS Communication: Established encrypted communication via mutual TLS (mTLS) between pods using service mesh technology, operating at the OSI layer 7, ensuring confidentiality and integrity of data in transit. (Not Implemented)
- etcd Security: Secured the etcd data store, a critical component of Kubernetes, to safeguard cluster configuration and state. (As it is managed Kubernetes cluster so we don't need to worry about etcd. AWS takes responsibility for securing the control plane components, including etcd, as part of the managed service offering.)
- Ingress Security: Enhanced cluster security by securing ingress traffic using TLS encryption, mitigating the risk of eavesdropping and unauthorized access. (Work in Progress)
To set up the workflow, follow these steps and modify the appropriate lines in the provided workflow file (./.github/workflows/multi-stage-workflow.yml
):
-
Build and Push Docker Image Job: Navigate to the
build
job section and make the following changes:- Line 17: Replace
saurabhkr952/dev-portfolio
with your desired image name.
- Line 17: Replace
-
Container Vulnerability Check Job: Navigate to the
scan_upload
job section and make the following change:- Line 29: Replace
'docker.io/saurabhkr952/gdev-portfolio'
with your image name.
- Line 29: Replace
-
Update Manifest Job: Navigate to the
update_manifest
job section and make the following changes:- Line 62: Replace
saurabhkr952/dev-portfolio-manifest
with your manifest repository name. - Line 42: Replace
saurabhkr952/dev-portfolio
with the appropriate image reference. - Line 67: Replace
[email protected]
with your GitHub Gmail. - Line 68: Replace
saurabhkr952
with your GitHub username. - Line 70: Replace
saurabhkr952/dev-portfolio
with your docker image at both place, and also replacedeployment.yaml
with filename.yaml with path. - Line 77: Replace
saurabhkr952/dev-portfolio-manifest
with your manifest repository name.
- Line 62: Replace
-
Slack Workflow Status Job: Navigate to the
slack-workflow-status
job section and make the following changes:- Line 96: Replace
#general
with your channel name.
- Line 96: Replace
To add the necessary secrets, follow these steps:
- Go to the repository settings.
- Under the security settings, access the 'Secrets and Variables' section.
- Inside the 'Secrets and Variables' section, proceed to the 'Actions' subsection.
- Add the following repository secrets:
DOCKERHUB_USERNAME
: Your DockerHub username.DOCKERHUB_TOKEN
: Your DockerHub password.PAT_TOKEN
: Your GitHub Personal Access Token.SLACK_WEBHOOK_URL
: Your Slack webhook URL. Refer to this guide on how to obtain the URL.
Refer to the official Argo CD documentation to install Argo CD on your EKS cluster.
Here's how the CI/CD pipeline works:
-
Developers push code to the GitHub repository.
-
GitHub Actions automatically triggers a workflow when new code is pushed.
-
The workflow starts by building the Docker image and then pushing it to DockerHub.
-
After the Docker image is successfully pushed, the workflow scans the container for vulnerabilities.
-
Following the vulnerability scan, the workflow updates the Kubernetes manifests in the Git repository with the new image tag.
-
Argo CD continuously monitors the Git repository and updates the application version in the Kubernetes cluster.
-
Security vulnerabilities are reported and can be reviewed in the security tab.
-
After the workflow is successfully completed, the status is reflected as shown below:
-
Notifications are sent to Slack to provide information about the workflow status.
Prometheus
and Grafana
. Prometheus serves as the data source for Grafana, enabling the visualization of metrics and performance data. A Grafana dashboard is employed to monitor the Kubernetes cluster using data collected by Prometheus.
Here's a preview of a Grafana dashboard monitoring the Kubernetes cluster:
Kubecost is used for Kubernetes cost allocation and resource management.
Kubernetes Manifest Repository: dev-portfolio-manifest
Note: This project is currently a work in progress. Certain sections are being actively developed and will be updated soon. The following enhancements are planned:
- Detailed instructions on setting up Prometheus and Grafana for monitoring.
- A guide on setting up Kubecost for Kubernetes cost management.