tungbq/devops-toolkit

🐳 Container image for an all-in-one DevOps environment with popular tools like Ansible, Terraform, kubectl, Helm, AWS CLI, Azure CLI, Git, Python and more.

devops-toolkit's Issues

container: create Dockerfile for the devops-toolkit

Based ubuntu:22.04 (or other??)
Install:

• python3
• pip3

ci: update checkout github action step to v4

https://github.com/tungbq/devops-toolkit/blob/main/.github/workflows/docker-image.yml#L11
Change from v3 -> v4

ci: update the scripts related to check version to allow using specify minor version

Like instead of using python 3.12.x, we could find a way to specify the python 3.11.x, so we only check that latest version

Improve the logging of `check_version_in_toolkit.sh` script

To improve the log, we need a minor change:

devops-toolkit/scripts/check_version_in_toolkit.sh

Line 23 in 5a9f72b

echo "[OK] $tool_name version is correct: $installed_version"

Suggest changes:

• From: echo "[OK] $tool_name version is correct: $installed_version"
• To: echo "[OK] $tool_name version is correct: $expected_version"

Define the Release and branching strategy

Define the Release and branching strategy, considering:

• SEM versioning
• New release CI workflow
• When to release
• Stable vs latest version?

Update issue open instruction the content of `README.md`

This task is to update the content of README.md, change as below:
from:
- Feel free to open a new issue if you want to request more content about DevOps Dockerfile
to:
- Feel free to open a new issue if you encounter the toolkit bug or want to request more content about DevOps toolkit
Or better sentence that you could suggest. Thanks!

usage: create doc to guide user build custom version of the toolkit

e.g:

docker build \
  --build-arg UBUNTU_VERSION=22.04 \
  --build-arg PYTHON_VERSION=3.11.3 \
  --build-arg ANSIBLE_VERSION=2.16.4 \
  --build-arg TERRAFORM_VERSION=1.7.4 \
  --build-arg KUBECTL_VERSION=1.29.2 \
  --build-arg HELM_VERSION=3.14.2 \
  --build-arg AWSCLI_VERSION=2.15.24 \
  -t custom-docker-image:latest .

code: implement dockerfile linter for code quality

Reference tool: https://github.com/hadolint/hadolint

• Dockerfile linter, validate inline bash, written in Haskell
• docker run --rm -i hadolint/hadolint < Dockerfile

docker: add Ansible to the toolkit

Install Ansible in the toolkit:

• Follow official document
• Versioning allowed via ARG
• https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-and-upgrading-ansible-with-pip

security: Vulnerabilities scanning for the release docker image

security: Vulnerabilities scanning for the release docker image
Using Trivy?

core: add awscli to the toolkit

core: add awscli to the toolkit:

• Doc: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#cliv2-linux-install
• https://docs.aws.amazon.com/cli/latest/userguide/getting-started-version.html

testing: write a shell script to test all version of toolkit locally

testing: write a shell script to test all version of toolkit locally

• something that executes command like this: docker run --rm devops-toolkit python3 --version
• and support for all the devops tools in toolkit
• should be easy to extend when there has new tool comes in to the toolbox

testing: Add basic test for newly created image

ci: Add basic test for newly created image

• versioning check: #28
• ci checker: #30

core: add docker to the toolkit?

Using concept dind docker in docker

doc: Have an instruction section for `podman` container run time

Along with the docker instruction it's nice to have an instruction section for podman container run time

deployment: investigate the Deployment feature of Github

Some area to be explored:

• Pricing for personal Github public repository
• Pros and cons
• Security concern

core: develop a `helper` script to show the toolkit info

It would be informative to have a script or command to show current version of tools in DevOps toolkit container.
Something like:

toolkit-info

OS: Ubuntu-22.04
Arch: amd64
UBUNTU_VERSION=22.04 
PYTHON_VERSION=3.11.3 
ANSIBLE_VERSION=2.16.3 
TERRAFORM_VERSION=1.7.0 
KUBECTL_VERSION=1.29.2 
HELM_VERSION=3.14.2 
AWSCLI_VERSION=2.15.24 

ci: Skip the docker-image workflow when there's only change in *.md

To save the runner resource (like this case #13), we could:

• ci: Skip the docker-image workflow when there's only change in *.md

process: draft release plan

process: draft release plan

• publish docker image to dockerhub per release?
• how to automate update tool verions
• release changelog process

docker: Add terraform to the toolkit

https://www.terraform.io/

• Install: https://developer.hashicorp.com/terraform/install

usage: init sample usage

usage: init sample usage

Have an sample folder for each tool which contains the basic usage of the tool.

Start with Ansible first then open follow up tickets to adapt for other tools

ci: support manual trigger capability in main CI workflow

Workflow: https://github.com/tungbq/devops-toolkit/blob/main/.github/workflows/docker-image-main.yml

Support manual trigger capability in main CI workflow by using workflow_dispatch:
Doc: https://docs.github.com/en/actions/deployment/about-deployments/deploying-with-github-actions#triggering-your-deployment

ci: Pull request created in a GitHub action does not trigger workflows with pull_request trigger

Pull request created in a GitHub action does not trigger workflows with pull_request trigger
See: #87

Needs to investigate - maybe related to https://github.com/orgs/community/discussions/65321 and/or https://github.com/orgs/community/discussions/57484

ci: apply docker build cache in Github action

To speed up the build process
Docs: https://docs.docker.com/build/cache/backends/gha/

Create a script to automate check latest version

• implement in python?
• compare with the current latest version

core: add azure to the toolkit

core: add azure to the toolkit

https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt

ci: update the dockerfile linter workflow naming

devops-toolkit/.github/workflows/dockerfile-lint.yml

Line 8 in 5a9f72b

build:

Change from build to lint to match with the workflow context

weekly-update: update all tools to latest version

weekly-update: update all tools to latest version

• python: 3.11.8
• ansible: 2.16.4
• terraform: 1.7.4
• kubectl: v1.29.2
• helm: 3.14.2
• awscli: 2.15.25
• azurecli: Version 2.58.0

ci: use verify toolkit version script to check the toolkit image

Update CI workflow to:

• use verify toolkit version script created in #28 to check the toolkit image

docker: Add kubectl to the toolkit

Add kubectl to the toolkit:

• https://kubernetes.io/docs/reference/kubectl/

release: Deploying image to DockerHub with GitHub Actions

Doc: https://docs.github.com/en/actions/deployment/about-deployments/deploying-with-github-actions

GitHub Actions offers features that let you control deployments. You can:

Trigger workflows with a variety of events.
Configure environments to set rules before a job can proceed and to limit access to secrets.
Use concurrency to control the number of deployments running at a time.

doc: add deployment badge to readme.md

Add:

to:
README.md

Add the sample code from `samples/run_sample.sh` to `samples/README.md`

Add sample code for kubectl and helm from

devops-toolkit/samples/run_sample.sh

Line 36 in 267b333

# Kubectl

to https://github.com/tungbq/devops-toolkit/blob/main/samples/README.md#kubectl (replace the TODO)

release: define approach to update tooling version

release: define approach to update tooling version

ci: create a weekly workflow to auto check new tool versions and create PR

ci: create a weekly workflow to auto check new tool versions and create PR
Using script from #80

first issue: initialize the project and structure, follow up task... 1️⃣

Initialize project

helm: add helm to the toolkit 🚢

add helm to the toolkit
https://helm.sh/docs/intro/install/

release: upload the toolkit image to public docker registry.

• DockerHub?
• quay.io?
• Publish for main first: combine with commit

ci: implement auto labeler for newly created issue

Some context could be automated:

• Issue with prefix doc/docker: should have labels doc and docker
• Issue with prefix docker/ansible: should have labels ansible and docker
• Issue with prefix ci: should have label ci
• ...and so on

Idea:

• https://github.com/github/issue-labeler?
• https://docs.github.com/en/actions/managing-issues-and-pull-requests/adding-labels-to-issues

ci: adding more label to support list

Add more labels to support list https://github.com/tungbq/devops-toolkit/blob/main/.github/workflows/auto-labeler.yml#L28 :

• enhancement
• sample
• usage

ci: Cancel previous runs in the PR when you push new commits (update the current)

To save the running resource, we could cancel the current run when there has new commit pushed to the pull requet.

Workflow to update:

• https://github.com/tungbq/devops-toolkit/blob/main/.github/workflows/dockerfile-lint.yml
• https://github.com/tungbq/devops-toolkit/blob/main/.github/workflows/docker-image-ci.yml

Basically this task could be completed by adding:

concurrency: 
  group: ${{ github.head_ref }}
  cancel-in-progress: true

Doc: https://stackoverflow.com/a/67939898

usage: add usage example for each tool

usage: add usage example for each tool

doc: documentation improvement

This story is opened to track the related document improvement PRs

doc: fix cloning instruction typo in README.md

In README.md:
Change from git clone https://github.com/your-username/devops-toolkit.git
to git clone https://github.com/tungbq/devops-toolkit.git

Apply terminal session recorder in to demo the usage

Using:

• https://asciinema.org/
• https://github.com/asciinema/asciinema?tab=readme-ov-file

ci: create CI workflow to verify the Dockerfile

Create CI workflow to verify the Dockerfile:

• Build the image when there has a push event to a Pull request
• Test image will be implemented in other Story

doc: create table to show supported tools with version information

doc: create table to show supported tools with version information

• toolname
• version

core: improve to reduce image size

core: improve to reduce image size
To provide more lightweight image for user

toolkit: define the base line toolkit tools 🗳️

Proposing tools:
(Open new issue for each)

• Ansible - via #12
• Terraform - via #21
• Python3 & pip - via #2
• Kubectl - via #23
• Helm - via #37
• AWS CLI - via #39
• Azure CLI - via #40

document: create a logo for this devops-toolkit

document: create a logo for this devops-toolkit
Idea: looks like a box??