tungbq/devops-basics

🚀 Practical and document place for DevOps toolchain

DevOps Basics

🚀 Practical and document place for DevOps toolchain 💝

With comprehensive documentation, basic hands-on scripts, advanced examples, and additional resources for a variety of DevOps tools

Introduction 👋

You are new to DevOps or want to learn some DevOps tools, or you are already a DevOps engineer, and you are looking for DevOps documents and a place to practice DevOps tools? This repository will assist you in enhancing your DevOps skills and serve as a bookmark for documents related to DevOps. Key features include:

• Wide range of topics: Explore 20+ important DevOps subjects like Docker, Kubernetes, Terraform, Ansible, Cloud services, System Architecture, Monitoring, and more.
• Useful resources: Each topic comes with an overview, official documentation links, cheatsheets, and extra resources to help you learn more.
• Hands-On practice: Get basic examples for each topic so you can try out what you learn.
• Advanced examples: Once you've grasped the basic concepts, exploring advanced examples will further elevate your skills.

Getting started 🚀

What is DevOps?

DevOps combines development (Dev) and operations (Ops) to increase the efficiency, speed, and security of software development and delivery compared to traditional processes. A more nimble software development lifecycle results in a competitive advantage for businesses and their customers (source: GitLab)

Getting started

• ➡️ getting-started

DevOps toolchain

• ➡️ DevOps toolchain

DevOps Roadmap

• ➡️ roadmap.sh/devops

DevOps topics 🔥

We cover a wide range of DevOps topics in our content library, explore them under topics. You can also refer to roadmap.sh/devops to see where each topic fits in the toolchain

	Item	Content	Official Docs	Hands-on
	Ansible	ansible	📖 docs.ansible.com	✔️ ansible-helloworld.sh
	Docker	docker	📖 docs.docker.com	✔️ docker-helloworld.sh
	Kubernetes (k8s)	k8s	📖 kubernetes.io/docs	✔️ k8s-helloworld.sh
	Istio	istio	📖 istio.io/latest/docs	✔️ getting-started
	Aws	aws	📖 docs.aws.amazon.com	✔️ EC2_GetStarted
	Helm	helm	📖 helm.sh/docs	✔️ helm-helloworld.sh
	Jenkins	jenkins	📖 www.jenkins.io/doc	✔️ Jenkins-Hello-World.md
	Terraform	terraform	📖 terraform/docs	✔️ terraform-helloworld.sh
	Shell	shell	📖 devdocs.io/bash	✔️ basic.sh
	Git	git	📖 git-scm.com/doc	✔️ git-helloworld.sh
	Elk	elk	📖 www.elastic.co/guide	✔️ elk/basic/helloworld
	ArgoCD	argocd	📖 argo-cd.readthedocs.io	✔️ argocd/basic
	Github-Action	github-action	📖 docs.github.com/actions	✔️ Create First Workflow
	Gitlab CI	gitlab-ci	📖 docs.gitlab.com/ee/ci	✔️ Create First Pipeline
	Groovy	groovy	📖 groovy-lang.org	✔️ groovy/basic
	Prometheus	prometheus	📖 prometheus.io/docs	✔️ helloworld.sh
	Python	python	📖 www.python.org/doc	✔️ python/basic
	Openstack	openstack	📖 docs.openstack.org	✔️ openstack/basic
	Azure-DevOps	azuredevops	📖 learn.microsoft.com	✔️ azuredevops/basic
	Coding	coding	📖 coding	✔️ practice.md
📖	Architecture	architecture	📖 architecture/README.md	✔️ Architecture
	Microservices	microservices	📖 aws/microservices	✔️ basic demo
	Nginx	nginx	📖 nginx.org/en/docs	⏩ nginx (in-progress)
	Packer	coming-soon	📖 www.packer.io	⏩ coming-soon
	HashiCorp Vault	coming-soon	📖 hashicorp.com/vault	⏩ coming-soon

• And more upcoming topics...⏩ you can star/follow this repository to get more up-to-dated content ⭐
• For additional resources to practice these tools, visit: devops-project

Extra content (from other repos) ↗️

Item	Link
The DevOps Hub	☁️ TheDevOpsHub
devops-project	🏗️ devops-project
90DaysOfDevOps	📆 90DaysOfDevOps
python	🐍 python-examples
aws-lab-with-terraform	☁️ aws-lab-with-terraform
AWS-LearningResource	🧰 AWS-LearningResource
bookmarks	🔖 bookmarks
devops-exercises	📚 devops-exercises
devops-resources	🔗 devops-resources
howtheydevops	❔ howtheydevops
devops-dockerfiles	🏗️ devops-dockerfiles

Contributing

• See: CONTRIBUTING.md
• If you find this repository helpful, kindly consider showing your appreciation by giving it a star ⭐ Thanks! 💖
• Looking for the issue to work on? Check the list of our open issues good first issue
• Feel free to open a new issue if you want to request more content about DevOps

Contributors

Thank you all for your contributions! 💖

devops-basics's Issues

Automatically generate README.md based on repo structure using Github Action

Automatically generate README.md based on repo structure using Github Action
Idea:

1. Loop and detect folder
2. Create relative link on the README.md file
3. Run on Nightly basis

[shell] Add a simple example for grep command in bash script

In https://github.com/tungbq/devops-basic/blob/4ad55a6ab16920bcda430d1456013221d2f20e9d/shell/basic.sh#LL52C1-L52C1
Let's add a simple grep example to demonstrate how it works. You are free to choose the data to be grepped (may be from a string, or an output from another command)

Reference: https://www.digitalocean.com/community/tutorials/grep-command-in-linux-unix

docs: add getting-started folder and provide the devops roadmap

Tasks:

• Create new folder named: getting-started
• Create new README.md under getting-started
• Add below content:

  # DevOps Roadmap
  - https://roadmap.sh/devops
  

[ci] Add shell check workflow for shell folder

It's good if we could develop a workflow for checking shell script under shell folder https://github.com/tungbq/devops-basic/tree/main/shell
It could be run on every PRs that needs to merge onto main branch and contains change in the shell folder

Reference: We could investigate this checker https://github.com/marketplace/actions/shellcheck

[shell] Adding switch case example into basic.sh

Tasks:

• Investigate the switch case in shell script
• Add switch case example into basic.sh

Script location: https://github.com/tungbq/devops-basic/blob/c1ccccbc1486db19b236f365513398a2a3bb5067/shell/basic.sh#L31

Shell scripting content - basic02

[investigate] Explore JFROG Artifactory and consider adding to the repo

See: https://jfrog.com/artifactory/

docs: Remove extra colon in README.md

Remove some extra colon in https://github.com/tungbq/devops-basic/blob/main/README.md

Adding git folder

Shell scripting content

Chap04-Part02-Working on ASG

Working on Auto Scaling Group:
TASKS:

• Self investigating ASG
• Code ASG using terraform

doc: Add `git` to README.md content table

Add git to README.md content table:
Content table location: https://github.com/tungbq/devops-basic/blob/main/README.md#content

[jenkins] [ci] Create jenkins folder to store knowledge and example related to Jenkins

Task:

• Under the root dir of https://github.com/tungbq/devops-basic
• Create a new folder name jenkins with a README.md file
• Add a brief description to that README.md file (just a place holder, maybe just add To be updated... into it, we will improve it later)

Add python content - basic01

[shell] create examples folder under shell

[github action] Add a github workflow to run shell script under `shell` folder

Create new workflow under .github/workflows:

• Name it as run-shell.yml (or you can propose a better name)
• Checkout the code of this repository
• Execute the script https://github.com/tungbq/devops-basic/blob/main/shell/basic.sh
• The workflow needs to be run on every PR

[All Contributors bot] Add Contributors section to README.md

Append these lines into our current README.md file:

## Contributors

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->

<!-- ALL-CONTRIBUTORS-LIST:END -->

[Terraform] Improve documents and folder structure

Checking for typo, and better organize terraform part.

[Hands-on] Deploying Containers on Amazon’s ECS using Fargate and Terraform

Reference: https://medium.com/@bradford_hamilton/deploying-containers-on-amazons-ecs-using-fargate-and-terraform-part-2-2e6f6a3a957f

[shell] Write an example to list all current txt file in given foler and save result to another txt

Under example folder of shell (https://github.com/tungbq/devops-basic/tree/main/shell/examples), create new bash script:

• to list all current txt file
• then save the result to another txt file
  You are free to choose the filename

[shell] Add pretty icon to switch case example

In the switch case example: https://github.com/tungbq/devops-basic/blob/4ad55a6ab16920bcda430d1456013221d2f20e9d/shell/basic.sh#L36
Make the example more colorful by adding icon to the fruit:

• For example echo "You entered apple." -> to be changed to echo "You entered 🍏."

docs: add getting-started to the content table in README.md

Add getting-started folder link to https://github.com/tungbq/devops-basic#content:

• The format should be the same as other current folder
• The getting-started folder link should be on the top of that table

[All Contributors bot] Implementing All Contributors bot to the repo

Reference:

• https://allcontributors.org
• https://allcontributors.org/docs/en/bot/usage

Sub-task:

• #49
• #50

Usage:

• https://allcontributors.org/docs/en/bot/usage#all-contributors-add
• @all-contributors please add @<username> for <contributions>

Add README file with short introduction for `helm`

In https://github.com/tungbq/devops-basic/tree/main/helm, make some updates:

• Create new README.md file
• Provide a short description to describe what Helm is
• Add the getting started link to README.md: https://helm.sh/docs/intro/quickstart/
• (Optional) It would be nice if we could have some other references/examples that make people getting started with helm in the README file

Reference: https://helm.sh/

feat: Add Github Action content

warning: LF will be replaced by CRLF

Need to fix it:
warning: LF will be replaced by CRLF

Reorg & Clean code for terraform topic

docs: Typo in terraform README

Fix typo in: https://raw.githubusercontent.com/tungbq/devops-basic/main/terraform/README.md
It should be # Init terraform section instead of # Init terrafrom section

ELK content - basic01

feat: ci - add a workflow to check broken link in the repo

add a workflow to check broken link in the repo:

• might be run weekly

[CI] Upgrade deprecated github action in .github/workflows/main.yaml

See: https://github.com/tungbq/devops-basic/blob/main/.github/workflows/main.yaml

list_folders_job
Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

Tasks:

• Read https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
• Upgrade uses: actions/checkout@v2 to higher version

Terraform: AWS Three-Tier Architecture Design

Verifying PR template

Verifying PR template from #36

Docker build error while building python based images for running ./docker practice script

When building python based images for running ./docker practice script we got this err:
How to reproduce: cd devops-basic; ./practice.sh

 => ERROR [3/5] RUN pip install -r requirements.txt                                                                                                                                                181.7s 
------
 > [3/5] RUN pip install -r requirements.txt:
#8 4.843 Collecting numpy==1.21.2
#8 5.170   Downloading numpy-1.21.2.zip (10.3 MB)
#8 128.0      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.3/10.3 MB 79.3 kB/s eta 0:00:00
#8 129.5   Installing build dependencies: started
#8 161.0   Installing build dependencies: finished with status 'done'
#8 161.0   Getting requirements to build wheel: started
#8 162.0   Getting requirements to build wheel: finished with status 'done'
#8 162.0   Preparing metadata (pyproject.toml): started
#8 162.8   Preparing metadata (pyproject.toml): finished with status 'done'
#8 162.8 Building wheels for collected packages: numpy
#8 162.8   Building wheel for numpy (pyproject.toml): started
#8 181.3   Building wheel for numpy (pyproject.toml): finished with status 'error'
#8 181.3   error: subprocess-exited-with-error

[common] Add contributing file

Create CONTRIBUTING.md file for this repo

Create new `CODEOWNERS` file for devops-basic repository

See: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners#example-of-a-codeowners-file
Task:

• Create new CODEOWNERS file
• Add * @tungbq to the newly created file
• Open the PR and waiting for merge
• Done! <3

[Chap03] Error while creating EC2 Instance - InvalidParameterValue: Value ()

Encountering this error:
aws_instance.example_chap03: Creating...
╷
│ Error: creating EC2 Instance: InvalidParameterValue: Value () for parameter groupId is invalid. The value cannot be empty
│ status code: 400, request id: e54cc3ac-f260-40d9-a8d0-b64e0560b08c
│
│ with aws_instance.example_chap03,
│ on main.tf line 38, in resource "aws_instance" "example_chap03":
│ 38: resource "aws_instance" "example_chap03" {
│
╵
How to reproduce:

1. Checkout to revision: 2760ea5ec2ee1ebc545ff03db2192e850270d25d
2. Navigate to chap03: cd devops-basic/terraform/terraform-up-and-running/chap03-terraform-state/stage/service/webserver-cluster
3. Run terraform apply

Create pull request template

[aws] Adding aws repo as submodules

k8s: deploy jenkins using helm

k8s: deploy jenkins using helm
store the persistence data on the VM

[jenkins] [ci] Create a simple pipeline to run stage with basic shell script

Create new README.md file for ansible

Tasks:

• Create new README.md file under https://github.com/tungbq/devops-basic/tree/main/ansible
• Then add following content:

Install Ansible
- Follow this guide: https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible

[All Contributors bot] Add devops-basic repo to bot configuration

Follow: https://allcontributors.org/docs/en/bot/installation#2-configure-your-repositories

feat: Create CODEOWNERS file

Create new CODEOWNERS file under root path of this repo
Then add below line into the new file:
* @tungbq
Find more info at: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners

[task] Resolve the conflict generated by all contributors bot in PR#55

Resolve the conflict generated by all contributors bot in: https://github.com/tungbq/devops-basic/pull/55/conflicts

[jenkins] [ci] Using docker as environment to work with Jenkins

Create document regarding this

ci: implement greetings workflow

Greetings WF: https://github.com/actions/first-interaction

Add contributing section to readme

[shell] Keep the shell folder clean

For now, under https://github.com/tungbq/devops-basic/tree/main/shell we have some txt files using as examples, to keep the shell folder clean, we can:

• Create a new data folder under shell
• Move all *.txt files to data folder (for example grep_example.txt)
• Update related scripts that use these txt files to use the new location shell/data/*.txt
• Ensure no breakage in all current *.sh scripts of shell folder

Work on chap04