Counter Online is just my learning project to experiment some technologies. Here you will find:
- Backend app written in Go language (using GO kit)
- Terraform to deploy infrastructure components on AWS
- Kubernetes to manage app containers
- GitHub Actions to CI/CD
- AWS as a cloud provider
Imagine following scenario: sellers wants to keep their product authenticity and users want to check it to get quality goods. You can provide together their product a unique identifier to be validated online. When user validate it for the first time, the counter is one. This means that no one else validated the code before and it can be considered authentic. If another manufacturer makes a copy and sell it as original, they need to give the validation code to customers. If customers tries to validate, the code can be already used or invalid.
| Feature | Description | State | Comment |
|---|---|---|---|
| Documentation | Document the process and architecture | In progress | - |
| Counter App Service | The counter application | Ready | - |
| Counter App Service | Use GO kit to create services | Ready | - |
| Counter App Service Build | The counter continuous integration (CI) | Ready | - |
| Counter App Service Deployment | The counter application deployment (CD) | Ready | - |
| AWS Infrastructure (Terraform) | Terraform files to create infrastructure | Ready | - |
| AWS Infrastructure (Terraform) Deployment | Terraform infrastructure pipeline (CD) | Ready | - |
| AWS Password Manager CSI Driver | CSI driver deployment | Ready | - |
| AWS Password Manager CSI Provider | CSI provider deployment | Ready | - |
| AWS Kubernetes Cluster Autoscaler | Automatically adjusts the number of nodes when needed | Ready | - |
| AWS Kubernetes Cluster Autoscaler Deployment | AWS Kubernetes Cluster Autoscaler pipeline | Ready | - |
| Ingress Nginx | Kubernetes ingress deployment | Ready | - |
| DNS Management | Implement DNS Management | Not started | - |
| CertManager | Implement Cert Manager | Not started | - |
This project is organized as follows:
| Folder | Description |
|---|---|
| .github/workflows | GitHub action workflow pipelines |
| app | App source code in GO Language |
| terraform | Terraform HCL files to deploy infrastructure |
| deployments | Kubernetes Deployment folders to create basic infrastructure software elements like ingress |
| scripts | Scripts used help to prepare terraform infrastructure, creating aws user, policies, s3, etc |
IMPORTANT: In a production environment, you must use different repositories to each component (application, kubernetes deployments, terraform...)
This is the use case for this project. Here we have the seller creating UUID v5 codes to be validated by the customer (maybe using qr code), as you saw in the beginning of this document.
Above you will find the application architecture. I used AWS provider and terraform to deploy infrastructure components.
I recommend you fork this repository and change it to use your own parameters. The next two sections, I'll show you what you gonna need to do.
These are GitHub Action variables needed by automation. Please, create these key-values inside your github repository:
| Variable Name | Description |
|---|---|
| AWS_SECRET_ACCESS_KEY | Store your AWS provider access key |
| AWS_ACCESS_KEY_ID | Store your AWS provider secret key |
| AWS_DEFAULT_REGION | Your AWS provider Region |
| TERRAFORM_GITHUB_TOKEN | Create a GitHub PAT (default token doesn't work] |
This project is using Terraform to maintain the infrastructure. You need to configure credentials, permissions and storage to maintain state. You can use the scripts/startup-terraform-backend-state.sh script to help you.
Some files have AWS account ID hardcoded. You can replace these values with your AWS account id using find ./ -type f -exec sed -i 's/043934856969/100000000001/g' {} \; where 100000000001 is your aws acccount id.
Open file go-counter-online/deployments/kustomize/common/base/service-account.yaml and change:
eks.amazonaws.com/role-arn: arn:aws:iam::043934856969:role/AmazonSCPRoleto
eks.amazonaws.com/role-arn: arn:aws:iam::<your aws account id>:role/AmazonSCPRoleOpen file go-counter-online/deployments/specs/aws-cluster-autoscaler-service/cluster-autoscaler-autodiscover.yaml and change:
eks.amazonaws.com/role-arn": "arn:aws:iam::043934856969:role/AmazonEKSClusterAutoscalerRoleto
eks.amazonaws.com/role-arn": "arn:aws:iam::<your aws account id>:role/AmazonEKSClusterAutoscalerRoleOpen file go-counter-online/deployments/specs/aws-load-balancer-controler-service/aws-load-balancer-controller-service-account.yaml and change:
eks.amazonaws.com/role-arn: arn:aws:iam::043934856969:role/AmazonEKSLoadBalancerControllerRoleto
eks.amazonaws.com/role-arn: arn:aws:iam::<your aws account id>:role/AmazonEKSLoadBalancerControllerRoleOpen file go-counter-online/deployments/specs/aws-load-balancer-controler-service/aws-load-balancer-controller-service-account.yaml and change:
eks.amazonaws.com/role-arn: arn:aws:iam::043934856969:role/AmazonEKSLoadBalancerControllerRoleto
eks.amazonaws.com/role-arn: arn:aws:iam::<your aws account id>:role/AmazonEKSLoadBalancerControllerRoleOpen file go-counter-online/terraform/iam.tf and change:
"Resource": ["arn:aws:secretsmanager:us-east-2:043934856969:secret:*"]to
"Resource": ["arn:aws:secretsmanager:us-east-2:<your aws account id>:secret:*"]Update kubeconfig to access EKS Cluster:
aws eks update-kubeconfig --name prd-go-counter-online-eksDocker build to test local:
docker build -t go-counter-online -f app/Dockerfile appAdd users and roles to access the cluster:
kubectl -n kube-system edit configmap aws-auth
# Add user test like this:
# 043934856969 is your account code
data:
(... other stuffs)
mapUsers: |
- userarn: arn:aws:iam::043934856969:user/test
username: test
groups:
- system:mastersCommand line to run Counter Online local
# memory database
go run . -port=8080 -datastore=memory
# or postgresql database
go run . -port=8080 -datastore=postgresql -extra-params='host=localhost dbname=go_counter_online user=go_counter_online password=go_counter_online_password sslmode=disable' -hide-extra-params=trueCurl commands to test Counter Online API services
# Create counter with UUID v5 AAAAAAAA-AAAA-5AAA-AAAA-AAAAAAAAAAAA using name test:
curl -v -XPOST localhost:8080/count/AAAAAAAA-AAAA-5AAA-AAAA-AAAAAAAAAAAA/test
# Consume counter with UUID v5 AAAAAAAA-AAAA-5AAA-AAAA-AAAAAAAAAAAA:
curl -v -XGET localhost:8080/count/AAAAAAAA-AAAA-5AAA-AAAA-AAAAAAAAAAAADeploy PSQL to test database
Use psql utility to connect to Postgres database:
#docker
docker run -it --rm postgres psql -h <dbhost> -U go_counter_online -d go_counter_online#kubernetes
kubectl run psql --rm -it --image=postgres -- psql -h <dbhost> -U go_counter_online -d go_counter_onlineAutoscaling is a function that automatically scales your resources up or down to meet changing demands. This is a major Kubernetes function that would otherwise require extensive human resources to perform manually.
More info: https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent