- Greater level of control of the configuration of Apache Airflow
- Need to run workflows that use more resources than AWS Managed services
- Total cost of Ownership
- Terraform
- Helm
- AWS CLI
- Kubectl
- AWS account with sufficient privileges
The resources created include
- EKS Cluster Control plane with public endpoint
- One managed node group - Node group made up of 2 instances spanning 2azs
resource "kubernetes_service_account" "eksadmin" {
metadata {
name = "eks-admin"
namespace = "kube-system"
}
}
In setting up Apache Airflow on Amazon EKS, the "eks-admin" service account is often granted necessary RBAC (Role-Based Access Control) permissions to perform actions within the Kubernetes cluster.
Apache Airflow often requires specific permissions to create and manage Kubernetes pods, services, and other resources. By creating a dedicated service account and configuring appropriate RBAC rules, you can ensure that Airflow has the necessary permissions to interact with the Kubernetes cluster.
resource "kubernetes_cluster_role_binding" "eksadmin" {
metadata {
name = "eks-admin"
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "ClusterRole"
name = "cluster-admin"
}
subject {
kind = "ServiceAccount"
name = "eks-admin"
namespace = "kube-system"
}
}
Putting it all together, this kubernetes_cluster_role_binding
resource is binding the "eks-admin" ServiceAccount in the "kube-system" namespace to the "cluster-admin" ClusterRole. This essentially gives the "eks-admin" ServiceAccount broad administrative permissions over the entire Kubernetes cluster.