Following actions will be incorporated in Terraform HCL language
- Create managed AWS Elasticsearch and kibana services up and running in the US region
- Define and Hot and Ultrawarm instances and policies (Hot is EBS and Ultrawarm is in S3)
- Configure ES Domain and required security policies and data encryption
- Elasticsearch and Kibana should be accessible from the browser
- NOTE: provider.tf has not been added in github for security reasons.
19/08/2021 Update: Commited tf files which created AWS managed ES and kibana running in US region with Ultrawarm defined.
NOTE: Problem 2 POC is done here https://github.com/jchowdhary/tf-lambda
Problem 2:
- Create a S3 bucket in the US region (bucket1) with retention as 30 days.
- Trigger an S3 event when object (text file) is placed in the S3 (bucket1)
- Trigger a lambda function to move the file to another bucket (bucket2) when S3 event is received
- Create a glue Job to compress the bucket2's object with gz