veezor/maestro

AWS CodeBuild Cloud Native Buildpacks Docker Image for Pack CLI

maestro's Issues

Build fail with cache enabled

#project.toml
[project]
id = "com.meu.site"
name = "site"
version = "1.0"

[build]
builder = "heroku/buildpacks:18"

[[build.buildpacks]]
uri = "heroku/nodejs"

[[build.buildpacks]]
uri = "heroku/ruby"

[[build.buildpacks]]
uri = "heroku/procfile"

#Procfile
web:     bundle exec unicorn -p $PORT -c ./config/unicorn.rb
jobs:    bundle exec sidekiq -C config/sidekiq.yml 
console: bundle exec unicorn -p $PORT -c ./config/unicorn.rb

Improve cache usage

improve the verification of cache variables, don't create and clear cache only if the Codebuild variables value is true.

• MAESTRO_NO_CACHE
• MAESTRO_CLEAR_CACHE

Build pull policy should follow cache settings

Currently pack build is hard coded to a --pull-policy if-not-present flag and that would eventually prevent downloading a newer buildpacksio/lyfecycle image when available. While this behavior is useful to expedite builds, if there is a need to update it we currently must remove the image from ECR which is not ideal. Adjusting this flag to follow MAESTRO_NO_CACHE environment variable would allow end users to benefit from it.

Check CS_SERVICE_TASK_PROCESSES syntax

Check the syntax of the ECS_SERVICE_TASK_PROCESSES variable that is used in the codebuild. If possible, issue an alert to warn the user that it is wrong and it will perform the default configuration for it.

Multiple web services for ECS

Implement the feature to create more than one web service on ECS cluster for a workload with it's own target group and ALB (It can be a shared ALB too).

Script error, too many arguments

This error is showed on every build, this not break the build.

Console task related improvements

Revoke/disable access through AWS CLI to other tasks there are not "Console"

Create a single task for "Console" (created with a command and auto-killed when user exits)

Set maestro environment variable's defaults

We need a comprehensive set of defaults for each environment variable that makes sense. For instance those that have true or false as options and maybe others that could be set to a default in case they are not explicitly set. @HaseoBoss could you please come up with a draft PR of the docs with those defaults considering the way which the majority of our workloads are currently set?

Force use of role instead of AWS credentials in secret manager

When AWS credentials exist in secrets manager, it override the AWS Role permissions and try to use AWS credentials, and this broke the build.

Enable/Disable/Remove Shceduledtasks

Scheduledtask are set through config file added to repository using the following format:

name rate(## minutes)/(cron expression) command

To enable or disable a task it should accept the following format:

name enable/disable rate(## minutes)/(cron expression) command

If a task is removed from file this task must be removed from eventbridge/ecs.

How to implement a build pre-image feature?

My question

It's possible implement a feature, utilizing a workflow file, to build a "pre-image" when a pull request is open for test the made changes?

Create the possibility to leave the root filesystem of the container as read-only

Allow when the user so needs to make the container root filesystem read-only. Leave the read and write option as default.

Parameter can be found here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html#cfn-ecs-taskdefinition-containerdefinition-readonlyrootfilesystem

Set Task Size in ScheduledTasks

In some specific cases, use {4096:8192} size isn't a good choice. It's possible set a different size to an individual a task?
For example, change this parameter on config/run_tasks.conf:

ss_my_simple_task{512:1024} rate(20 minutes) bundle exec rake simple_task

ss_my_heavy_task{4096:8192} rate(20 minutes) bundle exec rake heay_task

Add to docs information about forbidden envs

Add to documentation information about variables that can be used by the workload like AWS_ACCESS_KEY and SECRET_ACCESS_KEY.

Record and monitor new relic deployments

Creation of Auto scaling for memory metric not working

Even if I set the auto scaling for memory metric in the envs as described in readme it's not created in the service as expected.

Allow modifying circuit breaker behavior

Allow modifying circuit breaker behavior during build, so that the default behavior (which currently is enable=true,rollback=true) can be changed to other possible settings.

Automatizar ativação do Circuit Breaker

Auto scaling policies parameters overwritten

Running a new build parameters modified directly on ECS service for auto scaling policies are being overwritten with default values.

E.g.: Scale-out/in coodown period is always set to 300.

Build fails if there is an AS policy manually created

If there is an AS policy manually created build fails with the following error:

"An error occurred (ValidationException) when calling the PutScalingPolicy operation: Only one TargetTrackingScaling policy for a given metric specification is allowed."

Is it the expected behavior?

Use VPC endpoint on this resources, ECS, ECR and Cloudwatch

Implement and use VPC endpoint on this services, ECR, ECS and cloudwatch to reduce cost with traffic of data.

Here some documentation:
https://docs.aws.amazon.com/pt_br/AmazonECR/latest/userguide/vpc-endpoints.html

Override env variable set on secrets manager

In some cases it is required that one or more env variable set on secrets manager be override for a service task definition. This can be really useful for web services sharding strategy to specify specific parameters for each sharding (E.g., separate shardings as app in New Relic or other monitoring system).