brentley/ecsworkshop

THIS REPO HAS BEEN MOVED

7/19/21 - FOR ANY CHANGES, PLEASE HEAD TO THE FOLLOWING REPO.

ecsworkshop's Issues

Add a www redirect

figure out namespacing so many users can run in 1 account

Add CDK autoscale for NodeJS Backend API

Add autoscale with CDK for NodeJS Backend API

Watch it run has the wrong URL

For the 'watch it run' links in each of the microservice deployments points to https://console.aws.amazon.com/codepipeline/home?region=us-east-1#/view/mu-ecsdemo-frontend however, the pipelines that are created include the namespace which is mu + a random two character string.
I can see a few ways to fix this - either create the CodePipeline with the name of mu-escdemo-frontend or change the link to point to the CodePipeline landing page that shows a list of all pipelines.

deepdive: cloudformation

write: integrate with code inspector (ca?)

write: code change walkthrough

write: container scanning

Screenshot IAM role attachment

In: https://ecsworkshop.com/start_the_workshop/workspace/
Screenshot used to show how to attach IAM role it not up-to-date. It's using the old EC2 UI, need to update screenshot with the new UI.

Typo in AM role name for ecsworkshop

IAM role created as "eksworkshop-admin", where as its referenced as "ecsworkshop-admin" during verification using aws command.
https://ecsworkshop.com/start_the_workshop/workspace/
https://ecsworkshop.com/images/createrole.png

aws sts get-caller-identity --query Arn | grep ecsworkshop-admin -q && echo "IAM role valid" || echo "IAM role NOT valid"

write: docker multistage builds

Error in command in https://ecsworkshop.com/container_insights/setup/

the jq expression for clustername calculation return an error that it is not in quote

cluster_arn=$(aws ecs list-clusters | jq -r '.clusterArns[] | select(contains("container-demo"))')
clustername=$(aws ecs describe-clusters --clusters $cluster_arn | jq -r .clusters[].clusterName)

write: implement service autoscaling

Create a new backend service that will autoscale in and out by adding and then removing load.

write: use jenkins instead of codebuild

write: unit testing

write: daemon scheduling (1 per instance)

deepdive: docker native healthchecks

Could not find any VPCs matching error when running capacity providers install step

I've gone through the workshop so far in fargate mode, and when I get to the https://ecsworkshop.com/capacity_providers/fargate/#deploy step, when I run the command, I get:

[Error at /ecsworkshop-capacityproviders-fargate/ecsworkshop-capacityproviders-fargate] Could not find any VPCs matching {"account":"123x","region":"us-east-1","filter":{"tag:Name":"ecsworkshop-base/BaseVPC"},"returnAsymmetricSubnets":true}
Found errors

I seem to have two VPCs created for ECS, but none of them are tagged "ecsworkshop-base/BaseVPC" and its not clear to me which one should be.

write: add cluster autoscaling feature

Add cluster autoscaling feature for ec2 backed clusters

deepdive: deploy strategies

write: add capacity groups

Use capacity groups to deploy in two ways:

1. split fargate/fargate spot
2. split EC2 On Demand/EC2 Spot

DEPLOY ECS CLUSTER AUTO SCALING subnet creation fails

With CAPACITY PROVIDERS SECTION uncommented, cdk deploy fails with an error

The CIDR '10.0.0.160/27' conflicts with another subnet (Service: AmazonEC2; Status Code: 400; Error Code: InvalidSubnet.Conflict; Request ID: 7a1e1ab6-c1e2-4e0d-91cb-ed454e97bf77; Proxy: null)

Following instructions from page: https://ecsworkshop.com/capacity_providers/ec2/

deepdive: codepipeline

write: service discovery

Issue on task_id env variable

In:
https://ecsworkshop.com/microservices/frontend/#deploy-frontend-3
https://ecsworkshop.com/microservices/nodejs/#deploy-frontend-2
https://ecsworkshop.com/microservices/crystal/#deploy-crystal-service-2

Command:
task_id=$(ecs-cli compose --project-name ecsdemo-frontend service ps --cluster-config container-demo | awk -F \/ 'FNR == 2 {print $1}')
should be replaced by:
'task_id=$(ecs-cli compose --project-name ecsdemo-frontend service ps --cluster-config container-demo | awk -F \/ 'FNR == 2 {print $2}')

deepdive: codebuild

deepdive: mu

write: ecr custom domain

[BUG] Monitoring service logs: Make cloudwatch log group a CF output

Problem:

When reviewing service logs using awslogs, we dynamically grab the log group and set it as an environment variable. While this is good when using in a brand new AWS account, if you have done this workshop before and run through it again, there could be multiple log groups.

Solution:

Set the log group as a CF output, and when tailing the logs, use the CF output as the log_group variable.

Incorporate native HEALTHCHECK in dockerfiles

Add autoscale with CDK for Crystal Backend API

Add autoscale with CDK for Crystal Backend API

Cannot see SVG despite successful deployment using copilot

I've gone through the ecsworkshop using the copilot commands in Cloud9, but I can't see the SVGs which display the network traffic across the 9 different created instances. Instead, I only see 3 lines from the load balancer url displaying each service.

I feel like it could be an issue with the AZ suffix not populating correctly for the Rails frontend. I only see "AZ-" rather than "AZ-a" or "AZ-b", etc. like I see with the Node.js backend and Crystal backend lines.

Pipeline status shows failed - Github fork master/main conflict

During the pipeline steps, the forked repo has the legacy 'master' branch, but copilot pipeline.yml sets the default to main. This leads to the pipeline failing with codepipeline complaining that there is no branch 'main'

This can be worked around by editing ecsdemo-frontend/copilot/pipeline.yml and changing the branch to 'master' and then updating the pipeline with copilot pipeline update

I think the long term fix is to rename the master repo, but I'm leaving the note above as a workaround until then

figure out better icons for "main track" / "optional track" / "deep dive"

write: service mesh

Unable to delete mu created environments

$ mu env term acceptance
Loaded extension backend-service (version=1)
Terminating Services for environment 'acceptance' ...
Terminating Databases for environment 'acceptance' ...
Terminating ECS environment 'acceptance' ...
func1 ▶ ERROR ValidationError: Role arn:aws:iam::423786275406:role/mu-b4-cloudformation-common-us-west-2 is invalid or cannot be assumed
status code: 400, request id: 71be1686-3e87-11e8-a599-bb4d4f35654a

I get the same error when I try to delete the cfn stack through the cfn console

new chapter: managing workers and crons

write: bad code deploy

IAM error when creating ALBs

here is an example error:

mu-55-loadbalancer-acceptance: Elb (AWS::ElasticLoadBalancingV2::LoadBalancer) CREATE_FAILED API: elasticloadbalancingv2:CreateLoadBalancer User: arn:aws:sts::497382815317:assumed-role/mu-55-cloudformation-common-us-east-1/AWSCloudFormation is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::497382815317:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing

After researching and talking with @cplee we figured out that ALB now requires a service linked role also, and Mu only creates a missing service linked role for ECS:

https://github.com/stelligent/mu/blob/develop/templates/assets/common-iam.yml#L337

      - PolicyName: service-linked-roles
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Action:
            - iam:CreateServiceLinkedRole
            - iam:PutRolePolicy
            Resource: arn:aws:iam::*:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS*

Casey is cutting a new fix release to add ALB service linked role support. As a work-around you can also add this one-liner:

+++ backend-service/common-iam.yml
@@ -15,5 +15,6 @@ Resources:
               - route53:DeleteHostedZone
               - route53:UpdateHostedZoneComment
               - route53:ListQueryLoggingConfigs
+              - iam:CreateServiceLinkedRole
               Resource: '*'
               Effect: Allow

explore: ecs cli

Feature: Add copilot as mechanism to deploy microservices chapter

deepdive: immutable infrastructure

figure out IAM role setting to allow route53 deletion

IAM role name vs grep command mismatch

On the below page the IAM role is mismatched from the grep command to validate IAM Role.

Instructions say to call the IAM Role eksworkshop-admin
then, grep ecsworkshop-admin

this logic will always return, IAM Role NOT Valid

https://ecsworkshop.com/start_the_workshop/workspace/

write: use spot instances

write: dockerfile optimization

write: autoscaling walk-through

Siege binary not installed in ecs-cli mode

Hi,

We are using siege on the container insight module to stress the platform. This binary is installed in the "install and configure tools" for CDK but not for ecscli. Need to update the workshop accordingly.

Br,

need env var check

At https://ecsworkshop.com/nodejs/backend/#deploy-frontend-1 I ran into an issue while creating the service using the ecs-cli fargate mode path.

I had closed the terminal I used to setup Frontend, so I lost the vars I had set previously on the:

https://ecsworkshop.com/frontend/frontend/#deploy-frontend-1 > Set environment variables from our build section.

I ran the envsubst, and the yaml had empty fields. and as you would expect, the subsequent ecs-cli command failed.

So maybe some sort of check that the vars exist, saving them to the .bash_profile or just forcing the user to re-set the vars again would be good here.

deepdive: cloudwatch logs