aws/copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.

AWS Copilot CLI

Build, Release and Operate Containerized Applications on AWS.

• Documentation: https://aws.github.io/copilot-cli/

The AWS Copilot CLI is a tool for developers to build, release and operate production-ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.

Use Copilot to:

• Deploy production-ready, scalable services on AWS from a Dockerfile in one command.
• Add databases or inject secrets to your services.
• Grow from one microservice to a collection of related microservices in an application.
• Set up test and production environments, across regions and accounts.
• Set up CI/CD pipelines to release your services to your environments.
• Monitor and debug your services from your terminal.

Installation

To install with homebrew:

$ brew install aws/tap/copilot-cli

To install manually, we're distributing binaries from our GitHub releases:

Instructions for installing Copilot for your platform

Platform	Command to install
macOS	curl -Lo copilot https://github.com/aws/copilot-cli/releases/latest/download/copilot-darwin && chmod +x copilot && sudo mv copilot /usr/local/bin/copilot && copilot --help
Linux x86 (64-bit)	curl -Lo copilot https://github.com/aws/copilot-cli/releases/latest/download/copilot-linux && chmod +x copilot && sudo mv copilot /usr/local/bin/copilot && copilot --help
Linux (ARM)	curl -Lo copilot https://github.com/aws/copilot-cli/releases/latest/download/copilot-linux-arm64 && chmod +x copilot && sudo mv copilot /usr/local/bin/copilot && copilot --help
Windows	Invoke-WebRequest -OutFile 'C:\Program Files\copilot.exe' https://github.com/aws/copilot-cli/releases/latest/download/copilot-windows.exe

Getting started

Make sure you have the AWS command line tool installed and have already run aws configure before you start.

To get a sample app up and running in one command, run the following:

$ git clone [email protected]:aws-samples/aws-copilot-sample-service.git demo-app
$ cd demo-app
$ copilot init --app demo                \
  --name api                             \
  --type 'Load Balanced Web Service'     \
  --dockerfile './Dockerfile'            \
  --deploy

This will create a VPC, Application Load Balancer, an Amazon ECS Service with the sample app running on AWS Fargate. This process will take around 8 minutes to complete - at which point you'll get a URL for your sample app running! 🚀

Learning more

Want to learn more about what's happening? Check out our documentation https://aws.github.io/copilot-cli/ for a getting started guide, learning about Copilot concepts, and a breakdown of our commands.

Feedback

Have any feedback at all? 🙏 Drop us an issue or join us on gitter.

We're happy to hear feedback or answer questions, so reach out, anytime!

Security disclosures

If you think you’ve found a potential security issue, please do not post it in the Issues. Instead, please follow the instructions here or email AWS security directly at [email protected].

License

This library is licensed under the Apache 2.0 License.

copilot-cli's Issues

Add Coverage Check to go test

We should fail our tests if they are below a certain coverage:

https://stackoverflow.com/questions/50120427/fail-unit-tests-if-coverage-is-below-certain-percentage

I think 80% is a good base (we're currently at 82~).

Emoji and Spinner glyphs don't work on Windows

Good news is, colors and the prompt work - but the emojis and spinner don't work as well.

Fix failed TestLoadBalancedFargateManifest_Marshal

2019-09-20T01:05:29.6011129Z === RUN   TestLoadBalancedFargateManifest_Marshal
2019-09-20T01:05:29.6011391Z --- FAIL: TestLoadBalancedFargateManifest_Marshal (0.01s)
2019-09-20T01:05:29.6154674Z     lb_fargate_manifest_test.go:84: 
2019-09-20T01:05:29.6155099Z         	Error Trace:	lb_fargate_manifest_test.go:84
2019-09-20T01:05:29.6155336Z         	Error:      	Not equal: 
2019-09-20T01:05:29.6156708Z         	            	expected: "# Your application name will be used in naming your resources\n# like log groups, services, etc.\nname: SweetApp\n# The \"Type\" of the application you're running. For a list of all types that we support see\n# https://github.com/aws/PRIVATE-amazon-ecs-archer/app/template/manifest/\ntype: Load Balanced Web App\n\n# The port exposed through your container. We need to know\n# this so that we can route traffic to it.\ncontainerPort: 80\n\n# Size of CPU\ncpu: 256\n\n# Size of memory\nmemory: 512\n\n# Logging is enabled by default. We'll create a loggroup that is\n# the SweetApp/Stage\nlogging: true\n\n# Determines whether the application will have a public IP or not.\npublic: true\n\n# You can also pass in environment variables as key/value pairs\n#environment-variables:\n#  dog: 'Clyde'\n#  cute: 'hekya'\n#\n# Additional Sidecar apps that can run along side your main application\n#sidecars:\n#  fluentbit:\n#    containerPort: 80\n#    image: 'amazon/aws-for-fluent-bit:1.2.0'\n#    memory: 512\n\n# This section defines each of the release stages\n# and their specific configuration for your app.\nstages:\n  -\n    # The \"environment\" (cluster/vpc/lb) to contain this service.\n    env: test\n    # The number of tasks that we want, at minimum.\n    desiredCount: 1\n    # Any secrets via ARNs\n    #secrets:\n    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd\n  -\n    # The \"environment\" (cluster/vpc/lb) to contain this service.\n    env: prod\n    # The number of tasks that we want, at minimum.\n    desiredCount: 3\n    # Any secrets via ARNs\n    #secrets:\n    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd\n"
2019-09-20T01:05:29.6159090Z         	            	actual  : "# Your application name will be used in naming your resources\r\n# like log groups, services, etc.\r\nname: SweetApp\r\n# The \"Type\" of the application you're running. For a list of all types that we support see\r\n# https://github.com/aws/PRIVATE-amazon-ecs-archer/app/template/manifest/\r\ntype: Load Balanced Web App\r\n\r\n# The port exposed through your container. We need to know\r\n# this so that we can route traffic to it.\r\ncontainerPort: 80\r\n\r\n# Size of CPU\r\ncpu: 256\r\n\r\n# Size of memory\r\nmemory: 512\r\n\r\n# Logging is enabled by default. We'll create a loggroup that is\r\n# the SweetApp/Stage\r\nlogging: true\r\n\r\n# Determines whether the application will have a public IP or not.\r\npublic: true\r\n\r\n# You can also pass in environment variables as key/value pairs\r\n#environment-variables:\r\n#  dog: 'Clyde'\r\n#  cute: 'hekya'\r\n#\r\n# Additional Sidecar apps that can run along side your main application\r\n#sidecars:\r\n#  fluentbit:\r\n#    containerPort: 80\r\n#    image: 'amazon/aws-for-fluent-bit:1.2.0'\r\n#    memory: 512\r\n\r\n# This section defines each of the release stages\r\n# and their specific configuration for your app.\r\nstages:\r\n  -\r\n    # The \"environment\" (cluster/vpc/lb) to contain this service.\r\n    env: test\r\n    # The number of tasks that we want, at minimum.\r\n    desiredCount: 1\r\n    # Any secrets via ARNs\r\n    #secrets:\r\n    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd\r\n  -\r\n    # The \"environment\" (cluster/vpc/lb) to contain this service.\r\n    env: prod\r\n    # The number of tasks that we want, at minimum.\r\n    desiredCount: 3\r\n    # Any secrets via ARNs\r\n    #secrets:\r\n    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd\r\n"
2019-09-20T01:05:29.6160087Z         	            	
2019-09-20T01:05:29.6160229Z         	            	Diff:
2019-09-20T01:05:29.6160381Z         	            	--- Expected
2019-09-20T01:05:29.6160531Z         	            	+++ Actual
2019-09-20T01:05:29.6160687Z         	            	@@ -1,56 +1,56 @@
2019-09-20T01:05:29.6160900Z         	            	-# Your application name will be used in naming your resources
2019-09-20T01:05:29.6161103Z         	            	-# like log groups, services, etc.
2019-09-20T01:05:29.6161263Z         	            	-name: SweetApp
2019-09-20T01:05:29.6161531Z         	            	-# The "Type" of the application you're running. For a list of all types that we support see
2019-09-20T01:05:29.6161708Z         	            	-# https://github.com/aws/PRIVATE-amazon-ecs-archer/app/template/manifest/
2019-09-20T01:05:29.6161886Z         	            	-type: Load Balanced Web App
2019-09-20T01:05:29.6162091Z         	            	+# Your application name will be used in naming your resources
2019-09-20T01:05:29.6162282Z         	            	+# like log groups, services, etc.
2019-09-20T01:05:29.6162432Z         	            	+name: SweetApp
2019-09-20T01:05:29.6162698Z         	            	+# The "Type" of the application you're running. For a list of all types that we support see
2019-09-20T01:05:29.6162878Z         	            	+# https://github.com/aws/PRIVATE-amazon-ecs-archer/app/template/manifest/
2019-09-20T01:05:29.6163041Z         	            	+type: Load Balanced Web App
2019-09-20T01:05:29.6163173Z         	            	+
2019-09-20T01:05:29.6163385Z         	            	+# The port exposed through your container. We need to know
2019-09-20T01:05:29.6163588Z         	            	+# this so that we can route traffic to it.
2019-09-20T01:05:29.6163748Z         	            	+containerPort: 80
2019-09-20T01:05:29.6163862Z         	            	+
2019-09-20T01:05:29.6164010Z         	            	+# Size of CPU
2019-09-20T01:05:29.6164154Z         	            	+cpu: 256
2019-09-20T01:05:29.6164285Z         	            	+
2019-09-20T01:05:29.6164416Z         	            	+# Size of memory
2019-09-20T01:05:29.6164562Z         	            	+memory: 512
2019-09-20T01:05:29.6164692Z         	            	+
2019-09-20T01:05:29.6164920Z         	            	+# Logging is enabled by default. We'll create a loggroup that is
2019-09-20T01:05:29.6165194Z         	            	+# the SweetApp/Stage
2019-09-20T01:05:29.6165343Z         	            	+logging: true
2019-09-20T01:05:29.6165565Z         	            	+
2019-09-20T01:05:29.6165790Z         	            	+# Determines whether the application will have a public IP or not.
2019-09-20T01:05:29.6165948Z         	            	+public: true
2019-09-20T01:05:29.6166061Z         	            	+
2019-09-20T01:05:29.6166280Z         	            	+# You can also pass in environment variables as key/value pairs
2019-09-20T01:05:29.6166516Z         	            	+#environment-variables:
2019-09-20T01:05:29.6166669Z         	            	+#  dog: 'Clyde'
2019-09-20T01:05:29.6166856Z         	            	+#  cute: 'hekya'
2019-09-20T01:05:29.6166972Z         	            	+#
2019-09-20T01:05:29.6167205Z         	            	+# Additional Sidecar apps that can run along side your main application
2019-09-20T01:05:29.6167570Z         	            	+#sidecars:
2019-09-20T01:05:29.6167828Z         	            	+#  fluentbit:
2019-09-20T01:05:29.6168102Z         	            	+#    containerPort: 80
2019-09-20T01:05:29.6168275Z         	            	+#    image: 'amazon/aws-for-fluent-bit:1.2.0'
2019-09-20T01:05:29.6168488Z         	            	+#    memory: 512
2019-09-20T01:05:29.6168618Z         	            	+
2019-09-20T01:05:29.6168817Z         	            	+# This section defines each of the release stages
2019-09-20T01:05:29.6169024Z         	            	+# and their specific configuration for your app.
2019-09-20T01:05:29.6169155Z         	            	+stages:
2019-09-20T01:05:29.6169388Z         	            	+  -
2019-09-20T01:05:29.6169613Z         	            	+    # The "environment" (cluster/vpc/lb) to contain this service.
2019-09-20T01:05:29.6169849Z         	            	+    env: test
2019-09-20T01:05:29.6170036Z         	            	+    # The number of tasks that we want, at minimum.
2019-09-20T01:05:29.6170213Z         	            	+    desiredCount: 1
2019-09-20T01:05:29.6170389Z         	            	+    # Any secrets via ARNs
2019-09-20T01:05:29.6170639Z         	            	+    #secrets:
2019-09-20T01:05:29.6170835Z         	            	+    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd
2019-09-20T01:05:29.6170972Z         	            	+  -
2019-09-20T01:05:29.6171234Z         	            	+    # The "environment" (cluster/vpc/lb) to contain this service.
2019-09-20T01:05:29.6171438Z         	            	+    env: prod
2019-09-20T01:05:29.6171641Z         	            	+    # The number of tasks that we want, at minimum.
2019-09-20T01:05:29.6171808Z         	            	+    desiredCount: 3
2019-09-20T01:05:29.6171958Z         	            	+    # Any secrets via ARNs
2019-09-20T01:05:29.6172119Z         	            	+    #secrets:
2019-09-20T01:05:29.6172322Z         	            	+    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd
2019-09-20T01:05:29.6172480Z         	            	 
2019-09-20T01:05:29.6172693Z         	            	-# The port exposed through your container. We need to know
2019-09-20T01:05:29.6172877Z         	            	-# this so that we can route traffic to it.
2019-09-20T01:05:29.6173039Z         	            	-containerPort: 80
2019-09-20T01:05:29.6173229Z         	            	-
2019-09-20T01:05:29.6173416Z         	            	-# Size of CPU
2019-09-20T01:05:29.6173561Z         	            	-cpu: 256
2019-09-20T01:05:29.6173674Z         	            	-
2019-09-20T01:05:29.6173822Z         	            	-# Size of memory
2019-09-20T01:05:29.6173967Z         	            	-memory: 512
2019-09-20T01:05:29.6174097Z         	            	-
2019-09-20T01:05:29.6174299Z         	            	-# Logging is enabled by default. We'll create a loggroup that is
2019-09-20T01:05:29.6174475Z         	            	-# the SweetApp/Stage
2019-09-20T01:05:29.6174672Z         	            	-logging: true
2019-09-20T01:05:29.6174844Z         	            	-
2019-09-20T01:05:29.6175067Z         	            	-# Determines whether the application will have a public IP or not.
2019-09-20T01:05:29.6175208Z         	            	-public: true
2019-09-20T01:05:29.6175453Z         	            	-
2019-09-20T01:05:29.6175702Z         	            	-# You can also pass in environment variables as key/value pairs
2019-09-20T01:05:29.6175876Z         	            	-#environment-variables:
2019-09-20T01:05:29.6176010Z         	            	-#  dog: 'Clyde'
2019-09-20T01:05:29.6176162Z         	            	-#  cute: 'hekya'
2019-09-20T01:05:29.6176292Z         	            	-#
2019-09-20T01:05:29.6176987Z         	            	-# Additional Sidecar apps that can run along side your main application
2019-09-20T01:05:29.6177429Z         	            	-#sidecars:
2019-09-20T01:05:29.6177592Z         	            	-#  fluentbit:
2019-09-20T01:05:29.6177892Z         	            	-#    containerPort: 80
2019-09-20T01:05:29.6178162Z         	            	-#    image: 'amazon/aws-for-fluent-bit:1.2.0'
2019-09-20T01:05:29.6178322Z         	            	-#    memory: 512
2019-09-20T01:05:29.6178454Z         	            	-
2019-09-20T01:05:29.6178635Z         	            	-# This section defines each of the release stages
2019-09-20T01:05:29.6179151Z         	            	-# and their specific configuration for your app.
2019-09-20T01:05:29.6179652Z         	            	-stages:
2019-09-20T01:05:29.6179795Z         	            	-  -
2019-09-20T01:05:29.6180008Z         	            	-    # The "environment" (cluster/vpc/lb) to contain this service.
2019-09-20T01:05:29.6180178Z         	            	-    env: test
2019-09-20T01:05:29.6180449Z         	            	-    # The number of tasks that we want, at minimum.
2019-09-20T01:05:29.6180620Z         	            	-    desiredCount: 1
2019-09-20T01:05:29.6180817Z         	            	-    # Any secrets via ARNs
2019-09-20T01:05:29.6180952Z         	            	-    #secrets:
2019-09-20T01:05:29.6181163Z         	            	-    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd
2019-09-20T01:05:29.6181319Z         	            	-  -
2019-09-20T01:05:29.6181543Z         	            	-    # The "environment" (cluster/vpc/lb) to contain this service.
2019-09-20T01:05:29.6181701Z         	            	-    env: prod
2019-09-20T01:05:29.6181890Z         	            	-    # The number of tasks that we want, at minimum.
2019-09-20T01:05:29.6182055Z         	            	-    desiredCount: 3
2019-09-20T01:05:29.6182250Z         	            	-    # Any secrets via ARNs
2019-09-20T01:05:29.6182403Z         	            	-    #secrets:
2019-09-20T01:05:29.6182601Z         	            	-    #  lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd
2019-09-20T01:05:29.6182742Z         	            	-
2019-09-20T01:05:29.6183736Z         	Test:       	TestLoadBalancedFargateManifest_Marshal

Set up CI environment

Set up an integration test environment (CodeBuild) and travis builds.

We don't have to do this until we go public, since setting those things up in a private repo is tough.

Make project flag required on `env ls` command.

Current experience when running archer env ls:

ValidationException: The parameter doesn't meet the parameter name requirements. The parameter name must begin with a forward slash "/". It can't be prefixed with \"aws\" or \"ssm\" (case-insensitive). It must use only letters, numbers, or the following symbols: . (period), - (hyphen), _ (underscore). Special characters are not allowed. All sub-paths, if specified, must use the forward slash symbol "/". Valid example: /get/parameters2-/by1./path0_.
        status code: 400, request id: 45b53154-aa66-498b-b370-c2ce77ddf02d
Error: ValidationException: The parameter doesn't meet the parameter name requirements. The parameter name must begin with a forward slash "/". It can't be prefixed with \"aws\" or \"ssm\" (case-insensitive). It must use only letters, numbers, or the following symbols: . (period), - (hyphen), _ (underscore). Special characters are not allowed. All sub-paths, if specified, must use the forward slash symbol "/". Valid example: /get/parameters2-/by1./path0_.
        status code: 400, request id: 45b53154-aa66-498b-b370-c2ce77ddf02d

Add completion command

Story

As a user pressing <tab><tab>,
I want to have autocompletion on archer subcommands and flags,
So that I don't have to memorize the help menu.

Programming Model

Unfortunately, I couldn't find a way of having autocompletion be baked into the binary itself.
The recommendation from Cobra seems to be adding a new command archer completion bash|zsh that users can use to generate the shell completion code.
Kubectl has a section in their docs about it: https://kubernetes.io/docs/tasks/tools/install-kubectl/#enabling-shell-autocompletion

I'm going to follow the same pattern.

Deploy application command

ecs deploy

This command will:
0. Prompt the user to select which environment to deploy to (or accept it as a CLI param)

1. Build the current application's Dockerfile
2. Register it with ECR (Have we not created this yet?)
3. Build the Manifest to CloudFormation
4. Execute a CloudFormation update with the manifest's CF and ECR image

Indicate that go1.13 is required for development in README

With the introduction of errors.As users developing must have go1.13.

Move `includeLoadBalancer` flag into Environment structure

In our cloudformation package the DeployEnvironment method takes an archer.Environment and a bool that indicates whether or not we should create a load-balancer. We should probably just move that flag into the archer.Environment structure and remove it from the DeployEnvironment method.

Error handling when creating a CF stack that fails to spin up.

Currently we return a pretty cryptic error when a CF stack fails to create.

Workspace Package: Library to manage the local workspace:

ws := workspace.NewServce()
ws.RootDirectory() string
ws.ManifestDirectory() string
ws.LocalApplications() []*string
ws.WriteManifest(...) error
ws.Manifests() []*Manifest
ws.ProjectName() string

Settle on Name & Finalize going public plan

Validate project name

Story

As a user that entered the project name to the ecs init prompt or flag,
I want the CLI to give me an error before it proceeds creating my resources,
So that I don't do additional wasted work before getting an error message.

Acceptance Criteria

• Has to be less than 255 characters

This is because a project's name will be used as a tag value which has to be < 255 chars. Note: I'm not sure what this means for unicode characters we might want to test this out.

• Must be alphanumeric (A-Za-z0-9)

We will use a combination of project name and application name to create the Logical ID of CFN resources. CFN has a restriction of alphanumeric ids.

• Must be unique within the account.

Check if the tool account already has a tag value set to project.

Add init command

Story

As a user that's new to the archer CLI,
I want to type archer --help to see a list of available commands,
So that I know what I can use.

Acceptance Criteria

1. Add help menu to the top level

$ archer --help
Commands:
  Get started
      init                           Initialize a new application

Options:
    -h, --help                  Output usage information
    -v, --version              Output the version number
Examples:
  - Display the help menu for the init command
     `$ archer init --help`

2. Add help menu for init

$ archer init --help
   Create a new application from within an existing repository. 

   archer init [name] [options]

   Options:
      --project [name]    Name of the project this application belongs to

   Examples:
      - Guided walkthrough to create a `vote-api` app
        `$ archer init vote-api`
      - Skip prompts by using flags
        `$ archer init vote-api --project votes`

Add verbose or debug flag

If this flag is not provided, we set the logger to be /dev/null or something like that. If the flag is set we print to stdout.

Setting the output of the logger: https://golang.org/src/log/log.go?s=8239:8266#L255

Discard io.Writer: https://golang.org/pkg/io/ioutil/

Update project and env commands to use SSM Store

Currently we're calling SSM within the env and project structs. Move to calling the SSM Store.

Dummy issue to capture images

Add sidecar image support in the loadbalanced-fargate-service template

Update the loadbalanced-fargate-service.yml CloudFormation template to support sidecar images in the TaskDefinition

Fix failed TestManifestDirectoryPath

2019-09-20T17:46:28.5246958Z --- FAIL: TestManifestDirectoryPath (0.00s)
2019-09-20T17:46:28.5247605Z     --- FAIL: TestManifestDirectoryPath/same_directory_level (0.00s)
2019-09-20T17:46:28.5248236Z         workspace_test.go:265: 
2019-09-20T17:46:28.5248894Z             	Error Trace:	workspace_test.go:265
2019-09-20T17:46:28.5249217Z             	Error:      	Not equal: 
2019-09-20T17:46:28.5249522Z             	            	expected: "test/ecs"
2019-09-20T17:46:28.5249820Z             	            	actual  : "test\\ecs"
2019-09-20T17:46:28.5250431Z             	            	
2019-09-20T17:46:28.5250872Z             	            	Diff:
2019-09-20T17:46:28.5251327Z             	            	--- Expected
2019-09-20T17:46:28.5251853Z             	            	+++ Actual
2019-09-20T17:46:28.5252197Z             	            	@@ -1 +1 @@
2019-09-20T17:46:28.5252483Z             	            	-test/ecs
2019-09-20T17:46:28.5252764Z             	            	+test\ecs
2019-09-20T17:46:28.5253088Z             	Test:       	TestManifestDirectoryPath/same_directory_level
2019-09-20T17:46:28.5253743Z     --- PASS: TestManifestDirectoryPath/same_directory (0.00s)
2019-09-20T17:46:28.5254131Z     --- FAIL: TestManifestDirectoryPath/several_levels_deep (0.00s)
2019-09-20T17:46:28.5254697Z         workspace_test.go:265: 
2019-09-20T17:46:28.5255214Z             	Error Trace:	workspace_test.go:265
2019-09-20T17:46:28.5255634Z             	Error:      	Not equal: 
2019-09-20T17:46:28.5255938Z             	            	expected: "test/ecs"
2019-09-20T17:46:28.5256236Z             	            	actual  : "test\\ecs"
2019-09-20T17:46:28.5256844Z             	            	
2019-09-20T17:46:28.5257131Z             	            	Diff:
2019-09-20T17:46:28.5257428Z             	            	--- Expected
2019-09-20T17:46:28.5257715Z             	            	+++ Actual
2019-09-20T17:46:28.5257996Z             	            	@@ -1 +1 @@
2019-09-20T17:46:28.5258299Z             	            	-test/ecs
2019-09-20T17:46:28.5258576Z             	            	+test\ecs
2019-09-20T17:46:28.5258876Z             	Test:       	TestManifestDirectoryPath/several_levels_deep

Environment Initialization CloudFormation Template

We need to write a CloudFormation template that represents our Environment Infrastructure:

This should mostly be parametrized as much as possible. We'll focus on Fargate first, but in the future we'll add additional parameters to work with EC2 providers.

We'll also need to include 2 additional roles. One for CodePipelines and one for CLI management to use. These two roles have not been defined yet, so it's ok to get started without them, we can update them later.

The ALB should be optional (specified via a parameter).

A rough starting point would be:

AWSTemplateFormatVersion: '2010-09-09'
Description: This stack deploys a Fargate cluster that is in a VPC with both
             public and private subnets. Containers can be deployed into either
             the public subnets or the private subnets, and there are two load
             balancers. One is inside the public subnet, which can be used to
             send traffic to the containers in the private subnet, and one in
             the private subnet, which can be used for private internal traffic
             between internal services.

Parameters:
  ClusterName:
    Type: String

Mappings:
  # Hard values for the subnet masks. These masks define
  # the range of internal IP addresses that can be assigned.
  # The VPC can have all IP's from 10.0.0.0 to 10.0.255.255
  # There are four subnets which cover the ranges:
  #
  # 10.0.0.0 - 10.0.0.255
  # 10.0.1.0 - 10.0.1.255
  # 10.0.2.0 - 10.0.2.255
  # 10.0.3.0 - 10.0.3.255
  #
  # If you need more IP addresses (perhaps you have so many
  # instances that you run out) then you can customize these
  # ranges to add more
  SubnetConfig:
    VPC:
      CIDR: '10.0.0.0/16'
    PublicOne:
      CIDR: '10.0.0.0/24'
    PublicTwo:
      CIDR: '10.0.1.0/24'
    PrivateOne:
      CIDR: '10.0.2.0/24'
    PrivateTwo:
      CIDR: '10.0.3.0/24'
Resources:
  # VPC in which containers will be networked.
  # It has two public subnets, and two private subnets.
  # We distribute the subnets across the first two available subnets
  # for the region, for high availability.
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      EnableDnsSupport: true
      EnableDnsHostnames: true
      CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
      Tags:
         - Key: ecs-cli-project
           Value: !Ref ClusterName

  # Two public subnets, where containers can have public IP addresses
  PublicSubnetOne:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone:
         Fn::Select:
         - 0
         - Fn::GetAZs: {Ref: 'AWS::Region'}
      VpcId: !Ref 'VPC'
      CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']
      MapPublicIpOnLaunch: true
  PublicSubnetTwo:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone:
         Fn::Select:
         - 1
         - Fn::GetAZs: {Ref: 'AWS::Region'}
      VpcId: !Ref 'VPC'
      CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']
      MapPublicIpOnLaunch: true

  # Two private subnets where containers will only have private
  # IP addresses, and will only be reachable by other members of the
  # VPC
  PrivateSubnetOne:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone:
         Fn::Select:
         - 0
         - Fn::GetAZs: {Ref: 'AWS::Region'}
      VpcId: !Ref 'VPC'
      CidrBlock: !FindInMap ['SubnetConfig', 'PrivateOne', 'CIDR']
  PrivateSubnetTwo:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone:
         Fn::Select:
         - 1
         - Fn::GetAZs: {Ref: 'AWS::Region'}
      VpcId: !Ref 'VPC'
      CidrBlock: !FindInMap ['SubnetConfig', 'PrivateTwo', 'CIDR']

  # Setup networking resources for the public subnets. Containers
  # in the public subnets have public IP addresses and the routing table
  # sends network traffic via the internet gateway.
  InternetGateway:
    Type: AWS::EC2::InternetGateway
  GatewayAttachement:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref 'VPC'
      InternetGatewayId: !Ref 'InternetGateway'
  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref 'VPC'
  PublicRoute:
    Type: AWS::EC2::Route
    DependsOn: GatewayAttachement
    Properties:
      RouteTableId: !Ref 'PublicRouteTable'
      DestinationCidrBlock: '0.0.0.0/0'
      GatewayId: !Ref 'InternetGateway'
  PublicSubnetOneRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetOne
      RouteTableId: !Ref PublicRouteTable
  PublicSubnetTwoRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnetTwo
      RouteTableId: !Ref PublicRouteTable

  # Setup networking resources for the private subnets. Containers
  # in these subnets have only private IP addresses, and must use a NAT
  # gateway to talk to the internet. We launch two NAT gateways, one for
  # each private subnet.
  NatGatewayOneAttachment:
    Type: AWS::EC2::EIP
    DependsOn: GatewayAttachement
    Properties:
        Domain: vpc
  NatGatewayTwoAttachment:
    Type: AWS::EC2::EIP
    DependsOn: GatewayAttachement
    Properties:
        Domain: vpc
  NatGatewayOne:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt NatGatewayOneAttachment.AllocationId
      SubnetId: !Ref PublicSubnetOne
  NatGatewayTwo:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt NatGatewayTwoAttachment.AllocationId
      SubnetId: !Ref PublicSubnetTwo
  PrivateRouteTableOne:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref 'VPC'
  PrivateRouteOne:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PrivateRouteTableOne
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref NatGatewayOne
  PrivateRouteTableOneAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PrivateRouteTableOne
      SubnetId: !Ref PrivateSubnetOne
  PrivateRouteTableTwo:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref 'VPC'
  PrivateRouteTwo:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PrivateRouteTableTwo
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref NatGatewayTwo
  PrivateRouteTableTwoAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PrivateRouteTableTwo
      SubnetId: !Ref PrivateSubnetTwo

  # OPTIONAL: VPC Endpoint for DynamoDB
  # If a container needs to access DynamoDB this allows a container in the private subnet
  # to talk to DynamoDB directly without needing to go via the NAT gateway. This reduces
  # the amount of bandwidth through the gateway, meaning that the gateway is free to serve
  # your other traffic.
  DynamoDBEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action: "*"
            Principal: "*"
            Resource: "*"
      RouteTableIds:
        - !Ref 'PrivateRouteTableOne'
        - !Ref 'PrivateRouteTableTwo'
      ServiceName: !Join [ "", [ "com.amazonaws.", { "Ref": "AWS::Region" }, ".dynamodb" ] ]
      VpcId: !Ref 'VPC'

  # ECS Resources
  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
        ClusterName: !Ref ClusterName

  # A security group for the containers we will run in Fargate.
  # Three rules, allowing network traffic from a public facing load
  # balancer, a private internal load balancer, and from other members
  # of the security group.
  #
  # Remove any of the following ingress rules that are not needed.
  FargateContainerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the Fargate containers
      VpcId: !Ref 'VPC'
  EcsSecurityGroupIngressFromPublicALB:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from the public ALB
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG'
  EcsSecurityGroupIngressFromPrivateALB:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from the private ALB
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'PrivateLoadBalancerSG'
  EcsSecurityGroupIngressFromSelf:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from other containers in the same security group
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'FargateContainerSecurityGroup'

  # Load balancers for getting traffic to containers.
  # This sample template creates two load balancers:
  #
  # - One public load balancer, hosted in public subnets that is accessible
  #   to the public, and is intended to route traffic to one or more public
  #   facing services.
  # - One private load balancer, hosted in private subnets, that only
  #   accepts traffic from other containers in the Fargate cluster, and is
  #   intended for private services that should not be accessed directly
  #   by the public.

  # A public facing load balancer, this is used for accepting traffic from the public
  # internet and directing it to public facing microservices
  PublicLoadBalancerSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the public facing load balancer
      VpcId: !Ref 'VPC'
      SecurityGroupIngress:
          # Allow access to ALB from anywhere on the internet
          - CidrIp: 0.0.0.0/0
            IpProtocol: -1
  PublicLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    DependsOn: GatewayAttachement
    Properties:
      Scheme: internet-facing
      LoadBalancerAttributes:
      - Key: idle_timeout.timeout_seconds
        Value: '30'
      Subnets:
        # The load balancer is placed into the public subnets, so that traffic
        # from the internet can reach the load balancer directly via the internet gateway
        - !Ref PublicSubnetOne
        - !Ref PublicSubnetTwo
      SecurityGroups: [!Ref 'PublicLoadBalancerSG']
  # A dummy target group is used to setup the ALB to just drop traffic
  # initially, before any real service target groups have been added.
  DummyTargetGroupPublic:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 6
      HealthCheckPath: /
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 5
      HealthyThresholdCount: 2
      Name: !Join ['-', [!Ref 'AWS::StackName', 'drop-1']]
      Port: 80
      Protocol: HTTP
      UnhealthyThresholdCount: 2
      VpcId: !Ref 'VPC'
  PublicLoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    DependsOn:
      - PublicLoadBalancer
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref 'DummyTargetGroupPublic'
          Type: 'forward'
      LoadBalancerArn: !Ref 'PublicLoadBalancer'
      Port: 80
      Protocol: HTTP

  # An internal load balancer, this would be used for a service that is not
  # directly accessible to the public, but instead should only receive traffic
  # from your other services.
  PrivateLoadBalancerSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the internal load balancer
      VpcId: !Ref 'VPC'
  PrivateLoadBalancerIngressFromECS:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Only accept traffic from a container in the fargate container security group
      GroupId: !Ref 'PrivateLoadBalancerSG'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'FargateContainerSecurityGroup'
  PrivateLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Scheme: internal
      LoadBalancerAttributes:
      - Key: idle_timeout.timeout_seconds
        Value: '30'
      Subnets:
        # This load balancer is put into the private subnet, so that there is no
        # route for the public to even be able to access the private load balancer.
        - !Ref PrivateSubnetOne
        - !Ref PrivateSubnetTwo
      SecurityGroups: [!Ref 'PrivateLoadBalancerSG']
  # This dummy target group is used to setup the ALB to just drop traffic
  # initially, before any real service target groups have been added.
  DummyTargetGroupPrivate:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 6
      HealthCheckPath: /
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 5
      HealthyThresholdCount: 2
      Name: !Join ['-', [!Ref 'AWS::StackName', 'drop-2']]
      Port: 80
      Protocol: HTTP
      UnhealthyThresholdCount: 2
      VpcId: !Ref 'VPC'
  PrivateLoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    DependsOn:
      - PrivateLoadBalancer
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref 'DummyTargetGroupPrivate'
          Type: 'forward'
      LoadBalancerArn: !Ref 'PrivateLoadBalancer'
      Port: 80
      Protocol: HTTP

  # This is an IAM role which authorizes ECS to manage resources on your
  # account on your behalf, such as updating your load balancer with the
  # details of where your containers are, so that traffic can reach your
  # containers.
  ECSRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ecs.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:
      - PolicyName: ecs-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
              # Rules which allow ECS to attach network interfaces to instances
              # on your behalf in order for awsvpc networking mode to work right
              - 'ec2:AttachNetworkInterface'
              - 'ec2:CreateNetworkInterface'
              - 'ec2:CreateNetworkInterfacePermission'
              - 'ec2:DeleteNetworkInterface'
              - 'ec2:DeleteNetworkInterfacePermission'
              - 'ec2:Describe*'
              - 'ec2:DetachNetworkInterface'

              # Rules which allow ECS to update load balancers on your behalf
              # with the information sabout how to send traffic to your containers
              - 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
              - 'elasticloadbalancing:DeregisterTargets'
              - 'elasticloadbalancing:Describe*'
              - 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
              - 'elasticloadbalancing:RegisterTargets'
            Resource: '*'

  # This is a role which is used by the ECS tasks themselves.
  ECSTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ecs-tasks.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:
        - PolicyName: AmazonECSTaskExecutionRolePolicy
          PolicyDocument:
            Statement:
            - Effect: Allow
              Action:
                # Allow the ECS Tasks to download images from ECR
                - 'ecr:GetAuthorizationToken'
                - 'ecr:BatchCheckLayerAvailability'
                - 'ecr:GetDownloadUrlForLayer'
                - 'ecr:BatchGetImage'

                # Allow the ECS tasks to upload logs to CloudWatch
                - 'logs:CreateLogStream'
                - 'logs:PutLogEvents'
              Resource: '*'

# These are the values output by the CloudFormation template. Be careful
# about changing any of them, because of them are exported with specific
# names so that the other task related CF templates can use them.
Outputs:
  ClusterName:
    Description: The name of the ECS cluster
    Value: !Ref 'ECSCluster'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ClusterName' ] ]
  InternalUrl:
    Description: The url of the internal load balancer
    Value: !Join ['', ['http://', !GetAtt 'PrivateLoadBalancer.DNSName']]
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'InternalUrl' ] ]
  ExternalUrl:
    Description: The url of the external load balancer
    Value: !Join ['', ['http://', !GetAtt 'PublicLoadBalancer.DNSName']]
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ExternalUrl' ] ]
  ECSRole:
    Description: The ARN of the ECS role
    Value: !GetAtt 'ECSRole.Arn'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ECSRole' ] ]
  ECSTaskExecutionRole:
    Description: The ARN of the ECS role
    Value: !GetAtt 'ECSTaskExecutionRole.Arn'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ECSTaskExecutionRole' ] ]
  PublicListener:
    Description: The ARN of the public load balancer's Listener
    Value: !Ref PublicLoadBalancerListener
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicListener' ] ]
  PrivateListener:
    Description: The ARN of the private load balancer's Listener
    Value: !Ref PrivateLoadBalancerListener
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PrivateListener' ] ]
  VPCId:
    Description: The ID of the VPC that this stack is deployed in
    Value: !Ref 'VPC'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'VPCId' ] ]
  PublicSubnetOne:
    Description: Public subnet one
    Value: !Ref 'PublicSubnetOne'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetOne' ] ]
  PublicSubnetTwo:
    Description: Public subnet two
    Value: !Ref 'PublicSubnetTwo'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetTwo' ] ]
  PrivateSubnetOne:
    Description: Private subnet one
    Value: !Ref 'PrivateSubnetOne'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PrivateSubnetOne' ] ]
  PrivateSubnetTwo:
    Description: Private subnet two
    Value: !Ref 'PrivateSubnetTwo'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PrivateSubnetTwo' ] ]
  FargateContainerSecurityGroup:
    Description: A security group used to allow Fargate containers to receive traffic
    Value: !Ref 'FargateContainerSecurityGroup'
    Export:
      Name: !Join [ ':', [ !Ref 'AWS::StackName', 'FargateContainerSecurityGroup' ] ]

Validate project name uniqueness in an AWS account

Story

As a user that entered the project name to the ecs init prompt or flag,
I want the CLI to give me an error if the project name is not unique within my AWS account.

Acceptance Criteria

• Project name must be unique within the account.

Check if the tool account already has a tag value set to project.

Related Issues

• #5

Application permission design

Right now we use the default profile for folks using the ECS CLI for project management. We want to copy the ecs-cli and have a .config file with the name of an application as the profile name so that we can have reliable experience when running project commands

Fix the recent PTY unit test failures on Windows

Basically just due to the fact that the library used for unit testing does not support creating PTY on Windows.

=== RUN   TestEnvAdd_Ask/with_no_flags_set
panic: runtime error: invalid memory address or nil pointer dereference
	panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0xb4522b]

goroutine 20 [running]:
testing.tRunner.func1(0xc0003da800)
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/testing/testing.go:830 +0x6a4
panic(0xc25d20, 0x12cabe0)
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/runtime/panic.go:522 +0x1c3
github.com/Netflix/go-expect.(*Console).Close(0x0, 0x0, 0x0)
	D:/a/PRIVATE-amazon-ecs-archer/pkg/mod/github.com/!netflix/[email protected]/console.go:207 +0x3b
panic(0xc25d20, 0x12cabe0)
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/runtime/panic.go:522 +0x1c3
github.com/Netflix/go-expect.(*Console).Tty(...)
	D:/a/PRIVATE-amazon-ecs-archer/pkg/mod/github.com/!netflix/[email protected]/console.go:185
github.com/aws/PRIVATE-amazon-ecs-archer/internal/pkg/cli.TestEnvAdd_Ask.func3(0xc0003da800)
	D:/a/PRIVATE-amazon-ecs-archer/PRIVATE-amazon-ecs-archer/internal/pkg/cli/env_add_test.go:61 +0xb2

--- FAIL: TestEnvAdd_Ask (0.10s)
    --- FAIL: TestEnvAdd_Ask/with_no_flags_set (0.00s)
        env_add_test.go:56: 
            	Error Trace:	env_add_test.go:56
            	Error:      	Received unexpected error:
            	            	unsupported
            	Test:       	TestEnvAdd_Ask/with_no_flags_set
    --- FAIL: TestEnvAdd_Ask/with_env_flags_set (0.10s)
        env_add_test.go:56: 
            	Error Trace:	env_add_test.go:56
            	Error:      	Received unexpected error:
            	            	unsupported
            	Test:       	TestEnvAdd_Ask/with_env_flags_set

Fix conditional test failure - TestStore List/with multiple existing projects

The TestStore List/with multiple existing projects test randomly fails due to non-deterministic ordering:

--- FAIL: TestStore_List (0.01s)
    --- FAIL: TestStore_List/with_multiple_existing_projects (0.01s)
        store_test.go:59: 
                Error Trace:    store_test.go:59
                Error:          Not equal: 
                                expected: []string{"chicken", "cow"}
                                actual  : []string{"cow", "chicken"}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,4 +1,4 @@
                                 ([]string) (len=2) {
                                - (string) (len=7) "chicken",
                                - (string) (len=3) "cow"
                                + (string) (len=3) "cow",
                                + (string) (len=7) "chicken"
                                 }
                Test:           TestStore_List/with_multiple_existing_projects
FAIL

Add resource tags to Cloudformation templates

Add Resource tags to applicable resources in our environment.yml and loadbalanced-fargate-service.yml CloudFormation templates.

Related

#9

Generate Manifest

Story

As a user that ran ecs init,
I want to see a configuration file that captures my application configuration,
So that I can modify the file in the future to make changes to my app's infrastructure.

Acceptance Criteria

The prompts should ask the user for:

1. Project name
2. Dockerfile location
3. Application type

The Manifest file should be written under {project dir}/ecs/{app-name}-app.yml

The spec looks like:

version: 1.0
# First is the Project name. The Project is the grouping of the
# environments related to each other.
project: DavidsLemons

# This is the application specific to whatever code
# is in the same Directory as this file. Optionaly, you can
# model multiple applications in one template file. They'll
# all be released together. 
applications:
    # The name of the Application. This will be used to
    # prepend resources, name tasks, name log groups, etc.
    DavidsLemonsFrontend:
      # The "Type" of the application we're running. Currently we support
      #   LoadBalancedEc2Service
      #   LoadBalancedFargateService
      #   QueueProcessingEc2Service
      #   QueueProcessingFargateService
      #   ScheduledEc2Task
      #   ScheduledFargateTask
      type: LoadBalancedFargateService
      # The port exposed through your container. We need to know
      # this so that we can route traffic to it.
      containerPort: 80
      # This specifies the location of the Dockerfile for this application.
      # We'll use ./Dockerfile by default.
      # dockerfile: ./Dockerfile
      # Size of CPU
      cpu: '256'
      # Size of memory (all these params are stollen from the construct)
      memory: '512'
      # Logging is enabled by default. We'll create a loggroup that is
      # the Project/Application/Stage
      logging: true
      # Determines whether the application will have a public IP or not.
      public: true
      # You can also pass in environment variables as key/value pairs
      environment-variables:
       dog: 'Clyde'
       cute: 'hekya'
      #
      # You can also specify the loadbalancer configuration
      # you'd like. The LB is an environment configuration, so
      # we'll look it up from the ENV stack. You can tell
      # us which rules you'd like to route to our app.
      loadbalancer:
        name: 'DavidsLemons'
        rule:
          path: '/'
      # Additional Sidecar apps that can run along side your main application
      # We won't include the sidecars section by default.
      # The shape of the sidecar structure is a subset of the
      # container definition above.
      sidecars:
        fluentbit:
            containerPort: 80
            image: 'amazon/aws-for-fluent-bit:1.2.0'
            memory: '512'

# The stages section defines, esentially, the deployment stages of
# the Pipeline, while also giving us a place to store stage specific
# configuration.
stages:
  # Test and Production are the two default stages.
  # In every stage, you can override application wide parameters by
  # overwriting values. This will be used to generate stage specific taskdefs
  - test:
      # The "environment" (cluster/vpc/lb) to contain this service.
      env: test
      # The number of tasks that we want, at minimum.
      desiredCount: 2
      # Secrets are the same as environment variables, but the name
      # is a little more clear. We can also help out by making
      # sure non-arns aren't stored here? Also, default to secretsmanager
      # because SSM is no bueno.
      secrets:
        # The individual secrets can be added here, programatically.
        # archer app add-secret
        # secret name? lemonaidpassword
        # secret value for test stage? ***
        # secret value for prod stage? ***
        lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd/lemonaidpaw-jcXNEY
  - prod:
      # Again, the environment. You can put the CloudFormation stack
      # ARN instead of the short name. This allows for x-region, x-account
      # environments. Otherwise, it's realative to the Pipeline.s
      env: prod
      # Tagging a stage as production (or naming it prod) will trigger safe guards.
      # Safeguards like delete protection on CloudFormation, our own commands
      # will be more paranoid, etc. This may be an environment configuration, rather than a stage config in the future.
      production: true
      desiredCount: 2
      secrets:
        lemonaidpassword: arn:aws:secretsmanager:us-west-2:902697171733:secret:DavidsLemons/DavidsFrontEnd/lemonaidpaw-jcXNEY

Validate application name

Story

As a user that entered the app name to the ecs init prompt or flag,
I want the CLI to give me an error before it proceeds creating my resources,
So that I don't do additional wasted work before getting an error message.

Acceptance Criteria

• Has to be less than 255 characters

This is because an app's name will be used as a tag value which has to be < 255 chars. Note: I'm not sure what this means for unicode characters we might want to test this out.

• Must be alphanumeric (A-Za-z0-9)

We will use a combination of project name and application name to create the Logical ID of CFN resources. CFN has a restriction of alphanumeric ids.

• Must be unique within the project.

Maybe we should hold onto the SDK validations until we have the project management steps done.

Display progress with status updates

Story

As a user that's performing a long running operation (such as creating an application),
I want to see messages about which subtasks are being performed,
So that I know that the command is not stuck and still doing meaningful work.

Programming Model

We want to follow to display progress on long operations with a spinner and diagnostic messages as outlined in our style guide. This functionality should be in a separate package to make sure that progress display is re-usable.

Re-write manifest/ to follow new style

manifest/
manifest.go
  NewService(..)
  ListApplicationTypes()
  GenerateManifest(...) string

archer/
 manifest.go

e2e test that initialize a project

Fix the NPE in one of the unit tests on Windows

=== RUN   TestEnvList_Execute/with_production_envs
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0xb93777]

goroutine 30 [running]:
testing.tRunner.func1(0xc0003db400)
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/testing/testing.go:830 +0x6a4
panic(0xc28e80, 0x12cfc60)
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/runtime/panic.go:522 +0x1c3
github.com/Netflix/go-expect.(*Console).Tty(...)
	D:/a/PRIVATE-amazon-ecs-archer/pkg/mod/github.com/!netflix/[email protected]/console.go:185
github.com/aws/PRIVATE-amazon-ecs-archer/internal/pkg/cli.TestEnvList_Execute.func7(0xc0003db400)
	D:/a/PRIVATE-amazon-ecs-archer/PRIVATE-amazon-ecs-archer/internal/pkg/cli/env_list_test.go:98 +0xa7
testing.tRunner(0xc0003db400, 0xc000444210)
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/testing/testing.go:865 +0x16b
created by testing.(*T).Run
	C:/hostedtoolcache/windows/go/1.12.9/x64/src/testing/testing.go:916 +0x662
FAIL	github.com/aws/PRIVATE-amazon-ecs-archer/internal/pkg/cli	0.865s
?   	github.com/aws/PRIVATE-amazon-ecs-archer/internal/pkg/cli/mocks	[no test files]

Validate project name while listing environments

As a user running archer env ls,
I want the command to return an error when I don't provide the --project flag.
Otherwise, I see an AWS API error:

ValidationException: The parameter doesn't meet the parameter name requirements. The parameter name must begin with a forward slash "/". It can't be prefixed with \"aws\" or \"ssm\" (case-insensitive). It must use only letters, numbers, or the following symbols: . (period), - (hyphen), _ (underscore). Special characters are not allowed. All sub-paths, if specified, must use the forward slash symbol "/". Valid example: /get/parameters2-/by1./path0_.
	status code: 400, request id: ce73bb83-f04c-4bb6-8f9f-50f2e5c24b8f

Programming Model

Add a Validate() method to ListEnvOpts (https://github.com/aws/PRIVATE-amazon-ecs-archer/blob/master/pkg/cli/env_list.go#L65)

App Init: finish implementation

1. Write manifest using workspace pkg
2. Write to manager service when we create a projects
3. List Existing projects if we're not in a workspace
4. Check to see if we're in an existing ws (ws.ProjectName) to skip this question
5. Ask if we can deploy environment and then call deploy.DeployEnvironment

archer init is not writing version value to SSM

After running archer init and filling in the prompt values the version value is missing in SSM:

{"name":"MyProject","version":""}

Create HTTPS SecurityGruop in CloudFormation environment template

In environment.yml CloudFormation template the current SecurityGroup is only set up to support HTTP port 80 traffic. We should add to or change the SecurityGroupIngress property to include HTTPS port 443 traffic by default.

We need to discuss what defaults we expect for the PublicLoadBalancer construct and how we plan to enable HTTPS.

Validate application name uniqueness within project scope

Story

As a user that entered the application name to the ecs init prompt or flag,
I want the CLI to give me an error if the application name is not unique within my project.

Acceptance Criteria

• Application name must be unique within the project.

Check if the project already has an application with the same name.

Related Issues

• #6

Project show

ecs project show

This command should:

1. Detect which project the working directory is associated with via the Project.yml files in ecs/*Project.yml (if run in a working directory, otherwise it should prompt for the name of the project)

2. Show The environments associated with this project and the applications associated with this project.

Wire in manifest creator into INIT

Right now, the manifest creator isn't set wired in to the init command.

Environment Initialization

ecs env init (as well as the actual library component that will be called separately through ecs init)

Creates a CloudFormation stack for a new environment with the name {project}-{environment name} (or something like that) using the Environment Initialization CF template defined here.

Then, register that environment with the project via SSM.

When we create the env init CLI command, it should take in an optional AWS Credential Profile name.

Add support for Docker images from other repositories

Update the loadbalanced-fargate-service.yml CloudFormation template to support images from other repositories (e.g. private ECR repository).

LoadBalancedFargateService Manifest to CloudFormation converter

This component should be able to read a Project.yml manifest file of type "LoadBalancedFargateService" and convert it to CloudFormation.

archer init experience changes after creating first project

The first time a user runs archer init they're prompted for a Project Name. Running archer init a second time - If any project already exists users can only select the existing project. We should either default to the only existing project, or enable the user to create a new project.

Set spinner output to stderr

The default output is stdout. We should change it to stderr to be friendly to pipes.

Prompt the user for environment name in init flow

When a user runs the init command they are asked if they want to create a test environment. If they select y we're defaulting an environment name "test". We should also prompt them for a name with a default value of "test".

ecs project init

Implement SSM Pagination

Currently, we don't follow any pagination in the SSM store. This can lead to truncated lists.

Add a basic blackbox test

Add a simple e2e test that uses the archer cli

Project Initialization

ecs project init (as well as the library component that will be called separately from ecs init)

Creates a Project in SSM if one doesn't exist.

When a project is created, we’d create a record in SSM with the name of the project and a list of AWS Accounts:

ecs-cli/{project name}/linked-accounts

The value of that parameter will be a list of account IDs. This is important, because we can then import these parameters into CloudFormation, directly, when creating cross account policies for cross account CodePipelines.

ecs-cli/{project name}/environments

The environments parameter will maintain a map of environment name to environment stack ARN. Through this, we’ll get a list of all the named environments.

When a Project already exists, we don't recreate it.

Post Open Source Ops

• Add go report
• Add CodeBuild build for PRs
• Add travis build for linux/mac/windows
• Spruce up Issue Templates

Set up Integration Tests for Archer

We love integration tests - this task is to:

1. Set up the integration test structure in our CLI code
2. Write at least one integration test so that we know it runs

We son't set up an integration test environment until we go public (so that we can hook up CodeBuild to our GH repo)