lethalpaga/cfncli

Create Cloudformation stacks from the command line

cfncli's Issues

Catch and retry on API Throttling exceptions

When a stack is creating and cfncli is presumably just monitoring for events to display it can crash if the API attempts to date limit the actions.

These errors should be caught and rather than crashing, cfncli should do some sort of back-off and retry.

In the case logged below, the stack actually created successfully - it was only the process waiting that failed.

...
[17:21:54][Execute create_stack] favorit - 2016-06-29 05:23:01 UTC CREATE_IN_PROGRESS AWS::AutoScaling::AutoScalingGroup AppServerAutoScalingGroup 
[17:21:54][Execute create_stack] favorit - 2016-06-29 05:23:02 UTC CREATE_IN_PROGRESS AWS::AutoScaling::AutoScalingGroup AppServerAutoScalingGroup Resource creation Initiated
[17:22:18][Execute create_stack] 
[17:22:18]
[Execute create_stack] Aws::CloudFormation::Errors::Throttling: Rate exceeded

Stacktrace:
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-core-2.3.18/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-core-2.3.18/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-core-2.3.18/lib/aws-sdk-core/plugins/response_paging.rb:26:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-core-2.3.18/lib/seahorse/client/plugins/response_target.rb:21:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-core-2.3.18/lib/seahorse/client/request.rb:70:in `send_request'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-core-2.3.18/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-resources-2.3.18/lib/aws-sdk-resources/request.rb:24:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-resources-2.3.18/lib/aws-sdk-resources/operations.rb:41:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-resources-2.3.18/lib/aws-sdk-resources/operations.rb:61:in `call'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-resources-2.3.18/lib/aws-sdk-resources/resource.rb:147:in `load'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-resources-2.3.18/lib/aws-sdk-resources/resource.rb:120:in `data'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/aws-sdk-resources-2.3.18/lib/aws-sdk-resources/resource.rb:223:in `block in add_data_attribute'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/cfncli-0.4.0/lib/cfncli/stack.rb:120:in `finished?'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/cfncli-0.4.0/lib/cfncli/cloudformation.rb:48:in `block in apply_and_list_events'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/waiting-0.3.0/lib/waiting.rb:65:in `block in wait'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/waiting-0.3.0/lib/waiting.rb:55:in `loop'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/waiting-0.3.0/lib/waiting.rb:55:in `wait'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/cfncli-0.4.0/lib/cfncli/cloudformation.rb:47:in `apply_and_list_events'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/cfncli-0.4.0/lib/cfncli/cli.rb:139:in `apply'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/redacted-appserver-0.49.5/lib/redacted/appserver/rake/deploy.rake:84:in `block in <top (required)>'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:262:in `block in execute'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:90:in `target_exception_handling'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:268:in `execute'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/dogcatcher-0.2.1/lib/dogcatcher/ext/rake/task.rb:7:in `execute'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/home/teamcity/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:235:in `block in my_invoke_with_call_chain'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:90:in `target_exception_handling'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:234:in `my_invoke_with_call_chain'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:314:in `standard_exception_handling'
/home/teamcity/.rvm/gems/ruby-2.3.1/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rake_ext.rb:179:in `run'
/home/teamcity/BuildAgent/plugins/rake-runner/rb/runner/rakerunner.rb:40:in `<main>'

Event listing doesn't work when deleting a stack

When deleting a stack, the events are not listed, cfncli exists retutning 0.
This happens both when using the stack id (if the stack is already in deletion) and the stack name.

The stack is actually deleted, it's really just the event polling

Don't display logs form previous builds

When updating a stack, the events from the previous operations are displayed first, and not in chronological order.

Filter theses messages to only display the events concerning the current operation

Cannot recreate states in ROLLBACK_COMPLETE state

Given

I have a stack that has failed to create and is now in the ROLLBACK_COMPLETE state.

When

I rerun cfncli with the same template.

Then

I expect CloudFormation to attempt to recreate the stack.

But

cfncli attempts to update the stack, which fails with the following message from CloudFormation:

stack_name is in ROLLBACK_COMPLETE state and can not be updated.

Detect and stream events for child stacks

By detecting events with a Type of AWS::CloudFormation::Stack (excluding those with the same Logical ID as the current stack) an EventPoller and EventStreamer could be created (or the given the original) so that the events of child stacks also get streamed.

Default value on disable_rollback breaks the exclusivity requirement

Currently disable_rollback has a default value of false, and is mutually exclusive with on_failure. The fact that it has a default value means that you can never actually use on_failure as it will always fail the exclusivity check.

Parameters format

CloudFormation typically reads files with this format:

[
  {
    "ParameterKey": "KeyPairName",
    "ParameterValue": "MyKey"
  }, 
  {
    "ParameterKey": "InstanceType",
    "ParameterValue": "m1.micro"
  }
]

However, cfncli currently reads files with this format:

[
  {
    "KeyPairName": "MyKey"
  },
  {
    "InstanceType": "m1.micro"
  }
]

Ideally cfncli would read the format described by AWS.

Add delete and validate commands

Allow stack deletion and validation using the appropriate API endpoints