awslabs/aws-service-catalog-puppet

This is a framework where you list your AWS accounts with tags and your AWS Service Catalog products with tags or target accounts. The framework works through your lists, dedupes and spots collisions and then provisions the products into your AWS accounts for you. It handles the Portfolio sharing, its acceptance and can provision products cross account and cross region.

aws-service-catalog-puppet

Badges

What is it?

This is a python3 framework that makes it easier to share multi region AWS Service Catalog portfolios and makes it possible to provision products into accounts declaratively using a metadata based rules engine.

With this framework you define your accounts in a YAML file. You give each account a set of tags, a default region and a set of enabled regions.

Once you have done this you can define portfolios should be shared with each set of accounts using the tags and you can specify which regions the shares occur in.

In addition to this, you can also define products that should be provisioned into accounts using the same tag based approach. The framework will assume role into the target account and provision the product on your behalf.

Getting started

You can read the installation how to or you can read through the every day use guides.

You can read the documentation to understand the inner workings.

Going further

The framework is one of a pair. The other is aws-service-catalog-factory. With Service Catalog Factory you can create pipelines that deploy multi region portfolios very easily.

License

This library is licensed under the Apache 2.0 License.

aws-service-catalog-puppet's Issues

add 'status' to schema file

'status' needs to be added to the schema file for validating templates

Chained Dependencies

It would be useful if chained dependencies were respected in the manifest.

For example:

launch-A:
    portfolio: portfolio-example
    product: Product1
    version: v1
    deploy_to:
      accounts:
      - account_id: 012345678910
        regions: default_region

  launch-B:
    portfolio: portfolio-example
    product: Product2
    version: v1
    depends_on:
      - launch-A
    deploy_to:
      accounts:
      - account_id: 012345678910
        regions: default_region

  launch-C:
    portfolio: portfolio-example
    product: Product3
    version: v1
    depends_on:
      - launch-B
    deploy_to:
      accounts:
      - account_id: 012345678910
        regions: default_region

Would deploy Product1 followed by Product2 followed by Product3

Local Portfolio Setup - Imported Version Launched

At the moment I have an account that has a few products deployed to it from the “Imported Portfolio” setup. I now want to switch this over to use the new local portfolio setup (to make use of launch constraints).

I’ve added in config to switch it over to use the local setup, and this initially worked fine. I was expecting my products needing to be terminated & re-deployed after switching to the local portfolio but they didn’t (which I thought was a little odd, as the underlying product artefact should have changed).

I then tried terminating one of the products, and then asking Puppet to re-deploy it. I expected that Puppet would use the local product this time, but it instead used the imported one. Is this intended? I expected a local one to win over an imported one?

Thanks!

Enable working on aws-service-catalog-puppet branches

Complex dependencies cause duplicate tasks to be created

Puppet cannot create a plan when PuppetRole is re-created

In a spoke account, if the PuppetRole has been deleted & re-created, Puppet cannot execute a product plan as the new PuppetRole is no longer the owner of the provisioned product.

Ability to Nuke the provisioned product

Can we build the ability to Nuke the provisioned product from the spoke accounts? Just like we have for Factory?

something like:
All accounts:
servicecatalog-puppet nuke-product --protfolioname --productname --version --accounts all

Specific accounts:
servicecatalog-puppet nuke-product --protfolioname --productname --version --accounts '111111,2222222,3333333'

Specific OU
servicecatalog-puppet nuke-product --protfolioname --productname --version --accounts '/dev'

Portfolio shares with no launch constraints fail

allow the user to specify which regions to operate in

PuppetRole trust policy

Issue

When I enroll a new account as a spoke in an existing organization, it provisions a role in the spoke account that allows two sets of principals to assume it:

• All principals in the master account
• All principals in the spoke account

Description

The PuppetRole's permissions policy allows Administrator access (via managed policy) along with other permissions to accept portfolio shares.

Two issues here:

• Anyone with sts:AssumeRole in a spoke account can assume this role. While allowing sts:AssumeRole for all roles is a red flag in itself, it is unfortunately used as a pattern for 'jump accounts' where a security auditor might log into one account using SAML and then switch roles to different accounts with ReadOnly roles. This role (since it is a named IAM role) could allow them to become Admin.
• The PuppetRole has managed Admin as a policy but also other policies attached to it. Wouldn't the Admin managed policy give them all the permissions that they need (since it allows all actions for all resources)?

Potential solutions

• Allow the PuppetRole to be assumed by a known set of principals (either service principals, or standard role) from the master account
• Allow the PuppetRole to be assumed only by known principals in the spoke account. This is going to be tricky, since we may not know who these principals should be until they are provisioned. If the principal in the spoke account that can assume this is known (i.e. FooRole), someone else could create a low privileged role with that name and then switch before service-catalog-factory has had a chance to create the role that can assume PuppetRole

incorrect parameter name for bootstrap-spoke-as

[May 7, 2019, 6:21 PM] : return callback(*args, **kwargs)
TypeError: bootstrap_spoke_as() got an unexpected keyword argument 'iam_role_arn'

[Container] 2019/05/07 16:59:49 Command did not exit successfully servicecatalog-puppet bootstrap-spoke-as ${PUPPET_ACCOUNT_ID} ${ORG_IAM_ROLE_ARN} ${ORGANIZATION_ACCOUNT_ACCESS_ROLE_ARN} exit status 1
[May 7, 2019, 6:22 PM] : provisioned the account using GUI
[May 7, 2019, 6:34 PM] : Line 80 in in puppet cli.py
[May 7, 2019, 6:34 PM] : @click.argument('puppet_account_id')
@click.argument('iam_role_arn', nargs=-1)
def bootstrap_spoke_as(puppet_account_id, roles):

AWS Organizations / expand support for the pipeline

Currently the pipeline does not run the expand command on the manifest file.

Adding it to the pipeline will enable customers to use AWS Organizations features.

Terminate a product launch

It would be good to have the ability to terminate a product once it is launched into a spoke

Using parameters from SSM within a manifest

It would be nice to use SSM parameters within the manifest file.

Terminating a launch will not remove its SSM outputs

Allow configuring of associations in spoke accounts

It would be nice to allow the configuration of associations in the spoke accounts

docs/build is not gitignored

Updating parameter for already provisioned product doesn't trigger a product update

Changing the parameter on an already provisioned product doesn't trigger an update to the product.

It looks like Puppet only checks if the version or product artefact id has changed, whereas it should also check to see if any of the parameters have also changed to determine whether a product should be updated.

When running the pipeline with a product that has dependencies there is a failure: status is not a recognised parameter

Luigi execution summary showing pipeline success even if it has failed tasks.

It would be nice if we can get a failure in the deploy pipeline if Luigi fails in some of the tasks:

INFO:  
===== Luigi Execution Summary ===== 
 
Scheduled 15 tasks of which: 
* 9 ran successfully: 
    - 9 ProvisionProductTask(...) 
* 3 failed: 
    - 3 ProvisionProductTask(...) 
* 3 were left pending, among these: 
    * 3 had failed dependencies: 
        - 3 ProvisionProductTask(...) 
 
This progress looks :( because there were failed tasks 
 
===== Luigi Execution Summary ===== 
 
INFO MainThread  
===== Luigi Execution Summary ===== 
 
Scheduled 15 tasks of which: 
* 9 ran successfully: 
    - 9 ProvisionProductTask(...) 
* 3 failed: 
    - 3 ProvisionProductTask(...) 
* 3 were left pending, among these: 
    * 3 had failed dependencies: 
        - 3 ProvisionProductTask(...) 
 
This progress looks :( because there were failed tasks 
 
===== Luigi Execution Summary ===== 

[Container] 2019/06/11 11:57:09 Phase complete: BUILD State: SUCCEEDED 
[Container] 2019/06/11 11:57:09 Phase context status code:  Message:  
[Container] 2019/06/11 11:57:09 Entering phase POST_BUILD 
[Container] 2019/06/11 11:57:09 Phase complete: POST_BUILD State: SUCCEEDED 
[Container] 2019/06/11 11:57:09 Phase context status code:  Message:  

This will have as a result a green tick in the puppet pipeline but there were failed tasks that need to be looked into.

bootstrap-spoke-as should allow more than 1 iam role

My puppet account is not my org master account so I cannot make use of bootstrap-spoke-as.

Launching a product into the puppet account fails

• Listing imported shares is not working as the portfolio is not shared
• IAM principal association is missing

Launch constraints for spoke products

Service Catalog doesn't support launch constraints for imported products.

It would be nice for this framework to support the importing of products into spoke portfolios so that launch constraints could be set. Also setting launch constraints would be nice to have.

Error launching a product in another region

I am getting an error launching a product in another region due to list launch path not being able to see a path to launch the product

Updating a single account

Can you please add a command so it is possible to launch products for a single account only

Making regions optional in a launch

Can the regions option of the launches attribute become optional

When it is missing can it default to the default region.

The following:

launches:
  account-iam-for-prod:
    portfolio: demo-central-it-team-portfolio
    product: account-iam
    version: v1
    parameters:
      RoleName:
        default: DevAdmin
      Path:
        default: /human-roles/
    deploy_to:
      tags:
        - tag: type:prod
          regions: default_region

Would become:

launches:
  account-iam-for-prod:
    portfolio: demo-central-it-team-portfolio
    product: account-iam
    version: v1
    parameters:
      RoleName:
        default: DevAdmin
      Path:
        default: /human-roles/
    deploy_to:
      tags:
        - tag: type:prod

Allow adding SSM outputs to an already provisioned launch

When the AVM is launched and account is created, user decided that they needed to store the output from the stack to the SSM.
Today we support that, provided the product is not yet deployed. But if the product was deployed and only SSM paramaters were updated in the manifest file, the deploy process skips it.

It will be useful to support this.

Updating v1 to v2 of a product does nothing

Exclude Account from OU Expansion

It would be useful if I could exclude an account from a launches that targets an OU. E.g. maybe something like the below:

accounts:
  - ou: /
    name: 'all-accounts'
    default_region: eu-west-2
    regions_enabled:
      - eu-west-2
    tags:
      - ou:all-accounts

launches:
  product-all-accounts:
    portfolio: portfolio
    product: product
    version: v1
    deploy_to:
      tags:
        - tag: ou:all-accounts
          regions: default_region
          exclude: ['012345678910']

Or it could be defined in the accounts block.

Manual approval step in the pipeline

Could you please add a manual approval step into the pipeline before the deployment occurs?

Launch dependencies not working as expected

When launching a manifests with dependences like this:

launch-A:
    portfolio: portfolio-example
    product: Product1
    version: v1
    deploy_to:
      accounts:
      - account_id: *SomeAccountID
        regions: default_region

  launch-B:
    portfolio: portfolio-example
    product: Product2
    version: v1
    depends_on:
      - launch-A
    deploy_to:
      accounts:
      - account_id: *SomeAccountID
        regions: default_region

If launch-A fails launch-B will be executed, I believe that the expected outcome is that launch-B gets cancelled, along with other launches depending of launch-A, or launch-B and so on.

cleaner testing setup

dry run

Would it be possible to add a dry run command to the framework so we can see the impact of a change before it occurs.

Apply launch constraint to all products in a portfolio

At the moment when we specify Launch Constraints for a Portfolio, we need to do so for each product.

It would be great if there was an easy way to apply the constraint to all products in the portfolio.

spoke-local-portfolios:
  account-vending-for-spokes:
    portfolio: demo-central-it-team-portfolio
    depends_on:
      - account-iam-for-spokes
    associations:
      - arn:aws:iam::${AWS::AccountId}:role/MyServiceCatalogAdminRole
    constraints:
      launch:
        - product: <all_products>
          roles:
            - arn:aws:iam::${AWS::AccountId}:role/MyServiceCatalogAdminRole
    deploy_to:
      tags:
        - tag: scope:spoke
          regions: default_region

Don't mind how this is achieved. Maybe getting all products for a portfolio could be a macro, then we could pass the output of the macro into the launch constraint setup?

Product deployment failure stops all other product deployments to that account

It looks like when a product is failed to be deployed into an account, all further products meant to be deployed to that account aren't triggered.

I think this might be due to the entire thread for the account exiting when the product deployment fails.

I think it would be better if the account deployment thread continued on to try deploying the other products meant to be deployed to the account.

BucketPolicy and TopicPolicy in shares template do not make use of principal org id

OU exclusions from OU expansion

We have a concept of a Sandbox Account that users can request which have a fixed budget / duration before they are deleted. These are placed in a /sandbox OU. Some products that we'd normally target against all accounts, we would want to exclude from Sandbox accounts for various reasons (e.g. cost - we may not deploy a standard set of products into a sandbox account that cost $80 a month to run when the budget for the account may only be set to $100).

Improve Execution Summary

At the end of a Puppet run we get an output that looks something like below

===== Luigi Execution Summary ===== 
 
Scheduled 121 tasks of which: 
* 119 ran successfully: 
    - 119 ProvisionProductTask(...) 
* 2 failed: 
    - 2 ProvisionProductTask(...) 
 
This progress looks :( because there were failed tasks 
 
===== Luigi Execution Summary ===== 

It would be good if there were any failures, it also briefly summarised which launch & account the failed task occurred in.

Unable to terminate launch in v0.1.14

After upgrading to 0.1.14, setting the status of a launch to terminated is causing an exception (luigi.parameter.UnknownParameterException: TerminateProductTask[...]: unknown parameter worker_timeout) to be raised during the deploy state of the puppet pipeline (causing the pipeline to fail).

The error is raised during execution of servicecatalog-puppet --info deploy manifest-expanded.yaml

The worker_timeout parameter added in #68 to deploy_launches_task_builder_for_account_launch_region is missing from TerminateProductTask

sns:Publish is broken

When running a puppet deploy I am getting the error:

Exception: Plan was not successful: Error calling API cloudformation:CreateChangeSet. ErrorCode: ValidationError, Message: User: arn:aws:sts::123:assumed-role/PuppetRole/servicecatalog_for_account_123 is not authorized to perform: SNS:Publish on resource: arn:aws:sns:eu-west-1:321:servicecatalog-puppet-cloudformation-events (Service: AmazonSNS; Status Code: 403; Error Code: AuthorizationError;

Stuck: Need to wait for stack completion

When deploying an update to a product that was a dependency to another product, the build hung on the below log line:

Need to wait for stack completion

The build waited on this step for 10 minutes before I stopped it, even though the stack finished deploying in about 30 seconds.

A subsequent run of the pipeline (after stopping the initial hung build) succeeds.

Looks like the waiter never returns?

Exclude OU from definition

In our current Puppet manifest we have something like this defined to target a launch against all accounts

schema: puppet-2019-04-01

accounts:
  - ou: /
    name: 'all-accounts'
    default_region: eu-west-2
    regions_enabled:
      - eu-west-2
    tags:
      - ou:all-accounts

launches:
  my-product-all-accounts:
    portfolio: my-portfolio
    product: my-product
    version: v1
    deploy_to:
      tags:
        - tag: ou:all-accounts
          regions: default_region

When we close an AWS Account in our Organisation we clean up all resources inside it, close the account down, then move the account to a graveyard OU until the standard 90 days pass (where the account is automatically removed).

The issue we have at the moment is (using the above approach), Puppet still tries to deploy a product to the closed account in the /graveyard OU, causing the Puppet run to fail with an botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the AssumeRole operation: Access denied error.

It would be good if we could define something like the below to exclude certain OUs from the target OU:

  - ou: /
    name: 'all-accounts'
    default_region: eu-west-2
    regions_enabled:
      - us-east-1
      - eu-west-2
    exclude:
      - /graveyard
      - /path/to/my/ou
    tags:
      - ou:all-accounts

Local Portfolio - Deployment Fails

If I try and deploy a local portfolio product with Puppet I get the below error in my pipeline

INFO MainThread Getting path for product prod-<id>
Traceback (most recent call last): 
  File "/usr/local/bin/servicecatalog-puppet", line 11, in <module> 
    sys.exit(cli()) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__ 
    return self.main(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main 
    rv = self.invoke(ctx) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke 
    return _process_result(sub_ctx.command.invoke(sub_ctx)) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke 
    return ctx.invoke(self.callback, **ctx.params) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke 
    return callback(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli.py", line 27, in deploy 
    cli_commands.deploy(f, single_account) 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli_commands.py", line 64, in deploy 
    all_launch_tasks = cli_command_helpers.deploy_launches(manifest) 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli_command_helpers.py", line 498, in deploy_launches 
    all_tasks = deploy_launches_task_builder(deployment_map, manifest, puppet_account_id, section) 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli_command_helpers.py", line 517, in deploy_launches_task_builder 
    regional_details, 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli_command_helpers.py", line 538, in deploy_launches_task_builder_for_account_launch_region 
    PathId=aws.get_path_for_product(service_catalog, product_id), 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/aws.py", line 204, in get_path_for_product 
    response = service_catalog.list_launch_paths(ProductId=product_id) 
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call 
    return self._make_api_call(operation_name, kwargs) 
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call 
    raise error_class(parsed_response, operation_name) 
botocore.errorfactory.ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the ListLaunchPaths operation: No launch paths found for resource: prod-<id> 

I've added the PuppetRole as an association to the local portfolio. The target account has no other products deployed to it, and there are no imported portfolios available either.

Schema has diverged from its definition

Error sharing local portfolio

Getting an error when sharing a portfolio locally using 0.3.0. Snippet is below

INFO MainThread [test-base] 012345678910:eu-west-2 :: test-base-audit-compliance status: FAILED 
ERROR: [pid 329] Worker Worker(salt=011027649, workers=10, host=768f187a1b2a, username=root, pid=100) failed    ImportIntoSpokeLocalPortfolioTask(account_id=012345678910, region=eu-west-2, portfolio=test-base, hub_portfolio_id=port-gwr7wrqzepuny) 
Traceback (most recent call last): 
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 199, in run 
    new_deps = self._run_get_new_deps() 
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 139, in _run_get_new_deps 
    task_gen = self.task.run() 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/luigi_tasks_and_targets.py", line 525, in run 
    raise Exception(f"[{self.portfolio}] {self.account_id}:{self.region} :: Copying " 
Exception: [test-base] 012345678910:eu-west-2 :: Copying test-base-audit-compliance failed: Internal Failure 
ERROR MainThread [pid 329] Worker Worker(salt=011027649, workers=10, host=768f187a1b2a, username=root, pid=100) failed    ImportIntoSpokeLocalPortfolioTask(account_id=012345678910, region=eu-west-2, portfolio=test-base, hub_portfolio_id=port-gwr7wrqzepuny) 
Traceback (most recent call last): 
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 199, in run 
    new_deps = self._run_get_new_deps() 
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 139, in _run_get_new_deps 
    task_gen = self.task.run() 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/luigi_tasks_and_targets.py", line 525, in run 
    raise Exception(f"[{self.portfolio}] {self.account_id}:{self.region} :: Copying " 
Exception: [test-base] 012345678910:eu-west-2 :: Copying test-base-audit-compliance failed: Internal Failure 
INFO: Informed scheduler that task   ImportIntoSpokeLocalPortfolioTask_012345678910_port_gwr7wrqzepu_test_base_c1f5fe62bc   has status   FAILED 
INFO MainThread Informed scheduler that task   ImportIntoSpokeLocalPortfolioTask_012345678910_port_gwr7wrqzepu_test_base_c1f5fe62bc   has status   FAILED 

Manifest snippet:

spoke-local-portfolios:
  optional-products-for-spokes:
    portfolio: test-base
    depends_on: []
    associations:
      - arn:aws:iam::${AWS::AccountId}:role/ServiceCatalogDeploymentRole
      - arn:aws:iam::${AWS::AccountId}:role/servicecatalog-puppet/PuppetRole
    constraints:
      launch:
        - products: "test-*"
          roles:
            - arn:aws:iam::${AWS::AccountId}:role/ServiceCatalogDeploymentRole
    deploy_to:
      tags:
        - tag: scope:spoke
          regions: default_region

Portfolio is created in spoke with associations, but no products have been copied across.

Document how to add SCP guardrails to ensure only SC products can perform actions like creating a VPC

Can all buckets be made not public

Provide a clear description of what this is

I felt that the what this is page, describes how aws-service-catalog-puppet does stuff as opposed to what it is. I've created a PR to describe what it is, LMK if that looks like a reasonable description

Products are provisioned when nothing changes due to default params in the cfn template

Deploy fails because manifest_params is NoneType object

When running:

servicecatalog-puppet --info deploy manifest-expanded.yaml

If there are no portfolio level parameters in the manifest.yaml file this will result in the error:

Traceback (most recent call last): 
  File "/usr/local/bin/servicecatalog-puppet", line 11, in <module> 
    sys.exit(cli()) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__ 
    return self.main(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main 
    rv = self.invoke(ctx) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke 
    return _process_result(sub_ctx.command.invoke(sub_ctx)) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke 
    return ctx.invoke(self.callback, **ctx.params) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke 
    return callback(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli.py", line 234, in deploy 
    required_parameters, deployment_map, manifest, launch_details, account_id 
  File "/usr/local/lib/python3.7/site-packages/servicecatalog_puppet/cli.py", line 152, in get_parameters_for_launch 
    manifest_ssm_param = manifest_params.get(required_parameter_name, {}).get('ssm') 
AttributeError: 'NoneType' object has no attribute 'get'