sage-bionetworks/scicomp-provisioner

An service to auto provision resources

Overview

Auto provision resources into AWS org-sagebase-scicomp account.

Workflow

The workflow to provision AWS resources is done using pull requests. Request using PRs provide history, gating, reviewing and an approval process.

Contributions

Contributions are welcome.

Requirements:

• Install pre-commit app
• Clone this repo
• Run pre-commit install to install the git hook.

Testing

As a pre-deployment step we syntatically validate our sceptre and cloudformation yaml files with pre-commit.

Please install pre-commit, once installed the file validations will automatically run on every commit. Alternatively you can manually execute the validations by running pre-commit run --all-files.

Provision resources

Instructions and workflow to auto provision and de-provision resources are in Example PRs

Stack & config file names

The value of a stack's stack_name parameter must match the config's file name. Stack names can contain only alphanumeric characters (case-sensitive) and hyphens. They must start with an alphabetic character and can't be longer than 128 characters.

Deployments

We use sceptre and cloudformation to deploy resources onto an AWS account.

Continuous Integration

We have configured Travis to deploy CF template updates.

Issues

• https://sagebionetworks.jira.com/projects/IT

Builds

• https://travis-ci.org/Sage-Bionetworks/scicomp-provisioner

Secrets

• We use the AWS SSM to store secrets for this project. Sceptre retrieves the secrets using a sceptre ssm resolver and passes them to the cloudformation stack on deployment.

scicomp-provisioner's Issues

Provide code/instructions to validate CF script locally

It'd be nice to include the relevant scripts (with some usage instructions) in this repo or somewhere easily accessible such that users could check their YAML file and debug any problems before creating a PR.

It looks like some of the scripts are

jobs:
  include:
    - stage: validate
      script:
        - yamllint ./config ./templates
        - ./lint_stack.sh -l ./config/prod
        - cfn-lint ./templates/**/*

It looks like some of the scripts (or at least lint_stack.sh are in this repo. Some of the inputs might need to be reconfigured as URLs instead of relative paths. Not sure if there are other dependencies that would be needed...