vistaprint/terraformdevkit

Set of scripts to ease development and testing with Terraform.

terraformdevkit's Issues

Don't prompt for "apply" if "plan" is empty

When plan is empty, TerraformDevkit still prompts for a terraform apply. In case it's empty, we may want to skip terraform apply itself (while still doing pre_apply and post_apply)

Remove dependency on terragrunt

rake apply[env] currently fails because of gruntwork-io/terragrunt#316

At the moment we only use terragrunt to create the lock table in dynamodb. We can automate this using the projects name and the aws-sdk in ruby.

Make environment management independent of AWS

At this moment TerraformDevKit requires users to provide AWS credentials. In addition to that, it is not clear how easy or transparent is to use other providers (e.g., Azure or Google). For instance, prod and test environments require manual input of the AWS profile.

Perhaps users should just pass all credentials info as part of the extra vars that end up being used when rendering the templates. For safety reasons, instead of asking for the profile name, we could just use some sort of captcha (e.g., asking for the result of a mathematical operation as well as printing some clear warning message).

Hostname may contain invalid characters

Preflight task uses Socket.gethostname to obtain the hostname, and then construct the temporal environment name together with a timestamp.

If the hostname contains an invalid character (i.e., any non-alphanumeric character) the execution will fail.

Finish status on rake preflight

When running rake preflight , there is no finish status after the task has completed and one has to determine if it succeeded or not based on the presence of error messages in the console.

Printing the execution result ( preflight SUCCEEDED / preflight FAILED ) would be useful.

Enable Versioning on Remote State S3 bucket

Currently S3 buckets for remote state are set with default values. It would be useful to enable versioning so if a state is changed we can revert back,

Add post apply action

After creating the infrastructure it might be necessary to do some further action. For instance, Athena tables cannot be created with terraform (yet). So, they need to be created by using the SDK. A post_apply task would be the perfect fit for these type of actions.

Generic approach to CI execution

We currently use running_on_jenkins? to determine whether the execution is part of a continuous integration process. This is certainly not very generic.

For AWS infrastructures, for instance, one of the differences when running on jenkins is that the shared credentials file does not typically exist on the jenkins server. Instead, a key and secret are used. On development machines, using the profile information stored in the shared credentials file is the preferred option as it is more flexible.

It would be desirable to find a more generic approach to solve this requirement.

Use Travis to auto publish to RubyGems

We can use travis to auto publish to rubygems when a tag is added to the repo. See the Docs

Add ability to "apply" without manual input (i.e. from Jenkins)

TerraformDevKit/lib/TerraformDevKit/terraform_config_manager.rb

Line 31 in 9ade526

unless env.local_backend?

Abort execution when terraform fails

In some case the result of executing terraform is not checked (e.g., see

TerraformDevKit/tasks/devkit.rake

Line 98 in 28b0ecd

system("terraform plan -out=#{PLAN_FILE}")

).

Fix that. Ideally, we could use the latest implementation of Command.run (available in feature/use-fixed-command).

Add hook right at the beginning of the prepare process

In some cases it is desirable to auto-generate some files that later during the prepare step will be copied into the environment folder. To do so, an extra hook is needed.

Make skipping the update of modules the default behavior

While updating the modules each time prevents using an old version of the modules, it also takes significantly longer. If modules are obtained from git (using a tag) there should be automatically updated when the version tag is changed. So, other than when using local modules, it seems there shouldn't be much need to update them every time.

Delete tf files when a mustache.tf file is deleted.

When a mustache file is deleted the corresponding generated tf file in the envs folder is not deleted and must be removed manually. The same happens when a file is renamed, the original file is kept and the new file created.

We could remove all the tf files before generating them or check that the envs folder contains only tf files that have a corresponding mustache file, removing those that don't.

Support for auto apply in Terraform 0.11

Since version 0.11.0 Terrafrom requires user input to run an apply in the same way that destroy requires user input to destroy infrastructure.

Right now the apply task cannot be used to force create infrastructure like destroy can be forced to teardown.

We should update the apply task to allow users to force create infrastructure if we´re in dev.

Create method to wait for a URL to be ready

TerraformDevKit currently includes a command line utility to wait for a URL to be ready. There is no such a feature in the library itself. It would be desirable to move the request with retry implementation to the library, and then make the console application call the library.

Remove dependency to ROOT_PATH

There is no real need to use ROOT_PATH as long as rake is executed in the directory where the Rakefile and the rest of the infrastructure files are located. Using a global variable like ROOT_PATH makes testing more complicated and the code not so clear.

Add a cli to init a project

Given that we have to set up a few files to get off the ground it might be worthwhile to add a small cli to the package which would allow users to generate the initial project structure by supplying a couple of parameters on the command line.

Don't clean local environment if destroy fails

When performing a rake clean[myenv] command, if the terraform destroy process fails, the clean command removes the environment myenv. That prevents trying to destroy it later.

Do not expect template files to have mustache extension

Currently we only treat as template files any file that ends in .mustache. We could just support specifying folders/files to use as templates.