aws-ia/taskcat

Test all the CloudFormation things! (with TaskCat)

What is TaskCat?

TaskCat is a tool that tests AWS CloudFormation templates. It deploys your AWS CloudFormation template in multiple AWS Regions and generates a report with a pass/fail grade for each region. You can specify the regions and number of Availability Zones you want to include in the test, and pass in parameter values from your AWS CloudFormation template. TaskCat is implemented as a Python class that you import, instantiate, and run.

TaskCat was developed by the aws-ia team to test AWS CloudFormation templates that automatically deploy workloads on AWS. We’re pleased to make the tool available to all developers who want to validate their custom AWS CloudFormation templates across AWS Regions

See TaskCat documentation.

Support

GitHub

PyPi

taskcat's Issues

Add tags via command line argument

• Add cli option -t --tag

S3upload (does not clean up existing objects

If objects with the same key/prefix exist in s3 bucket clean up prior to uploading.

Erroneous data in Resource logs

Resource logs are showing all resources for previously completed test (previous region data is incorrectly appended)
Fix resource logs to only report on the individual test

Cannot find directory (when run inside the project)

Hi,

I'm using Taskcat version 2017.908.112800 on MacOS.

My project is:

• Path: /Users/abc/sample-taskcat-project
• Subfolders:

.
├── ci
│   ├── config.yml
│   ├── debug-input.json
│   ├── taskcat-autobucket.yml
│   └── taskcat.yml
├── scripts
│   └── scripts_userdata.sh
└── templates
    ├── debug-yaml.template
    └── debug.template

• Config.yml:

global:
  owner: [email protected]
  qsname: sample-taskcat-project
  regions:
    - ap-southeast-1

  reporting: true
tests:
  taskcat-test1:
    parameter_input: debug-input.json
    regions:
      - ap-southeast-1
    template_file: debug.template
``

The When I ran `taskcat -c ci/config.yml`. I got this error.

[taskcat] :Reading Config form: ci/config.yml
[taskcat] |Queing test => taskcat-test1
[INFO ] :Creating bucket taskcat-sample-taskcat-project-f3cab85a in us-east-1
[INFO ] :Staging Bucket => [taskcat-sample-taskcat-project-f3cab85a]
Hint: The name specfied as value of qsname (sample-taskcat-project)
must match the root directory of your project
[ERROR ] :!Cannot find directory [sample-taskcat-project] in /Users/cduong13/GDrive/Workspace/sample-taskcat-project
[INFO ] :Please cd to where you project is located

Another confusion is, in README, you say the `config.yml` file use the key `project` for the PROJECT NAME but in the example (the `taskcat.yml` use `qsname` key).
After the above error, I tried to use `project`, it would complain that **global: qsname missing from ci/config.yml**.

Best,

End-User Feature Request: command line parameter to skip uploading to S3

Branching this off from #52:

"Can you please make this as command line parameter where we skip upload s3 and just test ?"

cc: @pankajvkulkarni

Backlog: Integers to Strings before validation

Use-case to do type-validation on input parameters. If they're integers, convert to strings.

Changes to pypi breaks version check

Traceback (most recent call last):
File "/usr/local/bin/taskcat", line 58, in
main()
File "/usr/local/bin/taskcat", line 22, in main
tcat_instance.welcome('taskcat')
File "/usr/local/lib/python3.6/site-packages/taskcat/stacker.py", line 2192, in welcome
self.checkforupdate()
File "/usr/local/lib/python3.6/site-packages/taskcat/stacker.py", line 2173, in checkforupdate
if version in current_version:
TypeError: argument of type 'NoneType' is not iterable

Auto-gen'd S3 bucket not deleted after run

Taskcat is not removing the S3 bucket at the end of the test if --no_cleanup is not specified

Default region - aws_default_region environment variable

I noticed that on this line https://github.com/aws-quickstart/taskcat/blob/master/taskcat/stacker.py#L167
default region is set as "us-east-1". as follows

self.defult_region = "us-east-1"

I am setting the "aws_default_region environment variable" . Taskcast should use the "aws_default_region" environment variable instead setting its own.

You can set the default region to "us-east-1" if aws_default_region is not available in environment.

Test Creation fail (region not defined)

[DEBUG ] :Requested 3 az's
[ERROR ] :name 'region' is not defined

Public Read write access to the buckets where resources are uploaded.

Hi,

I see that the buckets which are getting created as got public-read

1. https://github.com/aws-quickstart/taskcat/blob/master/taskcat/stacker.py#L317
2. https://github.com/aws-quickstart/taskcat/blob/master/taskcat/stacker.py#L325

Also the objects which are being uploaded to the bucket has got public-read permission

https://github.com/aws-quickstart/taskcat/blob/master/taskcat/stacker.py#L349

Is it necessary to have the "public-read" for the buckets and the objects ?

Cannot install Taskcat

I am trying to install Taskcat on my Macbook, but I am getting this error each time I try to run the installation script.

Traceback (most recent call last):
File "/usr/local/bin/taskcat", line 13, in <module>
from taskcat import TaskCat
ImportError: No module named taskcat

The instructions in the readme for installing do not work, and I have had to try and curl the pip3-install-master file manually and run using my python install.

module docs linked in README.md are not publicly accessible

when following the [taskcat.Taskcat] and [taskcat.Sweeper] links in the readme s3 returns a 403 Forbidden... need to make the objects public perhaps ?

Testcase directories structure documentation defect

I noticed that the ASCII art depicting the directory structure on the main page under the heading

Example of project directory structure

shows that all the directories - scripts, templates, submodules etc. should be placed as subdirectories of 'ci'
However the testcase example directory has all of them at the same level as 'ci'.

I haven't tested if it works by placing everything under 'ci' too but this is a mismatch in the doc v/s example

Not able to generate a password with symbols

Running taskcat version 2017.821.154206
Docker version 17.06.1-ce, build 874a737

my parameter file has following:
{
"ParameterKey": "AdminPassword",
"ParameterValue": "$[alfred_genpass_32S]"
}
When I run the test locally on my mac, getting error:

[ERROR ] :An error occurred (ValidationError) when calling the CreateStack operation: Parameter DBMasterUserPassword failed to satisfy constraint: Must be at least 8 characters containing letters, numbers and symbols.

I pulled out and debugged the password generation code itself (see attached x.py) and see this:

param_value=$[alfred_genpass_32S]
passlen=32
gentype=a
[DEBUG ] :AutoGen values for $[alfred_genpass_32S]
[DEBUG ] :Auto generating password
[DEBUG ] :Pass size => 32
[DEBUG ] :Pass type => default alpha-numeric
param_value=rP7dK5aX7aJ6vB4mS9hS1wL5eQ7nJ5uE4

The gentype detected is 'a'.

Looking at the gentype_re, I see that re us matching the first 'a' (in alfred) and using that:

test.zip

Testing layered stacks

Is there a way we can test stacks with layered architecture? Especially when there are dependencies between stacks? So for example if I have the configuration like below:

---
global:
  owner: [email protected]  
  qsname: test-shared-services
  regions:
    - us-east-1
    - us-west-2
  reporting: true
  tests:
    test-vpc:
      parameter_input: vpc-parameters.json
      regions:
      - us-east-1
      template_file: vpc/vpc.yaml
    test-app
      parameter_input: app-parameters.json
      regions:
      - us-east-1
      template_file: app/app.yaml

Here I have a template app.yaml which requires the vpc stack to be present. So now if I run taskcat with this configuration, what is ordering of task creation and deletion?

Create schema documentation

Create scheme documentation

Convert from requests to S3 get_object calls

• Need to move away from dependency on public calls to S3 buckets. Involves converting references to 'requests' to the an equivalent s3.get_object call.

Sweeper deletion Info error (needs investigation)

2017-05-02 17:43:36,294 - Sweeper - INFO - Deleting all resources
2017-05-02 17:43:36,294 - Sweeper - DEBUG - Found Security Group resource
2017-05-02 17:43:36,461 - Sweeper - INFO - Deleting Security Group [DSESecurityGroup]
An error occurred (InvalidGroupId.Malformed) when calling the DeleteSecurityGroup operation: Invalid id: "DSESecurityGroup" (expecting "sg-...")
2017-05-02 17:43:36,986 - Sweeper - DEBUG - Found Security Group resource
2017-05-02 17:43:36,993 - Sweeper - INFO - Deleting Security Group [DSESecurityGroup]
An error occurred (InvalidGroupId.Malformed) when calling the DeleteSecurityGroup operation: Invalid id: "DSESecurityGroup" (expecting "sg-...")
2017-05-02 17:43:37,480 - Sweeper - DEBUG - Found Security Group resource
2017-05-02 17:43:37,488 - Sweeper - INFO - Deleting Security Group [DSESecurityGroup]
An error occurred (InvalidGroupId.Malformed) when calling the DeleteSecurityGroup operation: Invalid id: "DSESecurityGroup" (expecting "sg-...")
2017-05-02 17:43:37,964 - Sweeper - DEBUG - Found Security Group resource
2017-05-02 17:43:37,975 - Sweeper - INFO - Deleting Security Group [DSESecurityGroup]
An error occurred (InvalidGroupId.Malformed) when calling the DeleteSecurityGroup operation: Invalid id: "DSESecurityGroup" (expecting "sg-...")
2017-05-02 17:43:38,450 - Sweeper - DEBUG - Found Security Group resource
2017-05-02 17:43:38,460 - Sweeper - INFO - Deleting Security Group [OpsCSecurityGroup]
An error occurred (InvalidGroupId.Malformed) when calling the DeleteSecurityGroup operation: Invalid id: "OpsCSecurityGroup" (expecting "sg-...")
2017-05-02 17:43:38,952 - Sweeper - DEBUG - Found Bucket resource
2017-05-02 17:43:39,003 - Sweeper - INFO - Working on bucket [tcat-tag-datastax-21c7-s3bucket-ynx0dcfhj6kl]
2017-05-02 17:43:39,005 - Sweeper - INFO - Getting and deleting all object versions
2017-05-02 17:43:41,718 - Sweeper - INFO - Deleting bucket [tcat-tag-datastax-21c7-s3bucket-ynx0dcfhj6kl]

taskcat pip package (update)

Update pip package so all assets and are packaged and deployed to 'bin'
Simplify installer to pip install taskcat

Deletion not working after CREATE_FAILED

The stack is not deleting when it fails creation. I have to go into CFN control panel manually and do that. Can you look into this.

How to use with private S3 bucket

Hi,

If I use my private S3 bucket, I would face the "permission denied". Taskcat does not allow to Assume a Role (Which have access to my private bucket) when launching the Nest Stack.

CloudFormation console supports it, could Taskcat do the same.

Thanks,

Unable to locate Credentials

We have a Cloudformation template created and I am in the process of testing this using TaskCat. Here’s the command I am using

taskcat -c awstaskcat/ci/config.yml -A "AKIAJXXXXXXXE6GEVXX" -S "V6k2XXXXXXXXXXXXXXXXXXXXXXXiiXX"

Although the Access and Secret Keys are valid, I am getting the following error message
"Unable to locate Credentials"
"Cannot launch XXXXXXXXX.template"

Docker invocation lacks 'taskcat' command execution call

Installed docker version of the taskcat and was trying to invoke the taskcat help but failed with following message

$ sudo taskcat -h
[dockerize]
docker: Error response from daemon: oci runtime error: container_linux.go:262: starting container process caused "exec: \"-h\": executable file not found in $PATH".
ERRO[0000] error waiting for container: context canceled

I noticed that the taskcat binary is calling the container directly with "$@"

$ cat /usr/local/bin/taskcat
echo [dockerize]
docker run -it --rm -v $(pwd):/mnt  taskcat/taskcat $@

But it's missing the taskcat binary call. When I fixed the binary with that call, it now works fine

$ cat /usr/local/bin/taskcat
echo [dockerize]
docker run -it --rm -v $(pwd):/mnt  taskcat/taskcat taskcat $@

utf decode fail when invoked with --public_s3_bucket

Traceback (most recent call last):
File "/var/lib/jenkins/.local/bin/taskcat", line 58, in
main()
File "/var/lib/jenkins/.local/bin/taskcat", line 46, in main
tcat_instance.validate_template(taskcat_cfg, test_list)
File "/var/lib/jenkins/.local/lib/python3.6/site-packages/taskcat/stacker.py", line 723, in validate_template
self.define_tests(taskcat_cfg, test)
File "/var/lib/jenkins/.local/lib/python3.6/site-packages/taskcat/stacker.py", line 1520, in define_tests
if self.check_json(cfntemplate, quite=True, strict=False):
File "/var/lib/jenkins/.local/lib/python3.6/site-packages/taskcat/stacker.py", line 1567, in check_json
parms = json.loads(jsonin.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'
Build step 'Execute shell' marked build as failure
Checking console output
Build did not succeed and the project is configured to only push after a successful build, so no pushing will occur.
Finished: FAILURE

How to debug this case

Hi,

I created a custom template for Launching RDS inside a VPC from quickstart-magneto. When I tried to debug with -v option, however this Error Message is not helping me.

DEBUG  ] :jsonstatus = True
[PASS   ] :Validated [defaults.json]
[INFO   ] :|PREPARING TO LAUNCH => defaults
[INFO   ] :No cleanup value set
[INFO   ] : - (Defaulting to cleanup)
[INFO   ] :|Acquiring tests assets for .......[defaults]
[DEBUG  ] :|S3 Bucket     => [taskcat-test-ababa]
[DEBUG  ] :|Project       => [rds]
[DEBUG  ] :|Template      => [https://s3-ap-southeast-1.amazonaws.com/taskcat-test-ababa/rds/templates/rds.yaml.template]
[DEBUG  ] :|Parameter     => [https://s3-ap-southeast-1.amazonaws.com/taskcat-test-ababa/rds/ci/defaults.json]
[DEBUG  ] :|TemplateType  => [yaml]
[DEBUG  ] :|Defined Regions:
			 - [ap-southeast-1]
[PASS   ] :(Completed) acquisition of [defaults]


[INFO   ] :Preparing to launch in region [ap-southeast-1]
[DEBUG  ] :AutoGen values for $[alfred_genpass_32]
[DEBUG  ] :Auto generating password
[DEBUG  ] :Pass size => 32
[DEBUG  ] :Pass type => default alpha-numeric
[ERROR  ] :expected string or bytes-like object
[FAIL   ] :Cannot launch rds.yaml.template

The template and Parameter file I used are: https://github.com/chrisduong/quickstart-magento/blob/rds-only/templates/rdsmysql-only.yaml.template, https://github.com/chrisduong/quickstart-magento/blob/rds-only/ci/rdsmysql-only.json

What does this error mean? I had tested manually my template in CFN console, it works with the current Parameter JSON file I used.

[ERROR  ] :expected string or bytes-like object

Auto Input Generation

Scan CFN Parameters and build a scaffold for taskcat inputs
Tag Parameters without Defaults value as NEED_VALUES

When STACK_DELETED the report generation does not exit gracefully

[taskcat] |GENERATING REPORTS
[INFO ] :Creating report in [taskcat_outputs]
[INFO ] :(Collecting Resources)
[DEBUG ] :HTTPSConnectionPool(host='cloudformation.us-east-1.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x7f9be57145c0>, 'Connection to cloudformation.us-east-1.amazonaws.com timed out. (connect timeout=60)'))
[FAIL ] :Unable to get resources for stack tCaT-tag-datastax-74

If a stack is manually deleted
Add check in subsequent step to skip the attempt to collect resources

Running taskcat with --public_s3_bucket as the only arg causes exception

jenkins@develop:workspace:[alfred] taskcat --public_s3_bucket

---

| |_ __ _ | | _____ __ | |
| _/ / __| |/ / __/ _ | __|
| || (| _ \ < (| (| | |
__,|/|____,|__|

version 2018.410.184245
Traceback (most recent call last):
File "/var/lib/jenkins/.local/bin/taskcat", line 58, in
main()
File "/var/lib/jenkins/.local/bin/taskcat", line 28, in main
tcat_instance.set_config(args.config_yml)
File "/var/lib/jenkins/.local/lib/python3.6/site-packages/taskcat/stacker.py", line 240, in set_config
if os.path.isfile(config_yml):
File "/usr/lib64/python3.6/genericpath.py", line 30, in isfile
st = os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

Need a way to override upstream CI parameters without git conflicts

Need a way for local parameter overrides.

Edit: see #58 ,

To use:

• define your parameter overrides by adding them to ci/override.json.
• If keys exist in both the original parameter file, and override.json, the values in override.json will win.
• Additional keys present in the override file, but not the original parameters will be added to the parameter list.

Not able to start taskcat on windows

$ curl -s https://raw.githubusercontent.com/aws-quickstart/taskcat/master/installer/docker-install-master| python -E
[INITALIZING taskcat Installer!!]

Ready!! type 'taskcat' to start

nampatra@IBM144-PC0BY4WK MINGW64 ~
$ taskcat
bash: taskcat: command not found

nampatra@IBM144-PC0BY4WK MINGW64 ~
$ ./taskcat
bash: ./taskcat: No such file or directory

I am running on windows OS . Please help out.

Taskcat parameter files URL

Taskcat uses a path-style S3 URL to get and validate parameters file. This should be a virtual-hosted style URL or a URL that matches the bucket region.

Generate Random String

Add KeyGen [$_taskcat_randstring]

Which will generate a random string

Continuously fails with Python exception

It blows up with a Python exception, although AFAIK, i've configured everything correctly. Here's the stack trace:

Traceback (most recent call last):
File "/home/macinna/.local/bin/taskcat", line 58, in
main()
File "/home/macinna/.local/bin/taskcat", line 46, in main
tcat_instance.validate_template(taskcat_cfg, test_list)
File "/home/macinna/.local/lib/python3.5/site-packages/taskcat/stacker.py", line 717, in validate_template
self.define_tests(taskcat_cfg, test)
File "/home/macinna/.local/lib/python3.5/site-packages/taskcat/stacker.py", line 1522, in define_tests
if self.check_json(cfntemplate, quite=True, strict=False):
File "/home/macinna/.local/lib/python3.5/site-packages/taskcat/stacker.py", line 1569, in check_json
parms = json.loads(jsonin)
File "/usr/lib/python3.5/json/init.py", line 312, in loads
s.class.name))
TypeError: the JSON object must be str, not 'bytes'

I noticed that the template file's content-type in S3 is set to binary/octet-stream. So I suspect (although I've not tested it) that this is causing it to get read back by boto as an array of bytes rather than a string.

Bug: Inconsistency between S3 Bucket Name and QSS3BucketName with autobucket enabled

PR forthcoming.

Missing Capabilities [CAPABILITY_NAMED_IAM] option

I was unable to test an IAM roles/Policy CFT due to missing option to add CAPABILITY_NAMED_IAM

Error:
An error occurred (InsufficientCapabilitiesException) when calling the CreateStack operation: Requires capabilities : [CAPABILITY_NAMED_IAM]

Line 919 of taskcat/stacker.py:
self.set_capabilities('CAPABILITY_IAM')
should probably be:
self.set_capabilities('CAPABILITY_NAMED_IAM')

testnames with underscores

if tests are defined with underscores. Stack name will be invalid.
Suggestion: convert all underscores to dashes

issue with multiple python versions

Hi,

I was having issue due to multiple python versions installed. Since taskcat works only with python 2.7 and under my installation pip was installing using python 3. This was breaking the installation.

quick fix:
python3 -m pip install -U --force-reinstall pip
python -m pip install -U --force-reinstall pip

this will ensure your pip uses correct python version for taskcat installation

Fuzzy Matching with QSS3KeyPrefix variable.

[S3: -> ] s3://taskcat-quickstart-linux-bastion-1d3f0a7f/quickstart-linux-bastion.old/templates/linux-bastion-master.template
[S3: -> ] s3://taskcat-quickstart-linux-bastion-1d3f0a7f/quickstart-linux-bastion.old/templates/linux-bastion.template
[S3: -> ] s3://taskcat-quickstart-linux-bastion-1d3f0a7f/quickstart-linux-bastion/LICENSE.txt
[S3: -> ] s3://taskcat-quickstart-linux-bastion-1d3f0a7f/quickstart-linux-bastion/NOTICE.txt

Tracked relevant lines to
https://github.com/aws-quickstart/taskcat/blob/7a589b2b922991311b56c9f746bc6592c2e67512/taskcat/stacker.py#L336-L337

Reference function:
https://github.com/aws-quickstart/taskcat/blob/7a589b2b922991311b56c9f746bc6592c2e67512/taskcat/stacker.py#L98-L115

Looks like L113 is the culprit.

pudb debugging:

>>> pprint(fsmap)
(...)
 './quickstart-linux-bastion/README.md',
 './quickstart-linux-bastion/NOTICE.txt',
 './quickstart-linux-bastion/LICENSE.txt',
 './quickstart-linux-bastion.old/templates/linux-bastion.template',
 './quickstart-linux-bastion.old/templates/linux-bastion-master.template',
 './quickstart-linux-bastion.old/scripts/bastion_bootstrap.sh',
 './quickstart-linux-bastion.old/scripts/banner_message.txt',
 './quickstart-linux-bastion.old/README.md',
 './quickstart-linux-bastion.old/NOTICE.txt',
 './quickstart-linux-bastion.old/LICENSE.txt']

Serverless code testing

Is it possible to use taskcat for testing cloudformation templates that contain serverless code or the transform tag?
My organisation has an application that is launched by nested cloudformation templates that contain serverless code. When I try to test the templates through taskcat it gives me the following error:-
[ERROR ] :An error occurred (ValidationError) when calling the CreateStack operation: CreateStack cannot be used with templates containing Transforms.

If it is possible to test cloudformation templates containing transform tag, how can we achieve it?

Taskcat exits in get_teststate() when describestack fails

[taskcat] |GENERATING REPORTS
[INFO ] :Creating report in [taskcat_outputs]
[INFO ] :Collecting CloudFormation Logs
[INFO ] :Collecting logs for tCaT-tag-quickstart-a-master-7e225ad7"

[ERROR ] : Error trying to get the events for stack [tCaT-tag-quickstart-a-master-7e225ad7] in region [us-west-2 An error occurred (ValidationError) when calling the DescribeStackEvents operation: Stack [tCaT-tag-quickstart-a-master-7e225ad7] does not exist
[ERROR ] :No event logs found. Something went wrong at describe event call.

[INFO ] :(Generating Reports)
[INFO ] : - Processing quickstart-a-master
Traceback (most recent call last):
File "/usr/local/bin/taskcat", line 56, in
main()
File "/usr/local/bin/taskcat", line 50, in main
tcat_instance.createreport(testdata, 'index.html')
File "/usr/local/lib/python3.6/site-packages/taskcat/stacker.py", line 1971, in createreport
self.genreport(testdata_list, dashboard_filename)
File "/usr/local/lib/python3.6/site-packages/taskcat/stacker.py", line 1750, in genreport
state['region'])
File "/usr/local/lib/python3.6/site-packages/taskcat/stacker.py", line 1658, in get_teststate
test_query = cfn.describe_stacks(StackName=stackname)
File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 324, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 622, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id tCaT-tag-quickstart-a-master-7e225ad7 does not exist

CFNYAMLHandler.ordered_safe_dump adds aliases, which cfn does not support

CloudFormation does not support yaml aliases, the current version of the PyYaml library enables these by default.

from taskcat.utils import CFNYAMLHandler
a = [True, False]
print(CFNYAMLHandler.ordered_safe_dump([a,a]))
- &id001 [true, false]
- *id001

No Script examples

Please provide some examples for scripts as currently there are none.

Stacker doesn't validate that key found in metadata explicitly matches project prefix.

The stacker module does fuzzy matching on the prefix from the bucket object metadata. This leads to unexpected results when multiple prefixes match what's in the taskcat config.

More validation is needed to ensure only the keys with the prefix explicitly matching the config variable are used to generate the template URL.

Examples are sanitized, but this is a duplicate of the stacker module, lines 407-412 (get_s3_url function). The only difference is this, for debugging purposes.

foo.append(metdata[1])

/////

>>> for s3obj in contents:
...     for metadata in s3obj.items():
...             if metadata[0] == 'Key':
...                     if key in metadata[1]:
...                             foo.append(metadata[1])
... 
>>> foo
['example.old/LICENSE.txt', 'example/LICENSE.txt']

PR Forthcoming.

Add support for creating IAM resources with custom names

When deploying a cf template which creates IAM resources with custom names we get the following error

Requires capabilities : [CAPABILITY_NAMED_IAM]

The following PR resolves this: #50

On stack fail: SSH into instances and snag logs

There's a huge debug opportunity here. Especially when taskcat is invoked as part of a CI/CD pipeline and instances are long gone before human eyes can look at the results.

Taskcat css missing

Report style is not rendering correctly due to missing CSS file

botocore.exceptions.NoCredentialsError: Unable to locate credentials

I am hitting this error and not able to proceed.
root@ip-172-31-14-55:/usr/local/bin# taskcat -c projectx/ci/CONFIG_YML -A AKIAJBC4OWHO4R3TYPSQ -S Tavlui7b0m2XpyoE+NHRdivqXDA/vhqhT+FkaS/o
[dockerize]

---

| |_ __ _ | | _____ __ | |
| _/ / __| |/ / __/ _ | __|
| || (| _ \ < (| (| | |
__,|/|____,|__|

version 2017.706.143043

[taskcat] :AWS AccountNumber: [299743145002]
[taskcat] :Authenticated via: [role]
[TASKCAT ] :Reading Config form: projectx/ci/CONFIG_YML

[taskcat] :Reading Config form: projectx/ci/CONFIG_YML
[taskcat] |Queing test => taskcat-test1

[taskcat] |CONTENTS OF S3 BUCKET
[INFO ] :Staging Bucket => taskcat-projectx-2b212a28
taskcat-projectx-2b212a28/projectx/ci/CONFIG_YML
taskcat-projectx-2b212a28/projectx/ci/ibm-spectrum-scale-on-aws-quickstart.json
taskcat-projectx-2b212a28/projectx/ci/templates/ibm-spectrum-scale-on-aws-quickstart-master.template

[taskcat] :Validate Template in test[taskcat-test1]
[INFO ] :No cleanup value set
[INFO ] : - (Defaulting to cleanup)
Traceback (most recent call last):
File "/usr/local/bin/taskcat", line 53, in
main()
File "/usr/local/bin/taskcat", line 43, in main
tcat_instance.validate_template(taskcat_cfg, test_list)
File "/usr/lib/python3.5/site-packages/taskcat/stacker.py", line 582, in validate_template
self.define_tests(taskcat_cfg, test)
File "/usr/lib/python3.5/site-packages/taskcat/stacker.py", line 1187, in define_tests
self.get_s3_url(self.get_template_file()))
File "/usr/lib/python3.5/site-packages/taskcat/stacker.py", line 420, in get_s3_url
Bucket=self.get_s3bucket())
File "/usr/lib/python3.5/site-packages/botocore/client.py", line 310, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python3.5/site-packages/botocore/client.py", line 586, in _make_api_call
operation_model, request_dict)
File "/usr/lib/python3.5/site-packages/botocore/endpoint.py", line 141, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/lib/python3.5/site-packages/botocore/endpoint.py", line 166, in _send_request
request = self.create_request(request_dict, operation_model)
File "/usr/lib/python3.5/site-packages/botocore/endpoint.py", line 150, in create_request
operation_name=operation_model.name)
File "/usr/lib/python3.5/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/usr/lib/python3.5/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/usr/lib/python3.5/site-packages/botocore/signers.py", line 90, in handler
return self.sign(operation_name, request)
File "/usr/lib/python3.5/site-packages/botocore/signers.py", line 154, in sign
auth.add_auth(request)
File "/usr/lib/python3.5/site-packages/botocore/auth.py", line 408, in add_auth
super(S3SigV4Auth, self).add_auth(request)
File "/usr/lib/python3.5/site-packages/botocore/auth.py", line 340, in add_auth
raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials
root@ip-172-31-14-55:/usr/local/bin#

I configured aws as well but still I am getting the above error.
root@ip-172-31-14-55:/usr/local/bin# aws configure list
Name Value Type Location
---- ----- ---- --------
profile None None
access_key ****************YPSQ shared-credentials-file
secret_key ****************aS/o shared-credentials-file
region us-west-2 config-file ~/.aws/config
root@ip-172-31-14-55:/usr/local/bin#

Adding testcase scenario examples in documentation

This is more of a request than an issue.

From the documentation I could not figure out how to write sample testcases for my CFN template in config.yml
And what exactly can I test with this utility ?

Can you please point me to some examples ?

Remove version check in development container

Since docker development container, (taskcat/taskcat-develop) is using git hub source.
We need to remove the version check