Giter Site home page Giter Site logo

aws-cloudformation / custom-resource-helper Goto Github PK

View Code? Open in Web Editor NEW
367.0 22.0 56.0 103 KB

Simplify best practice Custom Resource creation, sending responses to CloudFormation and providing exception, timeout trapping, and detailed configurable logging.

License: Apache License 2.0

Python 100.00%

custom-resource-helper's Introduction

Custom Resource Helper

Simplify best practice Custom Resource creation, sending responses to CloudFormation and providing exception, timeout trapping, and detailed configurable logging.

PyPI Version Python Versions Build Status Test Coverage

Features

  • Dead simple to use, reduces the complexity of writing a CloudFormation custom resource
  • Guarantees that CloudFormation will get a response even if an exception is raised
  • Returns meaningful errors to CloudFormation Stack events in the case of a failure
  • Polling enables run times longer than the lambda 15 minute limit
  • JSON logging that includes request id's, stack id's and request type to assist in tracing logs relevant to a particular CloudFormation event
  • Catches function timeouts and sends CloudFormation a failure response
  • Static typing (mypy) compatible

Installation

Install into the root folder of your lambda function

cd my-lambda-function/
pip install crhelper -t .

Example Usage

This blog covers usage in more detail.

from __future__ import print_function
from crhelper import CfnResource
import logging

logger = logging.getLogger(__name__)
# Initialise the helper, all inputs are optional, this example shows the defaults
helper = CfnResource(json_logging=False, log_level='DEBUG', boto_level='CRITICAL', sleep_on_delete=120, ssl_verify=None)

try:
    ## Init code goes here
    pass
except Exception as e:
    helper.init_failure(e)


@helper.create
def create(event, context):
    logger.info("Got Create")
    # Optionally return an ID that will be used for the resource PhysicalResourceId, 
    # if None is returned an ID will be generated. If a poll_create function is defined 
    # return value is placed into the poll event as event['CrHelperData']['PhysicalResourceId']
    #
    # To add response data update the helper.Data dict
    # If poll is enabled data is placed into poll event as event['CrHelperData']
    helper.Data.update({"test": "testdata"})

    # To return an error to cloudformation you raise an exception:
    if not helper.Data.get("test"):
        raise ValueError("this error will show in the cloudformation events log and console.")
    
    return "MyResourceId"


@helper.update
def update(event, context):
    logger.info("Got Update")
    # If the update resulted in a new resource being created, return an id for the new resource. 
    # CloudFormation will send a delete event with the old id when stack update completes


@helper.delete
def delete(event, context):
    logger.info("Got Delete")
    # Delete never returns anything. Should not fail if the underlying resources are already deleted.
    # Desired state.


@helper.poll_create
def poll_create(event, context):
    logger.info("Got create poll")
    # Return a resource id or True to indicate that creation is complete. if True is returned an id 
    # will be generated
    return True


def handler(event, context):
    helper(event, context)

Polling

If you need longer than the max runtime of 15 minutes, you can enable polling by adding additional decorators for poll_create, poll_update or poll_delete. When a poll function is defined for create/update/delete the function will not send a response to CloudFormation and instead a CloudWatch Events schedule will be created to re-invoke the lambda function every 2 minutes. When the function is invoked the matching @helper.poll_ function will be called, logic to check for completion should go here, if the function returns None then the schedule will run again in 2 minutes. Once complete either return a PhysicalResourceID or True to have one generated. The schedule will be deleted and a response sent back to CloudFormation. If you use polling the following additional IAM policy must be attached to the function's IAM role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "lambda:AddPermission",
        "lambda:RemovePermission",
        "events:PutRule",
        "events:DeleteRule",
        "events:PutTargets",
        "events:RemoveTargets"
      ],
      "Resource": "*"
    }
  ]
}

Certificate Verification

To turn off certification verification, or to use a custom CA bundle path for the underlying boto3 clients used by this library, override the ssl_verify argument with the appropriate values. These can be either:

  • False - do not validate SSL certificates. SSL will still be used, but SSL certificates will not be verified.
  • path/to/cert/bundle.pem - A filename of the CA cert bundle to uses. You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.

Use CDK to depoy a Custom Resource that uses Custom Resource Helper

You can use the AWS Cloud Development Kit (AWS CDK) to deploy a Custom Resource that uses Custom Resource Helper. AWS CDK is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation.

Note: crhelper is not intended to be used with AWS CDK using the Provider construct.

AWS CDK template example

from aws_cdk import (
    ...
    aws_lambda as _lambda,
    CustomResource,
)

crhelperSumResource = _lambda.Function(...)

customResource = CustomResource(
  self, 
  'MyCustomResource'
  serviceToken = crhelperSumResource.function_arn,
  properties = {
    'No1': 1,
    'No2': 2
  },
)


Credits

Decorator implementation inspired by https://github.com/ryansb/cfn-wrapper-python

Log implementation inspired by https://gitlab.com/hadrien/aws_lambda_logging

License

This library is licensed under the Apache 2.0 License.

custom-resource-helper's People

Contributors

armaseg avatar asantos-fuze avatar carpnick avatar danielnovotny-wf avatar demartinofra avatar gene1wood avatar gruebel avatar harrywhite4 avatar hyandell avatar jaymccon avatar jaymecd avatar jonemo avatar mdaehnert avatar robreus avatar rstevens011 avatar shakirjames avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

custom-resource-helper's Issues

Issues to Rollback when the KMS Alias already exists

we are creating KMS Keys via a custom cloudformation stack and noticing the following functionality when a user tries to create a KMS Key with an alias that already exists:

  1. Cloudformation stack containing a 'name' param of an existing KMS Alias is created.
  2. Cloudformation sends a create event which raises an exception due to a KMS Key already existing with the 'name'.
  3. Due to the custom-resource-helper returning a PhysicalResourceId that is auto generated (on a failure) Cloudformation tries to rollback and sends a 'delete' event.
  4. Due to the PhysicalResourceId not actually being a real resource the delete event fails and the stack is stuck in a ROLLBACK_FAILED state.

I reckon the issue is the same than in #7, there was a branch with a solution, but it was closed.

`log_level` type should be `Union[str, int]` and not just `str`

Hello there,
First of all: thanks a lot for the great library. It has sensibly improved our life when writing custom resources.

I have found a small inconsistency in the type hints for the CfnResource's log_level constructor parameter.

Current behavior

Version affected: main branch f2095c5

CfnResource constructor specifies that log_level should be a string.
In practice though, the log_level is used in log_helper.py to set the log level on the root logger in the stdlib logging module.

    logging.root.setLevel(level)

The logging module supports both string and integers to specify the log level. As a matter of fact, the log level is stored internally as an integer. The string is translated to an int using the _checkLevel method:

def _checkLevel(level):
    if isinstance(level, int):
        rv = level
    elif str(level) == level:
        if level not in _nameToLevel:
            raise ValueError("Unknown level: %r" % level)
        rv = _nameToLevel[level]
    else:
        raise TypeError("Level not an integer or a valid string: %r" % level)
    return rv

Expected behavior

log_level type hint should support both string and int.

Contribution

I am more than happy to create a PR to fix this, provided that you think it should be fixed.

[Docs] clarify helper.Data for update

If I want to return properties after creation (helper.Data.update({})), what do I have to do when updating?

If an update does not change the returned properties, can I just not touch helper.Data? Or do I need to fetch the data from somewhere else and pass it in again?

If an update does change the returned properties, but not all of them, do I only need to pass the new ones to helper.Data.update({})?

"Connection timed out" when deleting the stack

Hi,

I use CDK and custom resource to create a database schema based on a python 3.8 lambda.

I am able to create the stack. I am having issues when I delete the stack.

When I delete the stack, it gets stuck on deleting the custom resource.

In Cloud Watch, I am getting the following error:

Unexpected failure sending response to CloudFormation [Errno 110] Connection timed out

When I run the same curl command manually from my linux machine, I am able to delete it.

Any hints?

Kind Regards,
Nicola

No PhysicalResourceId is generated when FAILED by init_failure()

When init_failure() function is used to send a failure to Cloudformation, it doesn't generate PhysicalResourceId. Hence, Cloudformation cannot provide a meaningful error message for the failed resource. Instead, it simply says- Invalid PhysicalResourceId.

In Cloudwatch log it shows-

[DEBUG] 2022-08-30T02:26:04.505Z b9753bbc-aa02-4cdd-9ee8-d94cf5d3f54e {"Status": "FAILED", "PhysicalResourceId": "", "StackId": "arn:aws:cloudformation:us-west-2:123456789012:stack/stack-name/guid", "RequestId": "123", "LogicalResourceId": "MyTestResource", "Reason": "An error occurred (InvalidParameter) when calling the RouteTableId operation: Must provide a RouteTableId", "Data": {}, "NoEcho": false}

My code snippet:

helper = CfnResource()

def lambda_handler(event, context):
    if some_condition is True:
        helper.init_failure(Exception("Test error"))

    helper(event, context)

I tried adding helper.PhysicalResourceId = "SomeID" before the helper() function. But it doesn't make any difference.

I guess this line (https://github.com/aws-cloudformation/custom-resource-helper/blob/main/crhelper/resource_helper.py#L121) is resetting it before it sends the response in this line (https://github.com/aws-cloudformation/custom-resource-helper/blob/main/crhelper/resource_helper.py#L135)

Lambda seems to fail writting back to the pre-signed URL with a permissions error.

I have really simple lambda that I am using to test this, most of which from the repo. When I call this with the simple CFT I get the following error.

[ERROR] 2019-07-23T23:18:38.572Z 53eb4c5b-fc49-4154-952e-a7f7b75bc1fe An error occurred (AccessDeniedException) when calling the PutRule operation:

Lambda
`from crhelper import CfnResource
import logging

#Intiate Logger
logger = logging.getLogger(name)

Initialise the helper, all inputs are optional, this example shows the defaults

helper = CfnResource(json_logging=False, log_level='DEBUG', boto_level='DEBUG')

#Define Netskope Vars
try:
myData= "123456789"
pass
except Exception as e:
helper.init_failure(e)

@helper.create
def create(event, context):
logger.info("Got Create")

helper.Data.update({"myData": myData})
return True

@helper.delete
def delete(event, context):
logger.info("Got Delete")
# Delete never returns anything. Should not fail if the underlying resources are already deleted. Desired state.

@helper.poll_create
def poll_create(event, context):
logger.info("Got create poll")
# Return a resource id or True to indicate that creation is complete. if True is returned an id will be generated
return True

def lambda_handler(event, context):
helper(event, context)`

CFT
--- AWSTemplateFormatVersion: '2010-09-09' Parameters: {} Resources: MyCustom: Type: Custom::Test Properties: ServiceToken: arn:aws:lambda:...

Handling multiple resource types in a single function

With this library and its decorators, is it possible to map the received events to multiple functions depending on their ResourceTypes?

For instance. I imagine multiple function definitions annotated with @helper.create and make CfnResource can decide which one to run depending on the resource type.

is it possible this way or in some other manner?

Handling incorrect parameters

hi, thanks for the project!

It's not clear to me how to handle raising errors during create/update when user provided properties are incorrect. Do I raise an exception? Or do I set helper.Status to FAILED myself?

poll_delete not working

I have a custom resource that will delete certain s3 files when the cloudformation template is deleted. In some cases, there are hundreds of thousands of files, so the 15 minute timeout will apply. I'm trying to use the poll_delete decorator to check if there are still files remaining to rerun the function. If the function takes longer then 15 min, it doesn't get called again and the CF delete fails. Am I doing something wrong?

from crhelper import CfnResource
import boto3

helper = CfnResource()

s3_client = boto3.client('s3')


@helper.delete
def delete_customer_data(event, _):
    bucket = event['ResourceProperties']['S3Bucket']
    prefix = event['ResourceProperties']['AccountCode'] + '/'

    while True:
        objects = s3_client.list_objects_v2(
            Bucket=bucket,
            Prefix=prefix
        )

        if 'Contents' not in objects.keys():
            break

        object_keys = []
        for object in objects['Contents']:
            object_keys.append({'Key': object['Key']})

        s3_client.delete_objects(
            Bucket=bucket,
            Delete={
                'Objects': object_keys
            }
        )


@helper.poll_delete
def poll_delete(event, context):
    bucket = event['ResourceProperties']['S3Bucket']
    prefix = event['ResourceProperties']['AccountCode'] + '/'

    objects = s3_client.list_objects_v2(
        Bucket=bucket,
        Prefix=prefix
    )

    if "Contents" in objects.keys():  # If there are any contents in the objects var, run the function again
        return None  # If the function should run again, return None
    else:
        return True  # If the function has completed, return True


@helper.poll_create
@helper.poll_update
def return_true(event, context):
    return True


@helper.update
@helper.create
def no_op(_, __):
    pass


def handler(event, context):
    helper(event, context)

VPC lambda timeout failure

Hi,

The behavior for a in-VPC Lambda function with no internet access lacks error messages and timeout.

  • there's no timeout, it should try (eventually retry) and fail with an error message, here:

  • It should be clear (documentation and error message) that the Lambda function needs access to S3 to report the status, by either full internet access or by having an S3 endpoint in the VPC

Bastien(AWS)

Validate and limit names for PutRule

I noticed for resources with really long name, appending 8 random characters would make the PutRule name too long, causing an exception like the following:

An error occurred (ValidationException) when calling the PutRule operation: 1 validation error detected: Value '<SomeLongNameOver64Characters>' at 'name' failed to satisfy constraint: Member must have length less than or equal to 64

It may be better to truncate the Name string below if it has more than 56 characters, and then append the 8 random characters to ensure the name is less than 64 characters:

Name=self._event['LogicalResourceId'] + self._rand_string(8),

Creating static type checks for mypy

Hi there,

I recently created a custom lambda resource where I used mypy for static type checking. This tool helps a lot in maintaining clean code during bigger python projects.
mypy also checks imports if calls are used right. Right now it stops with the error message.

src\app\index.py:12: error: Cannot find implementation or library stub for module named 'crhelper'

Workaround is to add an ignore comment for mypy:

from crhelper import CfnResource # type: ignore

I figured out how to easily it is to create stubs (type hints) for this project to be fully mypy compatible.

In a nutshell:

  • Install mypy
  • run stubgen ./ - it created *.pyi files
  • add empty py.typed file to this repository

If you're interested in supporting typed projects as well, then I can propose a PR with the solution mentioned above.

Best regards
Michael

Receiving Error AttributeError: 'dict' object has no attribute 'get_remaining_time_in_millis'

As I run my custom resource, it is successful but I receive a failure any way with

[ERROR] 2024-05-01T14:16:41.163Z 9407e51d-128f-4e7e-8295-24ef4980e717 'dict' object has no attribute 'get_remaining_time_in_millis'
Traceback (most recent call last):
File "/var/task/crhelper/resource_helper.py", line 76, in call
if not self._crhelper_init(event, context):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/crhelper/resource_helper.py", line 137, in _crhelper_init
self._set_timeout()
File "/var/task/crhelper/resource_helper.py", line 215, in _set_timeout
self._timer = threading.Timer((self._context.get_remaining_time_in_millis() / 1000.00) - 0.5,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'get_remaining_time_in_millis'

I am using python 3.12

Always get "Unable to marshal response" Error?

I have followed the example to write some codes that create custom resources in Lambda for Cloudformation. However, it seems that it always trigger "Unable to marshal response" Error:

[ERROR] Runtime.MarshalError: Unable to marshal response: Object of type set is not JSON serializable
Traceback (most recent call last):

I cannot searched anyone else meeting this issue. I am still not sure whether it was caused by my setup or not. For some cases, I can simply ignore this. But for some other cases, it seems to trigger the lambda to retry a few times. I know that my functions should be idempotent but sometimes it may not be so practical. My current workaround is to set MaximumRetryAttempts to zero but I don't think it is a good solution.

How to handle when resource fail to update?

Hi, I'm currently have a update stack as follow

@helper.update
def update(event, context):
    try:
        physical_id = event['PhysicalResourceId']
        request_type = event['RequestType']
        properties = event['ResourceProperties']
        old_properties = event['OldResourceProperties']

        # do something here
       create_repository()
       ....
       set_default_reviewer()       <------ this is where it fail when I input invalid user
       # end

        return physical_id

    except Exception as err:
        helper.Status = 'FAILED'
        helper.Reason = err
        return physical_id

However when the stack fail, it stuck in the UPDATE_ROLLBACK_FAILED state, I want the stack to be in UPDATE_ROLLBACK_COMPLETE. Am i doing wrong? Please help me

Thanks

CloudFormation returned status code: Forbidden

When does cloud formation returns Forbidden status? This happened with me during create. I had a code where a the Lambda was supposed to upload file to S3 bucket and while that went successful the template got stuck!

Here is what the log looks like

[DEBUG]	2020-07-21T15:02:51.816Z	8f3cc89a-9521-4272-82cc-587d7e8a0d4b	Executing task PutObjectTask(transfer_id=0, {'bucket': 'my_bucket_name', 'key': 'faq_document.xlsx', 'extra_args': {}}) with kwargs {'client': <botocore.client.S3 object at 0x7fca11a7e220>, 'fileobj': <s3transfer.utils.ReadFileChunk object at 0x7fca10db13a0>, 'bucket': 'my_bucket_name', 'key': 'faq_document.xlsx', 'extra_args': {}}
[DEBUG]	2020-07-21T15:03:23.593Z	8f3cc89a-9521-4272-82cc-587d7e8a0d4b	Releasing acquire 0/None
[INFO] 2020-07-21T15:03:23.594Z 8f3cc89a-9521-4272-82cc-587d7e8a0d4b File `faq_document.xlsx` uploaded successfully
[DEBUG]	2020-07-21T15:03:23.597Z	8f3cc89a-9521-4272-82cc-587d7e8a0d4b	enabling send_response
[DEBUG]	2020-07-21T15:03:23.616Z	8f3cc89a-9521-4272-82cc-587d7e8a0d4b	_send_response: True
[DEBUG] 2020-07-21T15:03:23.617Z 8f3cc89a-9521-4272-82cc-587d7e8a0d4b CFN response URL: https://cloudformation-custom-resource-response-my_region.s3.amazonaws.com ..........
[DEBUG]	2020-07-21T15:03:23.617Z	8f3cc89a-9521-4272-82cc-587d7e8a0d4b	
{
    "Status": "SUCCESS",
    "PhysicalResourceId": "S3PostAutoID000",
    "StackId": "arn:aws:cloudformation:my_region:my_account:stack/stack-cb3/f0da78d0-cb60-11ea-bbd9-0e3259ee6b3b",
    "RequestId": "0bed6b99-a998-49c0-933d-81386cad65be",
    "LogicalResourceId": "CustomS3Postrequisites",
    "Reason": "",
    "Data": {}
}
[INFO] 2020-07-21T15:03:23.843Z 8f3cc89a-9521-4272-82cc-587d7e8a0d4b CloudFormation returned status code: Forbidden
END RequestId: 8f3cc89a-9521-4272-82cc-587d7e8a0d4b

Its strange though never happened before, although one question, Am I suppose to randomize my PhysicalResourceId?, because right now its plain text

@helper.create
def create(event, _):
    try:
        bucket_name = event['ResourceProperties']['BucketName']
        s3 = boto3.resource('s3')
        s3.meta.client.upload_file('faq_document.xlsx', bucket_name, 'faq_document.xlsx')
        logger.info("File `faq_document.xlsx` uploaded successfully")
    except Exception as _:
        logger.critical(traceback.format_exc())

    return "S3PostAutoID000"

Add warning that this framework is incompatible with the AWS CDK custom resource framework

Hi there,

Would it be possible to add a warning to the README for confused folks like me saying that crhelper is NOT to be used with AWS CDK custom resources?

I wasted several hours today debugging a custom resource created using the aws_cdk.custom_resources.Provider framework. The lambda handler in question was using the crhelper framework, but I learned that crhelper was not intended to be used with AWS CDK in this manner.

Two incompatibilities I discovered with crhelper and the CDK custom resource framework:

  • crhelper wraps your code with a global try/except block. This means that your lambda handler cannot raise exceptions that cause the handler to exit. This is problematic with the CDK framework because raising such exceptions is the only mechanism for causing a resource to fail creation, deletion, etc. with the CDK framework.

  • crhelper emits an HTTP request to inform CloudFormation of the payload with attributes such as Data, PhysicalResourceId, etc. This, also, is not compatible with the CDK framework--which instead requires these values to be the returned values of the lambda handler.

Since I struggled with this, I wonder if others will too.

Several mypy errors when type-checking code that imports crhelper

Error 1: missing library stubs or py.typed marker

When type checking projects that depend on crhelper with default and/or strict mypy settings, the following error results in a type checking failure:

Skipping analyzing "crhelper": module is installed, but missing library stubs or py.typed marker

To reproduce:

  1. Install crhelper as dependency for a project
  2. Use the code from this repo's Readme file
  3. Run mypy my-project-directory to type check the project

On second glance, this is weird because there is a py.typed marker file in the repo root and there are .pyi files next to the code. However, neither py.typed nor .pyi files get bundled into the Python package.

Related information:

PEP 561: https://www.python.org/dev/peps/pep-0561/
mypy docs about the error: https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-library-stubs-or-py-typed-marker

Error 2: Unexpected keyword argument "ssl_verify" for "CfnResource"

This line of code from the Readme:

helper = CfnResource(json_logging=False, log_level='DEBUG', boto_level='CRITICAL', sleep_on_delete=120, ssl_verify=None)

results in this mypy error:

Unexpected keyword argument "ssl_verify" for "CfnResource"

This is because the function signature of CfnResource.__init__ in [log_helper.py](https://github.com/aws-cloudformation/custom-resource-helper/blob/main/crhelper/log_helper.py) doesn't match the corresponding function signature in log_helper.pyi

Error 3: Function is untyped after decorator transformation

Any use of the decorators @helper.create, @helper.update, ... results in this mypy error:

Function is untyped after decorator transformation

This is because the type annotation for the decorators only says that the decorator wraps a function, but omits the wrapped function's signature:

def create(self, func: Any): ...

Phsyical ID as argument to delete?

I'm trying to create a custom resource for a physical resource which has a non-deterministic ID/ARN.

i.e. when I receive the delete call, I cannot figure out which resource to delete based on the properties of the resource. I instead need to get whatever value was returned by create.

I'm hoping this value is somewhere inside the event. If so, can you please document where in the event it is?

Support for SNS Custom Resource Handling

I'd like to use this handler with an SNS triggered custom resource, to handle a multi account scenario rather than to lambda in the local account.

In theory the sns message content delivered to lambda in a remote account could be handled in the same way as this but would have been through custom resource in CF > SNS Remote > Lambda i.e rather than dealing with the event directly, something like json.loads(event['Records'][0]['Sns']['Message'])? It would be great to be able to use this handler in that scenario.

Make waiting for CloudWatch logs propogation configurable

Thanks for the great helper.

Why is resource deletion so slow? Is that a CloudFormation thing or this tool's peculiarity? I can see in CloudWatch that a deletion of the resource being removed has been successfully completed a couple of minutes ago and there is nothing else in the changeset yet the stack update is in the UPDATE_COMPLETE_CLEANUP_IN_PROGRESS state for a few minutes. I am not using the poll option.

Import Requests Directly instead of using Vendored

Would you consider adding requests to your requirements instead of importing from botocore?

The following Warning has started to pop up when invoking crhelper

/var/task/botocore/vendored/requests/api.py:67: DeprecationWarning: You are using the put() function from 'botocore.vendored.requests'. This is not a public API in botocore and will be removed in the future. Additionally, this version of requests is out of date. We recommend you install the requests package, 'import requests' directly, and use the requests.put() function instead. DeprecationWarning

Poller persistence

Hi,

I'm trying to create a Custom resource to create an account inside AWS Organization.
This action is asynchronous because you have to call two different boto3 method: first you need to call the CreateAccount API, this provide you a request id which needs to be used polling the DescribeCreateAccountStatus API until the create status is SUCCEEDED.

I would like to persist the RequestId putting it in the ChHelperData, but calling this is not enought
helper.Data.update({"CreateAccountRequestId": create_account_request_id})
I don't find anything inside the event inside the ChHelperData during the next invocation.

Here the code I'm developing:

@helper.poll_create
def create_account(event, _):
    organizations_client = boto3.client('organizations')
    create_account_request_id = helper.Data.get("CreateAccountRequestId")
    if not create_account_request_id:
        account_name = event.get("ResourceProperties", {}).get('AccountName')
        if not account_name:
            raise ValueError("AccountName is not specified")
        account_email = f"{account_name}@{INFO_DOMAIN}"
        response = organizations_client.create_account(
            Email=account_email,
            AccountName=account_name,
        )
        create_account_request_id = response['CreateAccountStatus']['Id']
        if not create_account_request_id:
            print(response)
            raise ValueError("CreateAccountRequestId not found")
        helper.Data.update({"CreateAccountRequestId": create_account_request_id})
    else:
        response = organizations_client.describe_create_account_status(
            CreateAccountRequestId=create_account_request_id,
        )
        status = response['CreateAccountStatus']['State']
        print(f"Account creation status: {status}")
        if status == 'FAILED':
            raise RuntimeError(response['CreateAccountStatus']['FailureReason'])
        if status == 'SUCCEEDED':
            account_id = response['CreateAccountStatus']['AccountId']
            helper.Data.update({
                "AccountId": account_id,
                "AccessRole": f"arn:aws:iam::{account_id}:role/OrganizationAccountAccessRole"
            })
            return account_id

def _put_targets multi partition support

I noticed an issue when deploying a custom resources in the govcloud region. It appears that line 271 has the aws partition hard coded to aws. The acceptable values for partition are:

"aws" - Public AWS partition
"aws-cn" - AWS China
"aws-us-gov" - AWS GovCloud

This request is to make this function aware of the current partition, instead of being hard coded to the default partition.

def _put_targets(self, func_name):
        region = self._event['CrHelperRule'].split(":")[3]
        account_id = self._event['CrHelperRule'].split(":")[4]
        rule_name = self._event['CrHelperRule'].split("/")[1]
        logger.debug(self._event)
        self._events_client.put_targets(
            Rule=rule_name,
            Targets=[
                {
                    'Id': '1',
                    'Arn': 'arn:aws:lambda:%s:%s:function:%s' % (region, account_id, func_name),
                    'Input': json.dumps(self._event)
                }
            ]
        )

PhysicalResourceId generated on failures

I am having an issue with the way PhysicalResourceId is generated.

I am currently using crhelper within a lambda written to implement a custom resource that use the boto3 SDK to create an ECS service. The problem I have encountered is that when create fails, like for instance when a parameter is missing or invalid, the function _cfn_response sees that there is no PhysicalResourceId and decides to generate one before returning the failure to Cloudformation.

When Cloudformation sees the response, it decides:

  • To rollback the change (since it encountered an error)
  • To invoke delete passing the generated PhysicalResourceId which also fails since PhysicalResourceId is not a real one, unless I add some extra code to detect and ignore failures when attempting to delete the ECS service (which is not great/ideal).

Would it be possible to not generated a PhysicalResourceId if there is none, on failures?

How to know current request is ROLLBACK type?

Hi,

I have some code that need to behave different when the request is update and rollback. But according to AWS document, the rollback request is just a update one with new properties and old one in reverse, there's nothing to distinguish between those two. Anyone have any idea to archive it?

Thanks.

CfnResource() should use default value ssl_verify=None

Hi,

I noticed that in 2.0.8 version, CfnResource() uses ssl_verify=True as the default value and feed it into boto3.client(verify=ssl_verify) :
https://github.com/aws-cloudformation/custom-resource-helper/blob/main/crhelper/resource_helper.py#L30

However, the default value in boto3 client is actually verify=None . And there's actually difference regarding how boto3 interprets these values. Based on my experiment, the boto3.client verify param values are:

  1. verify=None: (Default) will do ssl verify, using default CA bundle, or the one from AWS_CA_BUNDLE environment variable if specified.
  2. verify=True : will do ssl verify, using default CA bundle, ignoring AWS_CA_BUNDLE env var.
  3. verify=False: will not do ssl verify.
  4. verify=/path/to/ca_bundle: will do ssl verify, using the path in this param, ignoring AWS_CA_BUNDLE env var.

With the current implementation in crhelper, if I want to use the CA bundle from the AWS_CA_BUNDLE env var, I have to explicitly call `CfnResource(ssl_verify=None) which is awkward. That's why I suggest crhelper to change the default value of ssl_verify to None to match the boto3 default value. Thanks!

How to test Lambdas using crhelper

Sorry for this stupid question...

How do I run tests? When using cfnresponse I could simply create a standard Lambda Cloudformation TestEvent and somewhat test my lambda (besides the cfnresponse failing in the end).

When trying the same with a Lambda using crhelper I run into loads of errors as I do not want to specify a CFN Stack to send the response to. I simply want to run my code and not communicate with CFN-Stacks (as there are none so far).

Here is my sample test event which returns this error. Basically it's a createSnapshot for elastiCache, however I don't think this is relevant. The final product would be a CFN Template accessible via ServiceCatalog where customers enter their ReplicationGroupId to create a snapshot.
" socket.gaierror: [Errno -2] Name or service not known[ERROR] Unexpected failure sending response to CloudFormation [Errno -2] Name or service not known "

{ "RequestType": "Create", "ResponseURL": "http://pre-signed-S3-url-for-response", "StackId": "arn:aws:cloudformation:eu-central-1:123456789012:stack/MyStack/guid", "RequestId": "unique id for this create request", "ResourceType": "Custom::TestResource", "LogicalResourceId": "MyTestResource", "ResourceProperties": { "StackName": "MyStack", "ApplicationName": "abcd", "SnapshotMode": "Create", "CacheClusterId": "", "ReplicationGroupId": "abcd-ttwoshards", "CostReference": "abc", "KmsKeyId": "abc", "SnapshotName": "mySnapshotviaLambda" } }

Thanks you for helping a newb out. If this isn't the right place to ask this question please let me know the right place to ask, or link a me docs explaining what I am tryin to achieve.

Is JSON logging broken?

I am trying to use this helper to create a custom Transform for my templates. I started small by just using the example boilerplate:

from crhelper import CfnResource
import logging

logger = logging.getLogger(__name__)
helper = CfnResource(json_logging=True, log_level='DEBUG', boto_level='CRITICAL')

try:
    pass
except Exception as e:
    helper.init_failure(e)


@helper.create
def create(event, context):
    logger.info("Got Create")
    logger.debug(event)
    logger.debug(context)
    helper.Data.update({"test": "testdata"})
    return "MyResourceId"


@helper.update
def update(event, context):
    logger.info("Got Update")


@helper.delete
def delete(event, context):
    logger.info("Got Delete")



def handler(event, context):
    helper(event, context)

Then I deployed it as a macro and when I trigger this macro by using (Strings is the name of the Transform macro):

AWSTemplateFormatVersion: 2010-09-09
Transform: Strings
Description: Test template for strings

I am getting the following errors in CloudWatch:

{
    "timestamp": "2019-04-20 07:29:04,182",
    "level": "ERROR",
    "location": "crhelper.resource_helper.__call__:87",
    "RequestType": "ContainerInit",
    "message": "'RequestType'",
    "exception": "Traceback (most recent call last):\n  File \"/var/task/crhelper/resource_helper.py\", line 69, in __call__\n    self._log_setup(event, context)\n  File \"/var/task/crhelper/resource_helper.py\", line 102, in _log_setup\n    log_helper.setup(self._log_level, boto_level=self._boto_level, RequestType=event['RequestType'],\nKeyError: 'RequestType'"
}
{
    "timestamp": "2019-04-20 07:29:04,183",
    "level": "DEBUG",
    "location": "crhelper.utils._send_response:18",
    "RequestType": "ContainerInit",
    "message": "CFN response URL: "
}
{
    "timestamp": "2019-04-20 07:29:04,183",
    "level": "DEBUG",
    "location": "crhelper.utils._send_response:19",
    "RequestType": "ContainerInit",
    "message": {
        "Status": "FAILED",
        "PhysicalResourceId": "",
        "StackId": "",
        "RequestId": "",
        "LogicalResourceId": "",
        "Reason": "'RequestType'",
        "Data": {}
    }
}
{
    "timestamp": "2019-04-20 07:29:04,203",
    "level": "ERROR",
    "location": "crhelper.utils._send_response:27",
    "RequestType": "ContainerInit",
    "message": "Unexpected failure sending response to CloudFormation Invalid URL '': No schema supplied. Perhaps you meant http://?",
    "exception": "Traceback (most recent call last):\n  File \"/var/task/crhelper/resource_helper.py\", line 69, in __call__\n    self._log_setup(event, context)\n  File \"/var/task/crhelper/resource_helper.py\", line 102, in _log_setup\n    log_helper.setup(self._log_level, boto_level=self._boto_level, RequestType=event['RequestType'],\nKeyError: 'RequestType'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/var/task/crhelper/utils.py\", line 23, in _send_response\n    response = put(response_url, data=json_response_body, headers=headers)\n  File \"/var/runtime/botocore/vendored/requests/api.py\", line 122, in put\n    return request('put', url, data=data, **kwargs)\n  File \"/var/runtime/botocore/vendored/requests/api.py\", line 50, in request\n    response = session.request(method=method, url=url, **kwargs)\n  File \"/var/runtime/botocore/vendored/requests/sessions.py\", line 451, in request\n    prep = self.prepare_request(req)\n  File \"/var/runtime/botocore/vendored/requests/sessions.py\", line 382, in prepare_request\n    hooks=merge_hooks(request.hooks, self.hooks),\n  File \"/var/runtime/botocore/vendored/requests/models.py\", line 304, in prepare\n    self.prepare_url(url, params)\n  File \"/var/runtime/botocore/vendored/requests/models.py\", line 362, in prepare_url\n    to_native_string(url, 'utf8')))\nbotocore.vendored.requests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps you meant http://?"
}

I did not even start to add my logic yet. What am I doing wrong?

Lambda sends Delete SUCCESS before complete

I don't see anyway to modify this, but it seems the the delete function sends "Status":"SUCCESS" before actually completing any of the items in the delete function?

Is this expected behavior?

I have issue because when CloudFormation gets this success it moves on to delete the lambda function and so the delete function never does anything. Is there anyway to change this?

Send a Reason for success results

Currently it seems the Reason portion of the response object payload can only be controlled in failure scenarios.

I'd like to send a Reason message for successful creates / updates in order to make it easier for system operators to understand the impact of the action taken. For me, I am implementing a schema migrator so I'd like to output that "3 migrations have executed successfully".

Adding Outputs to templates creating Custom Resources.

Can anyone share an example of creating outputs in crhelper. I have a function which creates a directory connector, and I need to know how to return the DirectoryId to the stack as an output. Digging through the code it looks like i need to add it to CrHelperData somehow.

import boto3
import os
import logging
from crhelper import CfnResource
from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
# Initialise the helper, all inputs are optional, this example shows the defaults
helper = CfnResource(json_logging=False, log_level='INFO', boto_level='CRITICAL')

try:
    ## Init code goes here
    pass
except Exception as e:
    helper.init_failure(e)


@helper.create
def create(event, context):
    logger.info("Got Create")
    logger.info(event)
    # Optionally return an ID that will be used for the resource PhysicalResourceId, 
    # if None is returned an ID will be generated. If a poll_create function is defined 
    # return value is placed into the poll event as event['CrHelperData']['PhysicalResourceId']
    #
    # To add response data update the helper.Data dict
    # If poll is enabled data is placed into poll event as event['CrHelperData']
    client = boto3.client('ds')
    response = client.connect_directory(
        Name=event['ResourceProperties']['Name'],
        ShortName=event['ResourceProperties']['ShortName'],
        Password=event['ResourceProperties']['Password'],
        Description=event['ResourceProperties']['Description'],
        Size=event['ResourceProperties']['Size'],
        ConnectSettings={
            'VpcId': event['ResourceProperties']['VpcId'],
            'SubnetIds': [
                event['ResourceProperties']['Subnet1'],
                event['ResourceProperties']['Subnet2'],
            ],
            'CustomerDnsIps': [
                event['ResourceProperties']['DNS1'],
                event['ResourceProperties']['DNS2'],
            ],
            'CustomerUserName': event['ResourceProperties']['CustomerUserName'],
        },
        Tags=[
            {
                'Key': 'Name',
                'Value': event['ResourceProperties']['Name'],
            },
        ]
    )
    print(f"Created Resource Share: \n {response}.")
    helper.Data.update({"test": "testdata"})

    return "MyResourceId"


@helper.update
def update(event, context):
    logger.info("Got Update")
    logger.info(event)
    # If the update resulted in a new resource being created, return an id for the new resource. CloudFormation will send
    # a delete event with the old id when stack update completes


@helper.delete
def delete(event, context):
    logger.info("Got Delete")
    logger.info(event)
    # Delete never returns anything. Should not fail if the underlying resources are already deleted. Desired state.
    client = boto3.client('ds')

    response = client.describe_directories()

    for directory in response['DirectoryDescriptions']:
        if directory['Type'] == "ADConnector" and directory['Description'] == event['ResourceProperties']['Description']:    
            status = client.delete_directory(
                DirectoryId=directory['DirectoryId']
            )
    print(f"Deleted Directory: \n {status}.")



@helper.poll_create
def poll_create(event, context):
    logger.info("Got create poll")
    # Return a resource id or True to indicate that creation is complete. if True is returned an id will be generated
    return True


# Lambda Handler
def lambda_handler(event, context):
    print(event)
    helper(event, context)

Invalid PhysicalResourceId on custom resource when using KMS

I'm using a Cloudformation custom resource lambda to connect to an RDS instance and run some commands as part of a stack. The resource works perfectly as long as the RDS instance is not encrypted with KMS. If it is, the RDS commands still runs, but the stack fails with the status reason: "Invalid PhysicalResourceId", thus triggers a delete of all the command ran during the create portion.

Is there something I can do to work around this?

Thanks!

Edit: I should note that I'm not returning a resource id as the documentation says it will generate one if not provided. I have tried generating one with the same result.

Example code produces error with 'sleep_on_delete'

image

When attempting to launch a custom resource with the provided example code in the README the above error informing that sleep_on_delete is an unexpected keyword argument is produced. Removing this variable allows the resource to succeed.

Probably related to: #29

Possible reasons for logs not appearing on Delete call?

I have a particular example were logs are not generated when delete is called, but they are on Create.

I tried looking into the particulars about how the sleep_on_delete parameter is handled, but I'm a bit confused. Could you explain how line 87 in the resource_helper achieves the requested wait time?

EDIT: More context

After debugging a bit here's the behavior I'm observing:

We're using custom resources to clean up AWS resources on CFN delete. For example, we empty S3 buckets, then delete them, because CFN fails to delete non-empty buckets.

Our CFN conditionally launches some resources that take a long time to spin up and delete (SageMaker notebook instances) (~5 min to create and delete).

When I launch the CFN with the condition set to true, the SageMaker notebook instance is created, and when deleting the stack, the cleanup function finishes and logs its output to CloudWatch as expected, as the stack takes ~5 minutes to delete.

When I launch with the condition set to false, the "slow" resource is not created, so the deletion of the stack is very fast. As a result, it seems like my cleanup function is not done by the time the custom resource lambda function starts getting deleted itself. There are no logs of the Delete call on CloudWatch, and the custom resource fails to delete with the error

Failed to delete resource. An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

crhelper appears tp break

I've had issues when using crhelper to get it return attributes back to the stack.

This very simple lambda works.

def on_event(event, context):
	props = event["ResourceProperties"]
	print("create new resource with props %s" % props)
	message = event['ResourceProperties']['RERUN']

	product_id = sc.search_products_as_admin(
		Filters={
			'FullTextSearch': ['AWS Control Tower Account Factory']
		},
	)['ProductViewDetails'][0]['ProductViewSummary']['ProductId']
	
	artifact_id = sc.describe_product_as_admin(
		Id = product_id
	)['ProvisioningArtifactSummaries'][0]['Id']
	

	attributes = {
		'ProductId': product_id,	
		'ArtifactId': artifact_id
	}
	return { 'Data': attributes }

if i replace it with this;

import crhelper
import boto3

helper = crhelper.CfnResource()

@helper.create
@helper.update
def get_account_factory_id(event, _):

	sc = boto3.client('servicecatalog')

	
	product_id = sc.search_products_as_admin(
		Filters={
			'FullTextSearch': ['AWS Control Tower Account Factory']
		},
	)['ProductViewDetails'][0]['ProductViewSummary']['ProductId']
	helper.Data['ProductId'] = product_id
	
	artifact_id = sc.describe_product_as_admin(
		Id = product_id
	)['ProvisioningArtifactSummaries'][0]['Id']
	helper.Data['ArtifactId'] = artifact_id
	
	
@helper.delete
def no_op(_, __):
	pass

def on_event( event, context ):
	helper(event, context)

I end up with this error.

The stack named AccountfactoryStack failed to deploy: CREATE_FAILED (CustomResource attribute error: Vendor response doesn't contain ArtifactId key in object arn:aws:cloudformation:ap-southeast-2:4xxxxxxxx7:stack/AccountfactoryStack/71875c50-f5a8-11ec-ac83-06c5a6d5f3f0|accountTahiprovisioningArtifactIdD6FC5169|5c9f7268-d6e5-45f5-b2fb-8cafdb68e1ee in S3 bucket cloudformation-custom-resource-storage-apsoutheast2)

It appears that somethign might be broken here. I love the idea of crhelper, its great, but not being able to use attributes is a killer.

Do not send response to CloudFormation when testing with sam local

When testing my custom resource with sam local, the resource helper tries to send a response to CloudFormation.

It generates an error since I am using fake information in my request payload.

Example:

{
  "RequestType": "Create",
  "ResponseURL": "http://pre-signed-S3-url-for-response",
  "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/MyStack/guid",
  "RequestId": "unique id for this create request",
  "ResourceType": "Custom::TestResource",
  "LogicalResourceId": "MyTestResource",
...

It would be nice to disable response sending to cfn when testing with sam local.

crhelper version: 2.0.10

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.