binxio/cfn-certificate-provider

A custom CloudFormation resource provider for creating DNS validated certificates in AWS

CloudFormation custom Certificate provider

Almost two years after the release of this custom provider, AWS finally provides native support for creating certificates with CloudFormation without manual intervention. Checkout https://aws.amazon.com/blogs/security/how-to-use-aws-certificate-manager-with-aws-cloudformation/

So, it is highly likely you do not need this custom provider anymore. If you want to create certificates in different regions, this provider is still pretty useful.

Custom Certificate Provider with DNS validation support

AWS Certificate Manager is a great service that allows the creation and renewal of certificates to be automated. It provides two ways of validating a certificate request: through email and through DNS.

When you are creating immutable infrastructure, the email validation method is a no-go as it requires human intervention. The DNS validation is of course the way to go! With 'Route53' we have full control over the DNS domain and can create the required records.

Although the CloudFormation AWS::CertificateManager::Certificate resource allow you to specify that you want DNS validation, it did not reveal the DNS records that you need to create. It writes them in the CloudFormation log file so that another human has to collect them and manually update the DNS record.

With this custom provider you can fully automated the creation of certificates with CloudFormation!

How do I request certificates fully automatically?

As a prerequisite, you need to have the hosted zones for the domain names on your certificate in Route53. If you have that, you can fully automate the provisioning of certificates, with the following resources:

1. Custom::Certificate to request a certificate without waiting for it to be issued
2. Custom::CertificateDNSRecord which will obtain the DNS record for a domain name on the certificate.
3. Custom::IssuedCertificate which will actively wait until the certificate is issued.
4. AWS::Route53::ResourceRecordSet to create the validation DNS record.

Checkout the sample in cloudformation/demo-stack.yaml.

Installation

To install this custom resource, type:

git clone https://github.com/binxio/cfn-certificate-provider.git
cd cfn-certificate-provider
aws cloudformation deploy \
        --capabilities CAPABILITY_IAM \
	--stack-name cfn-certificate-provider \
	--template-file cloudformation/cfn-certificate-provider.yaml

This CloudFormation template will use our pre-packaged provider from s3://binxio-public-${AWS_REGION}/lambdas/cfn-certificate-provider-1.0.4.zip.

Demo

To install the simple sample of the Custom Resource, type:

read -p "domain name: " DOMAIN_NAME
read -p "hosted zone id: " HOSTED_ZONE
aws cloudformation deploy --stack-name cfn-certificate-provider-demo \
	--template-file cloudformation/demo-stack.yaml \
	--parameter-overrides DomainName=$DOMAIN_NAME HostedZoneId=$HOSTED_ZONE

cfn-certificate-provider's Issues

tuple index error when deploying

When I add all custom resources to my cloud formation script I get this error when I deploy

Failed to create resource. tuple index out of range

Not sure how to debug this

IssuedCertificate

I think IssuedCertificate timed out waiting for the state change.

I'm using the CustomResource: IssuedCertificate waiting the certification of my Route53 HostedZone.
I think it timed out. The event log is: "Custom Resource failed to stabilize in expected time"
So... Do you know if is there a way to increase the time that the cloudformation stack could wait ?
I need the certificate valid to create other resources.

IssuedCertificate DependsOn

I noticed that it can take a while in bigger stacks for thing to complete this can be improved by adding the following to IssuedCertificate.

DependsOn:
  - DomainValidationRecord

Doing so will force CF to execute them sequentially instead of executing them concurrently with other resources. And since there is no point in waiting on issued cert until the record is added it should not cause any side effects.

cfn-certificate-provider/cloudformation/demo-stack.yaml

Line 20 in 616aa5f

IssuedCertificate:

Error installing custom resource

Keep getting

Invalid template path cloudformation/cfn-resource-provider.yaml

At installation step. Feel like I'm missing something basic here not sure what it is...

Tags not supported for Custom::Certificate resource

AWS::CertificateManager::Certificate does support tagging of the certificate, and it would be desirable if Custom::Certificate did too.

AWS natively supporting this now?

Seems like there is a post about something similar. Just wanted to confirm and wonder, is AWS natively supporting this now?

https://aws.amazon.com/about-aws/whats-new/2020/06/aws-certificate-manager-extends-automation-certificate-issuance-via-cloudformation/

Question: Pipfile entry seems to cause issues for serverless-python-requirements

Wrapping this work in a serverless package to fit with a deployment pipeline, I noticed that https://github.com/binxio/cfn-certificate-provider/blob/master/Pipfile#L11 causes issues for the serverless-python-requirements plugin.

I'm out of my depth with respect to the python context here, so I thought I'd ask if the "requirements-txt" dependency in the Pipfile is something special, or if it's a real dependency that I just didn't satisfy on my local.

When I remove that line from the Pipfile, the serverless plugin is happy.

Wildcard DNS Causes "No validation option found for domain"

It looks like specifying a wildcard causes the error "No validation option found for domain"

Looking over the code it looks like it uses the verbatim domain name of the certificate to look for the validation method. I suspect it should be able to query for a matching domain in that case.

The following is my template.

Description: Stack to contain the SSH Certificates

Parameters:

  Domain:
      Type: String
      Description: The DNS Domain Name for this instance

Resources:

  MainCertificate:
    Type: Custom::Certificate
    Properties:
      DomainName: !Sub "app.${Domain}"
      ValidationMethod: DNS
      ServiceToken: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:binxio-cfn-certificate-provider'

  MainIssuedCertificate:
    Type: Custom::IssuedCertificate
    DependsOn:
      - MainCertificate
    Properties:
      CertificateArn: !Ref MainCertificate
      ServiceToken: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:binxio-cfn-certificate-provider'

  MainCertificateDNSRecord:
    Type: Custom::CertificateDNSRecord
    Properties:
      CertificateArn: !Ref MainCertificate
      DomainName: !Ref Domain
      ServiceToken: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:binxio-cfn-certificate-provider'

  ValidationRecord:
    Type: AWS::Route53::RecordSet
    DependsOn:
      - MainCertificateDNSRecord
    Properties:
      HostedZoneName: !Sub "${Domain}."
      Type: !GetAtt MainCertificateDNSRecord.Type
      Name: !GetAtt MainCertificateDNSRecord.Name
      TTL: 60
      ResourceRecords:
        - !GetAtt MainCertificateDNSRecord.Value

Outputs:

  CertificateArn:
    Value: !Ref MainCertificate
    Description: The Certificate ARN

Custom::IssuedCertificate doesn't wait for Certificate creation before timing out.

IssuedCertificate:
Type: Custom::IssuedCertificate
Properties:
CertificateArn: !Ref Certificate
ServiceToken: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:binxio-cfn-certificate-provider'

It seems like this function is suppose to wait forever until the certificate is issue but it still seems to time out waiting for validation causing the cf template to fail and roll back.

[2019-10-22 12:21:39] - sceptre.stack - 2019-10-22T19:21:39+00:00 qlh/wdc/sec/cert/acm sq-qlh-wdc-sec-cert-acm AWS::CloudFormation::Stack �[33mROLLBACK_IN_PROGRESS�[0m The following resource(s) failed to create: [IssuedCertificate]. . Rollback requested by user.
[2019-10-22 12:21:58] - sceptre.stack - 2019-10-22T19:21:55+00:00 qlh/wdc/sec/cert/acm IssuedCertificate Custom::IssuedCertificate �[33mDELETE_IN_PROGRESS�[0m

Key "lambdas/cfn-certificate-provider-1.0.2.zip" does not exist

I think you forgot to upload cfn-certificate-provider-1.0.2.zip to S3:

$ aws s3 cp s3://binxio-public-us-east-1/lambdas/cfn-certificate-provider-1.0.2.zip .
fatal error: An error occurred (404) when calling the HeadObject operation: Key "lambdas/cfn-certificate-provider-1.0.2.zip" does not exist

I've tried us-east-1, eu-central-1 and eu-west-1.

S3 bucket binxio-public-eu-west-3 hosted in eu-central-1

As stated in the title, the binxio-public-eu-west-3 S3 bucket is hosted in the wrong AWS region. This breaks deployments to the Paris (eu-west-3) region for people who grab the code from your buckets, leaving them confused as to why their deployments are not working because the bucket looks like it should be in the right region.

Custom::Certificate SubjectAlternativeNames support

Hi,

Nice custom resource!. Very useful.
In my case I need to generate a certificate to cover various subdomains, and I see this feature is missing now. Fortunately it's a very simple change: just need to include this dict in properties dict in certificate_provider.py. I just tested it and it works perfectly:

"SubjectAlternativeNames": { "type": "string" }

Will this package be available through pip?

NoSuchKey cfn-certificate-provider-0.2.3.zip

Hi guys,

It seems like the lambdas/cfn-certificate-provider-0.2.3.zip is not in the public bucket?

Once I deploy the stack (in eu-west-1) I get Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist.

I also tried hitting the S3 URL directly in eu-west-1 - https://binxio-public-eu-west-1.s3.eu-west-1.amazonaws.com/lambdas/cfn-certificate-provider-0.2.3.zip - without success.

Thanks for the effort on this - awesome if it can fully automate the ACM provisioning 👍

NoSuchKey while deploying in ap-east-1

Hi I'm creating the stack in ap-east-1 and I get the error

ResourceLogicalId:CFNCustomProvider, ResourceType:AWS::Lambda::Function, ResourceStatusReason:Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: b7304fcc-8c27-44fc-b699-6529b0be4ee7; Proxy: null).

Should I use S3 regional endpoint?

Domain is using validation method EMAIL, not DNS

Today this error popped up on a code that has been running without a problem for many months: Received response status [FAILED] from custom resource. Message returned: domain is using validation method EMAIL, not DNS.

This is the relevant part of template:

    Certificate:
      Type: Custom::Certificate
      Properties:
        DomainName: ${self:custom.domain}
        SubjectAlternativeNames:
          Fn::If:
            - HasProductionAlias
            - - ${self:custom.ProductionDomainAlias}
            - !Ref AWS::NoValue
        ValidationMethod: DNS
        Region: us-east-1
        ServiceToken: ${self:custom.serviceToken}
    CertificateBlocker:
      Type: Custom::IssuedCertificate
      DependsOn:
        - DnsRecordsCertificateValidation
      Properties:
        CertificateArn: !Ref Certificate
        ServiceToken: ${self:custom.serviceToken}
    CertificateDnsRecord:
      Type: Custom::CertificateDNSRecord
      Properties:
        CertificateArn: !Ref Certificate
        DomainName: ${self:custom.domain}
        ServiceToken: ${self:custom.serviceToken}
    DnsRecordsCertificateValidation:
      Type: AWS::Route53::RecordSetGroup
      Properties:
        HostedZoneId: ${self:custom.hostedZone}
        RecordSets:
          - Name: !GetAtt CertificateDnsRecord.Name
            Type: !GetAtt CertificateDnsRecord.Type
            TTL: 60
            Weight: 1
            SetIdentifier: !Ref Certificate
            ResourceRecords:
              - !GetAtt CertificateDnsRecord.Value

LogGroup Not Deleted

We're using cloudformation/cfn-resource-provider.yaml as a nested stack in another CF Template. When that stack fails to deploy, the log group is not cleaned up.

My best guess is that this is the cause. Specifically the Log Group is destroyed while another lambda is still running. When it completes, it logs messages which (due to its permissions) recreate the log.

This seems to be resolved by revoking logs:CreateLogGroup (since CF handles the create anyway) e.g. by reducing the logs permissions from logs:* to:

- logs:CreateLogStream
- logs:PutLogEvents

The log is created and completely deleted. I've tried it twice both ways in an effort to confirm.

EDIT: It's also possible (but untested) that reversing the DependsOn (so the lambda is deleted before the Log Group) would be adequate.

Custom::Certificate Tags support

AWS::CertificateManager::Certificate supports Tags property, but Custom::Certificate seems not

Failed to create resource. 
Parameter validation failed: Unknown parameter in input: "Tags", must be one of: DomainName, ValidationMethod, SubjectAlternativeNames, IdempotencyToken, DomainValidationOptions, Options, CertificateAuthorityArn

Example:

Certificate:
    Type: Custom::Certificate
    Properties:
      DomainName: !Sub '${EnvironmentId}.${RootDomain}'
      SubjectAlternativeNames:
        - !Sub '*.${EnvironmentId}.${RootDomain}'
      ValidationMethod: DNS
      ServiceToken: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:binxio-cfn-certificate-provider'
      Tags:
        - Key: EnvironmentId
          Value: !Ref EnvironmentId
        - Key: example
          Value: com

Optional Email Notification

I'm trying to create both a Public Hosted Zone and a Certificate at the same time. We use exports to make these values available to other templates. The catch-22 is that our domain name is currently hosted at Google Domains.

When I run the stack, the Public Zone is created. Before validation completes, I need to update my nameservers. They're already pointed to awsdns servers from the last time I tested the stack, but that doesn't appear to be enough. They actually need to be updated to the awsdns servers for the new hosted zone. Since they're in the same template, my create doesn't return to do external processing.

I understand that the goal of this package is fully automated validation, but suspending a CF create while the manual update is made is almost as valuable (and likely very common). I think the simplest approach would be an extra Custom:: step that accepts the CertificateArn, DomainName, HostedZoneId, and SnsTopicArn and sends a notification to that SNS Topic with the domain, name servers, and CNAME entries.

If your nameservers are already at AWS, you skip the step.

How to add additional names?

Thanks for your awesome work!
Is there anyway to provide additional names??

SubjectAlternativeNames support

It seems the current version does not support SubjectAlternativeNames.

The Custom::Certificate resource does support this element, but adresses entered here do not get validated automatically. The main domain name does get validated, but I have to validate the record for my subject alternative name with a www prefix myself.

Delete with no Physical ID

I'm reasonably confident that I've read (somewhere, can't find it) that a Custom Resource should return DELETE_FAILED if there is no resource to delete. I'm wondering if the current behavior of returning None if no physical_resource_id exists is actually correct.

Under what situations is delete called without a physical_resource_id or -- more concerning -- with a physical_resource_id that does not begin with arn:aws:acm:? Is that actually a valid condition or should it error?

Trying to use the certificate with CloudFront but I get this error `InvalidViewerCertificate`

I'm adding a CloudFront resource to my current template that uses your custom resource

"CloudFront": {
            "Type": "AWS::CloudFront::Distribution",
            "Properties": {
                "DistributionConfig": {
                    "Aliases": [
                        {
                            "Ref": "DomainName"
                        }
                    ],
                    "Enabled": true,
                    "Origins": [
                        {
                            "DomainName": {
                                "Fn::Select": [
                                    1,
                                    {
                                        "Fn::Split": [
                                            "//",
                                            {
                                                "Fn::GetAtt": [
                                                    "DistBucket",
                                                    "WebsiteURL"
                                                ]
                                            }
                                        ]
                                    }
                                ]
                            },
                            "Id": "origin",
                            "CustomOriginConfig": {
                                "OriginProtocolPolicy": "http-only"
                            }
                        }
                    ],
                    "DefaultCacheBehavior": {
                        "TargetOriginId": "origin",
                        "DefaultTTL": 5,
                        "MaxTTL": 30,
                        "ForwardedValues": {
                            "QueryString": false
                        },
                        "ViewerProtocolPolicy": "redirect-to-https"
                    },
                    "ViewerCertificate": {
                        "AcmCertificateArn": {
                            "Ref": "Certificate"
                        },
                        "SslSupportMethod": "sni-only"
                    }
                }
            }
        }
    },

Where the "Ref": "Certificate" is resource of "Type": "Custom::Certificate",

I get the error CREATE_FAILED:

The specified SSL certificate doesn't exist, isn't in us-east-1 region, isn't valid, or doesn't include a valid certificate chain. (Service: AmazonCloudFront; Status Code: 400; Error Code: InvalidViewerCertificate; Request ID: d27ebd45-5939-11e9-ada3-edb69a0d9adc)

Any tips?

Incorrect Readme S3

I believe the readme should be updated to

"This CloudFormation template will use our pre-packaged provider from s3://binxio-public-${AWS_REGION}/lambdas/cfn-certificate-provider-latest.zip."

`binxio-public-eu-west-3` bucket is not in the `eu-west-3` region

Hello 😃

I'm using this nice template (version 0.2.4) in the us-east-1 region and it's working like a charm, however when I try to setup this stack in the eu-west-3 region I get a PermanentRedirect that makes my entire stack fail.
The reason behind that is because the code used by AWS::Lambda::Function must point to "An Amazon S3 bucket in the same AWS Region as your function." source from docs

As I try to curl the file, it appears that this bucket's region is eu-central-1.

> curl -I https://binxio-public-eu-west-3.s3.eu-west-3.amazonaws.com/lambdas/cfn-certificate-provider-0.2.4.zip
HTTP/1.1 301 Moved Permanently
x-amz-bucket-region: eu-central-1
...
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 21 Jan 2020 21:39:10 GMT
Server: AmazonS3

> curl -I https://binxio-public-eu-west-3.s3.eu-central-1.amazonaws.com/lambdas/cfn-certificate-provider-0.2.4.zip
HTTP/1.1 200 OK
...
Date: Tue, 21 Jan 2020 21:40:20 GMT
Last-Modified: Wed, 20 Mar 2019 17:30:17 GMT
Accept-Ranges: bytes
Content-Type: application/zip
Content-Length: 11419689
Server: AmazonS3

Is it possible from you to move this bucket in the eu-west-3 region so it's usable there aswell or should I deploy my whole stack in another region? 😅

Thanks for your work 😄