Cloudfront: Route53 cname record not removed on stack deletion

[dflook]

Cloudfront: Route53 cname record not removed on stack deletion

[linford-bacon]

I really can't see how this could possibly be described as 'working as intended'.

I have a CDK stack that provisions a Route53 hosted zone as a subdomain which is used solely routing to the stacks resources. When I create certificates, I use DNS validation against my newly created hosted zone.

This all works fine on creation, but rollback and destroy are BROKEN!

The stack can never roll-back correctly because the route53 hosted zone can never be deleted. Instead, the stack ends up in a broken state that requires manual intervention.

I think that what I am describing is a pretty standard way to use hosted zones through route53. As things stand it feels like the only options are:

1. Manually setup and teardown hosted zones (i.e. they are just not supported by CDK/CloudFormation)
2. Accept that all automated rollbacks/destroys won't work (i.e. essentially that CDK does not support CI/CD)

[CompuGlobalHyperMegaNet-VP]

First, thank you for investigating that issue.

Is this required for your use case?

From a maintenance point of view, it would be desirable to avoid piling up deprecated CNAME records if there are no matching certificates left. It would be interesting to understand under which circumstances the same validation record is used for multiple certificates in ACM. As you created an duplicate of an existing certificate, it seems somehow understandable that they use the same validation record. However, I was not able to reproduce that with certs for different hosts/subdomains. Do you have additional insights here? If only multiple certs for the same domain are affected, one could image a check for the existence of other certs with the same subject, and if there are none delete the validation record as well?

Edit: Checked the AWS documentation.

https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-validate-dns.html

• Structure of generated validation record: The values are long random strings generated by ACM, so it's not a hash or something one can derive programmatically.

• This seems to describe the behavior you observed:
  You can request additional ACM Certificates for your FQDN for as long as the DNS record remains in place. That is, you can create multiple certificates that have the same domain name. You do not need to get a new CNAME record.

• Another case would be this particular example, where the validation record is the same for two "different" domains: ...the random string created by ACM for the wildcard name *.example.com is the same as that created for the base domain name example.com.

Assuming these two are the only exceptions from the rule, maybe it's possible to consider an improvement covering them, and otherwise delete the validation record.

Anyway, thanks for your time and the effort you put it here.

[dflook]

Hi, thanks for creating an issue.

This is expected behaviour. I've observed ACM use the same validation record for identical but distinct Certificates. This makes it difficult to delete the records along with the certificate, as I can't be sure they aren't being used by another Certificate.

I'll leave this issue open while I check this is still the case.

[dflook]

I've just created a duplicate of an existing certificate using this custom resource, and ACM used the same validation record.

If the resource deleted the record when the certificate was deleted, the remaining certificate would fail to renew. I don't believe this behaviour can be safely implemented 😬

Is this required for your use case?

[dflook]

I'm going to think about this a bit more :)

[gfodor]

Thanks for this info - probably worthy of an option (off by default), for CF stacks for which it's known they are the exclusive thing managing the domain/SSL.

[gfodor]

the issue we've been hitting with this is that if you re-create a stack on the same domain and ACM chooses the same validator record, the step fails. so an alternative would be to ignore duplicate records.

[dflook]

Hi @gfodor, re-creating the same certificate should not fail and is a defect. Can you create another issue to describe what you've seen?

[gfodor]

Will do if this sounds right - we have had customers of our aws marketplace template delete a stack and subsequent creates result in a “cname already exists error”

…

On Tue, Apr 7, 2020 at 1:04 AM Daniel Flook ***@***.***> wrote: Hi, re-creating the same certificate should not fail and is a defect. Can you create another issue to describe what you've seen? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABVW5HOBMRQRL3K6YLTPTDRLLNCVANCNFSM4INC25VA> .

[jeff1evesque]

It's kind of hilarious - I opened a ticket with AWS about this feature. And they sent me a link to this GitHub repository:

I was also able to get an open GitHUb issue where the users have discussed about different use cases where the automatic deletion of the DNS validation records will be an issue. Please check and let me know if you have any questions.

#7

For now, I am going to resolve this case as the time frame is not guaranteed and usually will not be shared with the AWS Premium Support teams.

What in the world is this repository and issue going to do for me - magically fix the internal AWS infrastructure? Is this even an AWS codebase?

[dflook]

@jeff1evesque I can confirm this is not an AWS codebase 😂.

This cloudformation resource pre-dates the ability for the AWS::CertificateManager::Certificate to do DNS validation.
It is still useful for cross-region certificates and cross-account validation, but won't help with removing unused validation records.

[linford-bacon]

Just as an update the functionality requested in this issue is actually available using acm.DnsValidatedCertificate via the cleanup_route53_records = True option. But not on the standard Certificate construct.

However, I understand that DnsValidatedCertificate is being deprecated in favour of Certificate, despite the latter not supporting the cleanup_route53_records or region options, making it essentially useless...