adobe/helix-purge

Purge two CDNs at once

Helix Purge

Purge two CDNs at once

Status

Installation

Usage

curl -X POST https://adobeioruntime.net/api/v1/web/helix/helix-services/purge@v1?host=…&xfh=…&path=…

• host: the inner cdn hostname
• xfh: the outer cdn hostnames, comma separated
• path: the path to purge

For more, see the API documentation.

Development

Deploying Helix Purge

Deploying Helix Purge requires the wsk command line client, authenticated to a namespace of your choice. For Project Helix, we use the helix namespace.

All commits to master that pass the testing will be deployed automatically. All commits to branches that will pass the testing will get commited as /helix-services/purge@ci<num> and tagged with the CI build number.

Operations

To temporarily stop purges, edit OK.md, so that it says "not ok".

helix-purge's Issues

Do not dedupe

helix-purge/src/index.js

Line 114 in ac1d658

.filter((fwhost) => fwhost !== host))) // skip inner CDN host

This might improve purge consistency a bit.

Purging a non-existing URL should not fail

It would be great if helix-purge could be relaxed (or have a relax parameter) when purging URLs that 404, so ...

curl -X HLXPURGE https://theblog--adobe.hlx.page/en/2019/05/07/adobe-summit-emea-2019.html 
[{
  "status": "error",
  "url": "https://theblog--adobe.hlx.page/en/2019/05/07/adobe-summit-emea-2019.html"
}]

... would return the same status as ...

curl -X PURGE https://theblog--adobe.hlx.page/en/2019/05/07/adobe-summit-emea-2019.html 
{ "status": "ok", "id": "4026-1611233774-3941357" }

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: config
Error type: Invalid allowedVersions
Message: The following allowedVersions does not parse as a valid version or range: "<15>"

Directory and file purging logic

helix-purge should handle directories and files as follows:

Inner CDN:

• Surrogate key purge
• also content-proxy cache: https://${branch}--${repo}--${owner}.hlx.page/${pathPrefix}.md

Outer CDN:

• /: directory, also purge /index and /index.html
• /index: also purge / and '/index.html`
• /index.html: also purge / and '/index`
• /foo.html: also purge /foo
• /foo: also purge /foo.html
• /bar.json: only purge /bar.json

and report all purged URLs in the response.

cc @tripodsan @trieloff

No Universal Runtime Support

## Purge Flexy CSS
curl -X "HLXPURGE" "https://pages--adobe.hlx.live/styles/blocks/flexy.css?1"

HTTP/1.1 404 Not Found
Connection: close
Content-Length: 91
Content-Type: text/plain; charset=utf-8
Apigw-Requestid: akt1JgtdIAMEMGA=
X-Backend-Health: 1 1
Accept-Ranges: bytes
X-Version: 517; src=517; cli=8.4.6; rev=online
Content-Security-Policy: frame-ancestors 'self'
Via: 1.1 varnish, 1.1 varnish
Cache-Control: max-age=7200
Date: Thu, 11 Feb 2021 09:19:03 GMT
X-Served-By: cache-bwi5121-BWI, cache-ams21025-AMS
X-Cache: MISS, MISS
X-Cache-Hits: 0, 0
X-Timer: S1613035143.165249,VS0,VE183
Vary: X-Debug,X-Strain,X-Request-Type,X-Forwarded-Host,X-OW-Version-Lock, Accept-Encoding
Strict-Transport-Security: max-age=31557600
Function not found: arn:aws:lambda:us-east-1:118435662149:function:helix-services--purge:v1

Check word and gdocs status, too

Do not purge when either is down.

archive-repository

• update readme: > Note: this repository and respective service is no longer maintained (commit with [skip ci])
• close all issues
• close all PRs
• disable status checks
• close this issue and archive

Use main as default branch instead of master

See adobe/helix-home#140

403s cannot be purged (if rejected by regexp)

Description
@davidnuescheler says pages that have a cached 403 can't be purged

To Reproduce
Steps to reproduce the behavior:

1. Believe @davidnuescheler

Expected behavior

Page should get purged

Static ok.html contains reference to branch `include-drives`

helix-purge/ok.html

Line 10 in 0e6c8e9

<meta property="og:url" content="https://include-drives--helix-purge--adobe.hlx.page/OK.html">

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update dependency nanoid to 3.1.31 [security]
• chore(deps): update dependency node-fetch to 2.6.7 [security]
• chore(deps): update dependency node-forge to 1.0.0 [security]
• chore(deps): update dependency simple-get to 3.1.1 [security]

---

• Check this box to trigger a request for Renovate to run again on this repository

Remove surrogate key purging

There is currently no added value in having asynchronous surrogate key purging in addition to the PURGE calls.

cc @davidnuescheler @trieloff

Remove dead-man switch

And use the purge stop in helix-publish instead: adobe/helix-publish#765

emergency stop switch

in a situation where the inner CDN produces faulty output due to system wide issues, it would be great if there was an easy way to centrally suspend the purge service temporarily to avoid purges of functioning outer CDN caches and therefore propagating inner CDN issues to the outer CDNs.

i think the initial activation of the emergency stop switch possibly initially should be triggered by the person acknowledging the outage and we should describe the procedure in the on-call runbook.

"Not purging inner CDN for null due to missing fastly credentials"

Description
As of 1.5.14 we're seeing the following error msg in the logs:

Not purging inner CDN for null due to missing fastly credentials

Logs: https://helix.coralogix.com/#/query/logs?id=2GMiohKfrn3

Looking at the code it seems that at least host and path are null.

@trieloff Regression of #103

Version:
helix-purge 1.5.14

Emergency stop switch: check plain html rendition

The "dead man switch" that blocks purging when something is wrong with Helix Pages currently checks the full rendition of OK.md against a static copy. This includes metadata tags with GH branches exposed in the URLs. This was the reason for a regression (see #50) and might cause further issues in the future.

Checking against a plain html rendition instead (/OK.plain.html) would be more streamlined, branch-safe and require less frequent updates of the static copy.

Disallow access to purge directly to the runtime

As suggested by @davidnuescheler in #190 (comment)

• Limit access to Fastly, possibly only the Helix Pages service
• Make sure CORS headers on outer CDN(s) allow sending HLXPURGE requests from *.hlx.page

detect correct default branch

Description
when purging an url w/o branch, purge uses master as default which is wrong.
it should detect the default branch.

Surrogate key purging

#2 @rofe

Nice! But...

Try https://adobeioruntime.net/api/v1/web/helix/helix-services/purge@ci?host=theblog--adobe.hlx.page&xfh=blog.adobe.com&path=/en/2020/08/14/6-ways-ta-adapt-advance-your-business-during-pandemic.html

... obviously in the case of theblog it's not so simple, because the paths differ between inner and outer CDN... it would be cool if instead of the path parameter, one could alternatively specify a surrogate key. This way we could also use this service to bust all cached index json segments at once.

We could then create a "Purge cache" bookmarklet, similar to the "Preview" one, which calculates the surrogate key from the URL of the source document.

Always purge URLs with and without html extension on the outer CDN

Is your feature request related to a problem? Please describe.
A Helix Pages resource can be accessed with or without html extension, and is therefore also cached separately.

Describe the solution you'd like
helix-purge should always purge both