Standardise Whitehall and Mainstream Cache-Control headers

[chrisroos]

Standardise Whitehall and Mainstream Cache-Control headers

[chrisroos]

We've used Kibana to count the number of 304 responses for Mainstream assets from Asset Manager over the last 7 days (http_host:assets.publishing.service.gov.uk AND status:304 AND request:/GET \/media\/.*/ #=> 219,087).

Mainstream assets are cached for 24 hours. By changing this to 4 hours, we'd expect to see around 6 times the number of requests resulting in 304 responses from Asset Manager (i.e. 1,200,000 instead of 200,000). In the same 7 day period Asset Manager responded to 2,208,911 GET requests (application:asset-manager AND request:/GET.*/). The additional 1,000,000 requests represents nearly a 50% increase in total requests so it's not obvious this is the way to go.

[chrisroos]

We've used Kibana to count the number of 304 responses for Whitehall assets from Asset Manager over the last 7 days (http_host:assets.publishing.service.gov.uk AND status:304 AND request:/GET \/government\/.*/ #=> 1,047,031).

Whitehall assets are cached for 4 hours. By changing this to 24 hours, we'd expect to see around 6 times fewer requests resulting in 304 responses from Asset Manager (i.e. 174,505 instead of 1,047,031).

[chrisroos]

We're preparing to start serving about 1.5m Whitehall attachments from Asset Manager so we're currently thinking that it might be better to increase the cache time across the board to 24 hours.

[chrisroos]

@chrislo and I have started working on standardising the caching in #523.

[chrisroos]

The changes in #523 were merged yesterday. I've confirmed that they're working as expected in integration by requesting the same asset from integration and production and comparing the Cache-Control header. You can see that it was 4 hours in production (the previous value) and it's now 24 hours in integration (the new value).

# Production
$ curl -v "https://assets.publishing.service.gov.uk/government/uploads/system/uploads/person/image/6/s216_PM_portrait_960x640.jpg" > /dev/null

< Cache-Control: max-age=14400, public

# Integration
$ curl -v "https://assets-origin.integration.publishing.service.gov.uk/government/uploads/system/uploads/person/image/6/s216_PM_portrait_960x640.jpg" > /dev/null

< cache-control: max-age=86400, public

[chrisroos]

These changes have now been deployed to production and I've confirmed that the Cache-Control header is now set to 24 hours:

$ curl -v "https://assets.publishing.service.gov.uk/government/uploads/system/uploads/person/image/6/s216_PM_portrait_960x640.jpg?CJR$RANDOM" > /dev/null

< Cache-Control: max-age=86400, public

[chrisroos]

This is all done. Closing.