GitHub Pages and GCS resource efficiency

[sffc]

GitHub Pages and GCS resource efficiency

[sffc]

• @sffc Biggest cost is in data egress (downloading from GCS every time we want to upload to GH pages)
• @ambiguousname We could use https://github.com/marketplace/actions/github-pages-action#%EF%B8%8F-keeping-existing-files-keep_files, which has a keep files option if artifact creation fails
• @robertbastian - docs and wasm demo don't need to be stored on GCS, they can always be regenerated from code so we really don't lseo anything
  • WASM files keep getting renamed to avoid caching issues. GCP has an issue with this, Github pages does not. We can change the WASM file back to its regular name.
• @robertbastian - so we can save a lot of download by using GH actions artifacts to pass stuff between jobs
• @sffc Another option would be to just delete the obsolete wasm files periodically.
• @robertbastian - let's just not put the wasm demo gcs, it's less painful to keep it in GH only

[sffc]

@srl295 @rickmcgowan How should we prioritize this issue, relative to the other ICU4X issues?

[rickmcgowan]

@sffc As far as prioritization... The increase in data usage is concerning, but it's not a "crisis" at this point. Still, it would be good to increase efficiency and reduce the data flow, trying to remain inside our 100% budget each month. Some suggestions above seem like good places to start, at a moderate priority. Maybe some low-hanging fruit that would be easy to fix, and then see how that goes for a month or so.

[sffc]

The main reason we generate unique names each time we build WASM files is so that the old WASM files don't get cached.

It seems that GitHub Pages sets Cache-Control: public, max-age=3600 (1 hour)

The other way to dump the cache, one of the oldest tricks in the book, is to request foo.wasm?12345. Instead of making a file named 36e45f64df2543fe57b6.wasm, we could keep out WASM file always named icu_capi.wasm and then request foo.wasm?36e45f64df2543fe57b6, for example.

[sffc]

Actually, where in the code is the wasm file being renamed? Is it in webpack?

[ambiguousname]

I believe so, I'll see about fixing that

[ambiguousname]

@sffc PR for that is live here: #5423

Not sure what else you might want to add in terms of fixes, but this should hopefully help some

[ambiguousname]

I don't think this necessarily solves the timeout problem that we're seeing right now in Github pages deployments, however

[ambiguousname]

I can also make a PR to remove the docs from Google Cloud. That'd cut down on egress costs, but it would mean that if a run fails for these, we couldn't build to Github pages.

We might be able to find some sort of download-pages-artifact action? Then update the folder with our successful artifacts, then send it back.

[sffc]

The new WASM demo is deployed and it is loading with a query string hash:

I will manually move/delete the old WASM files and then it should resolve the issue.

[sffc]

Before this PR was merged:

https://github.com/unicode-org/icu4x/actions/runs/10476967726/job/29017522713

Download: 6m 30s
Upload: 5m 45s
Deploy: 8m 49s

Today:

https://github.com/unicode-org/icu4x/actions/runs/10588774112/job/29342053919

Download: 6m 3s
Upload: 24s
Deploy: 42s

Pretty huge improvement!