1.14 Release Review

[hawksight]

1.14 Release Review

[SgtCoDFish]

My suggestion from the standup:

In the release thread for 1.14.0 @wallrj shared a snippet to generate a diff between the old chart and the new, which pretty much immediately made it obvious what there was another bug (a Service that was deleted when it shouldn't have been).

It'd be helpful to have that diff for any change to the chart, and it could be automatically generated. The current diff is noisy because it includes version number changes, but those could be filtered out before we diff to make the actual changes obvious!

E.g.

diff -u \
  <(helm template cert-manager --repo https://v1-14-0.jetstack-charts.pages.dev --version v1.13.x | grep -v 'app.kubernetes.io/version\|helm.sh/chart') \
  <(helm template _bin/cert-manager-v1.14.1-1-g6e52ae13660281.tgz | grep -v 'app.kubernetes.io/version\|helm.sh/chart')

[inteon]

Another possible improvement: we should specify as part of the release process when and who should validate a beta release by locally testing it.

[erikgb]

I am a big fan of automation, and I think @SpectralHiss agrees with me. So I would suggest introducing some post-release automation doing e2e-tests/smoke-tests on the release. That should allow us to detect any problems with the release, so we could eventually "yank" it - as they occasionally do with releases to PyPi. If our latest beta release and the final release are more or less identical, we only risk yanking alpha/beta releases. Manual tasks are doomed to fail at any time.

[wallrj]

While releasing the OperatorHub package, I've setup local automated testing and I've updated the Release Process:

• cert-manager/cert-manager-olm#104

[cert-manager-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
/lifecycle stale