Is there way to hot restart envoy proxy using istio-csr? I'm trying to renew root certificate by changing the istio-ca secret manually. The workload does not pick the new root certificate unless I delete the workload pods

[cert-manager]

Is there way to hot restart envoy proxy using istio-csr? I'm trying to renew root certificate by changing the istio-ca secret manually. The workload does not pick the new root certificate unless I delete the workload pods

[JoshVanL]

Hi @lokeshwaran100, thanks for opening the issue. I'm also interested in being able to do this in a less destructive way.

Another option could be to instead kill the istio-proxy container, which causes the container to restart, but not recycle the entire pod:

$  kubectl exec -it httpbin-577f7b7c47-r4mrj -c istio-proxy -- kill -s SIGINT 1

[lokeshwaran100]

Hi @lokeshwaran100, thanks for opening the issue. I'm also interested in being able to do this in a less destructive way.

Another option could be to instead kill the istio-proxy container, which causes the container to restart, but not recycle the entire pod:

$  kubectl exec -it httpbin-577f7b7c47-r4mrj -c istio-proxy -- kill -s SIGINT 1

Hi @JoshVanL, Thank you. Terminating the istio-proxy container worked.
But this current approach to work, it is also required to delete istio-csr agent pod and istiod pod.
Will there be a better way to do this in future releases?

[JoshVanL]

@lokeshwaran100 for istiod and istio proxies, this functionality needs to come from istio itself really. I believe CA rotation in a more first class way is something they would like to tackle in 2021.

If istio-csr is reading the CA from file, we could definitely do a better job in detecting changes and writing out the new bundle.

[SpectralHiss]

One potential solution to help would be to have another sidecar on the workloads running https://github.com/jimmidyson/configmap-reload,
which can watch when the trust bundle configmap changes and send a restart proxy call to the proxy api on localhost:15000/quitquitquit.

I've tested that by just by rollout restarting istio-csr and calling quitquitquit manually it works.
This could slightly improve the management overhead if there is an easy way to inject the configmap watcher sidecar at deploy time, perhaps by extending istioctl kube-inject somehow.

A slight variation is using the more popular configmap watcher project which does a full blown restart on a configmap change: https://github.com/stakater/Reloader , which could be used on both istio-csr and the workload pods.

I will test this out and let you know what works best for me.

[SpectralHiss]

After further consideration it's probably not a good idea to run a watcher just for this particularly infrequent operation, can just have a step to run the exec SIGINT as suggested by @JoshVanL until Istio provides with a way to do this. Should note that any istio CA including citadel will have the exact same issue here.