Rancher-webhook stuck in pending-upgrade
In case when rancher-webhook stuck in pending-upgrade state we can expect multiple pods creation which can deplete resources on the node, to resolve this issue please follow below steps
Using shell
Find secret what has pending-upgrade status
kubectl get secret -A -l status=pending-upgrade
Take backup of current secret [below is example only]
kubectl get secret -n cattle-system sh.helm.release.v1.webhook-manager -o yaml > webhook_backup.yaml
Delete the secret:
kubectl delete secret -n cattle-system sh.helm.release.v1.webhook-manager
This will trigger new installation of rancher-webhook, please ensure that rancher webhook is installed after and that it has appropriate version
Using Rancher UI
Check Apps->Installed Apps menu in rancher to see status of rancher-webhook application
identify problematic secret search for webhook in search box and cattle-system namespace you will see few secrets with name in this style [sh.helm.release.v1.rancher-webhook.v243],
use edit yaml and see which of them has status pending-upgrade
Copy content of yaml in another file and after that delete this secret
Wait for few minutes and ensure that rancher-webhook is healthy again
Related content
Classified as Getvisibility - Partner/Customer Confidential