Flink-jobmanager stuck in CLBO

Sometimes you may see flink-jobmanager crashing and not coming up, which means one of the checkpoint snapshots is corrupted. To fix it you can use one of the following methods:

In Rancher

Scale down flink-jobmanager deployment to 0.
Scale down flink-taskmanager deployment to 0.
Go to storage/configmaps
Delete
1. gv-flink-cluster-config-map
2. gv-flink-*-config-map
Scale up flink-taskmanager deployment to 1.
Wait 15-20 sec
Scale up flink-jobmanager deployment to 1.

In terminal

kubectl scale --replicas=0 deployment/flink-jobmanager
kubectl scale --replicas=0 deployment/flink-taskmanager
kubectl get configmap -n default | grep "gv-flink"
kubectl delete configmap gv-flink-cluster-config-map -n default
kubectl delete configmap gv-flink-*-config-map -n default     <--- insert config map names from previous command
kubectl scale --replicas=1 deployment/flink-taskmanager
sleep 20
kubectl scale --replicas=1 deployment/flink-jobmanager

GV-KnowledgeBase

Flink-jobmanager stuck in CLBO

Analytics

In Rancher

In terminal

Related content