Self-healing
- Docker & Kubernetes course
- Intermediate
- Lesson 35/82
- 4 min
Learn that ReplicaSet recreates deleted Pods.
Self-healing
The ReplicaSet controller runs a continuous reconciliation loop: it observes the actual pod count, compares it with the desired one, and closes the gap. If a pod is deleted, a node crashes or a pod is evicted, the missing pod is recreated:
$ kubectl delete pod web-7c5b9f6d8-abcde
$ kubectl get events --field-selector involvedObject.kind=ReplicaSet | tail -2
Normal SuccessfulDelete replicaset/web-7c5b9f6d8 Deleted pod: web-7c5b9f6d8-abcde
Normal SuccessfulCreate replicaset/web-7c5b9f6d8 Created pod: web-7c5b9f6d8-mnopqContainer restarts
There are two levels of healing. If a container crashes, the kubelet restarts the container inside the same pod (restartPolicy: Always) and the RESTARTS counter goes up. If the pod itself disappears, the ReplicaSet creates a new pod. The log of the previously crashed container is seen with --previous:
$ kubectl get pod api-5d8f7c9b4-qrstu
NAME READY STATUS RESTARTS AGE
api-5d8f7c9b4-qrstu 1/1 Running 3 (2m ago) 20m
$ kubectl logs api-5d8f7c9b4-qrstu --previousPractice Observe the healing
Write three commands: delete the web-7c5b9f6d8-abcde pod; list the pods labelled app=web; show the previous container log of the api-5d8f7c9b4-qrstu pod.
Starter code
# buraya yazSample solution
kubectl delete pod web-7c5b9f6d8-abcde
kubectl get pods -l app=web
kubectl logs api-5d8f7c9b4-qrstu --previousExpected output
pod "web-7c5b9f6d8-abcde" deleted NAME READY STATUS RESTARTS AGE web-7c5b9f6d8-fghij 1/1 Running 0 2h web-7c5b9f6d8-mnopq 1/1 Running 0 4s panic: runtime error: invalid memory address
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does self-healing not cover?
- A deleted pod
- A pod on a crashed node
- An evicted pod
- A bug in the application; the new pod crashes with the same bug Correct answer
Explanation: A constantly crashing pod falls into CrashLoopBackOff.
2 Show the log of the previous (crashed) container.
kubectl logs api-5d8f7c9b4-qrstu ____
kubectl logs api-5d8f7c9b4-qrstu --previous
Explanation: --previous (-p) is the previous container instance's log.
3 Match.
| Term | Meaning |
|---|---|
kubelet |
Restarts the container |
ReplicaSet |
Recreates the pod |
RESTARTS |
Container restart count |
--previous |
Log of the crashed one |