CodeBlanks Logo Icon CodeBlanks

Self-healing

Learn that ReplicaSet recreates deleted Pods.

Self-healing

The ReplicaSet controller runs a continuous reconciliation loop: it observes the actual pod count, compares it with the desired one, and closes the gap. If a pod is deleted, a node crashes or a pod is evicted, the missing pod is recreated:

$ kubectl delete pod web-7c5b9f6d8-abcde
$ kubectl get events --field-selector involvedObject.kind=ReplicaSet | tail -2
Normal  SuccessfulDelete  replicaset/web-7c5b9f6d8  Deleted pod: web-7c5b9f6d8-abcde
Normal  SuccessfulCreate  replicaset/web-7c5b9f6d8  Created pod: web-7c5b9f6d8-mnopq

Container restarts

There are two levels of healing. If a container crashes, the kubelet restarts the container inside the same pod (restartPolicy: Always) and the RESTARTS counter goes up. If the pod itself disappears, the ReplicaSet creates a new pod. The log of the previously crashed container is seen with --previous:

$ kubectl get pod api-5d8f7c9b4-qrstu
NAME                  READY   STATUS    RESTARTS      AGE
api-5d8f7c9b4-qrstu   1/1     Running   3 (2m ago)    20m
$ kubectl logs api-5d8f7c9b4-qrstu --previous

Practice Observe the healing

Write three commands: delete the web-7c5b9f6d8-abcde pod; list the pods labelled app=web; show the previous container log of the api-5d8f7c9b4-qrstu pod.

Starter code

terminal
# buraya yaz

Sample solution

terminal
kubectl delete pod web-7c5b9f6d8-abcde
kubectl get pods -l app=web
kubectl logs api-5d8f7c9b4-qrstu --previous

Expected output

pod "web-7c5b9f6d8-abcde" deleted
NAME                  READY   STATUS    RESTARTS   AGE
web-7c5b9f6d8-fghij   1/1     Running   0          2h
web-7c5b9f6d8-mnopq   1/1     Running   0          4s
panic: runtime error: invalid memory address

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 What does self-healing not cover?

  • A deleted pod
  • A pod on a crashed node
  • An evicted pod
  • A bug in the application; the new pod crashes with the same bug Correct answer

Explanation: A constantly crashing pod falls into CrashLoopBackOff.

2 Show the log of the previous (crashed) container.

Question
kubectl logs api-5d8f7c9b4-qrstu ____
Answer
kubectl logs api-5d8f7c9b4-qrstu --previous

Explanation: --previous (-p) is the previous container instance's log.

3 Match.

Term Meaning
kubelet Restarts the container
ReplicaSet Recreates the pod
RESTARTS Container restart count
--previous Log of the crashed one

All lessons in Intermediate