Operating Kubernetes means keeping an installed cluster in working order: reading its state, understanding an incident, and acting without breaking production. This module is about diagnosis. You learn to look at a running cluster, to follow a method when something goes wrong, and then to find the cause of a failure, whether it comes from an application or from the cluster itself. It is aimed at administrators and SRE profiles who take over a cluster in service, and at anyone preparing for the CKA.
Why does this module come after installation and administration?
A diagnosis assumes a cluster that exists and that you know how to put back in working order. The previous modules had you install a cluster with kubeadm, then operate etcd and back up and restore the cluster. You therefore know the components the cluster depends on, and you have a way back if something goes wrong. What is left is to know what is wrong, fast, without touching what works.
The modules on cluster maintenance and upgrades, and then on reliability and operations, build on this reflex. You do not prepare a maintenance window without having read the state of the nodes, and you do not tune a reliability indicator without knowing how to read an incident. Each of these modules has its own introduction.
What is the thread running through the lessons?
The lessons follow the course of an incident, from the broadest view to the most precise. You first learn to observe the health of a cluster (in French), that is, to know what a healthy cluster looks like. Without that reference, no symptom can be judged. The diagnostic method (in French) then gives the order of the checks, from the cluster to the node, to the Pod and then to the network. The lesson on events (in French) teaches you to read what Kubernetes itself says about what is happening to it.
Application failures come next. The page
Application incidents (in French)
is a routing page, followed by lessons on the common errors:
CrashLoopBackOff,
Pending Pods
and ImagePullBackOff (in French).
The module ends with the cluster itself: a
control plane failure,
where kubectl no longer answers, then a
node in NotReady,
where the kubelet is at fault. The control plane lesson and the NotReady node
lesson are practised on the node itself, not from your workstation.
Where do you start when an incident hits?
Start from the symptom you observe, not from the lesson that seems closest to your job. The cases are told apart as follows.
If kubectl no longer answers (connection refused, timeout), first check that
it targets the right cluster, with kubectl config current-context and the
address quoted in the message. A localhost:8080 address means it found no
kubeconfig at all, which is not a cluster failure. If the context and the address
are the right ones, open the lesson on the
control plane failure.
If a single node is NotReady, open the lesson on
troubleshooting a node.
If the cluster answers and a Pod is in error, the page
Application incidents (in French)
sends you to the right error according to what kubectl get pods shows.
If you do not know which of these cases you are in, or if no symptom stands out, follow the diagnostic method (in French), which stops as soon as a step reveals an anomaly. Outside an incident, start by reading the state of the cluster: it is the only way to have a point of comparison the day something breaks.
Which cluster should you prepare to follow the module?
The lessons assume a Kubernetes 1.37 cluster and a kubectl configured to
connect to it. You also need to know
Pods,
Deployments and
Services, because most
incidents are observed on them.
A kind cluster is enough to read the cluster state, analyze events and
reproduce the CrashLoopBackOff, Pending and ImagePullBackOff errors.
Control plane and kubelet failures are practised on a kubeadm cluster made of
virtual machines with SSH access to the nodes: the certification labs run as in
the exam, on real machines where the kubelet runs as a systemd service on the
host. On kind, it runs inside the node container, and breaking it does not
reproduce what you will do on a VM. This is the cluster the labs of these lessons
use.