A Pod does not start, crashes in a loop, or runs while the application does not answer: the diagnosis always starts from the symptom that kubectl get pods displays, and each symptom has a first command. In this lesson, you learn this method on four cases: a Pod blocked by a missing ConfigMap, a Pod that crashes, a Pod with no shell that you examine with an ephemeral container, and an application that is unreachable while its Pod is Running. The symptoms already covered elsewhere (image not found, Pending Pod, OOMKilled) are summarized with a link.
This lesson assumes that you have followed Pods, Services (the Service of the demonstration is provided without explanation), ConfigMaps and Requests and limits. You also have the demo cluster created in First cluster.
What you will learn
- Choose the first diagnostic command from the status of the Pod
- Read the events of a blocked Pod and unblock it without recreating it
- Read the logs of a container, including
kubectl logs --previous, and aState/Last State - Diagnose a failing
kubectl execand examine a Pod with no shell usingkubectl debug - Locate in three steps an application that is unreachable while the Pod is running
Quick decision table
The status displayed by kubectl get pods guides the diagnosis: the table gives the first command for each symptom.
| Symptom | First command | For what comes next |
|---|---|---|
CreateContainerConfigError | kubectl events --for pod/<pod> | This lesson |
ContainerCreating that lasts | kubectl events --for pod/<pod> | This lesson |
CrashLoopBackOff or Error | kubectl logs <pod> then --previous | CrashLoopBackOff |
ImagePullBackOff | kubectl describe pod <pod> | ImagePullBackOff (in French) |
Pending | kubectl events --for pod/<pod> | Pending Pods |
OOMKilled | kubectl describe pod <pod> | Requests and limits |
Running but unreachable | kubectl describe service <svc> | This lesson |
For a complete incident method, see Incident diagnosis method (in French) and Analyze events (in French). To copy files (kubectl cp), measure the consumption (kubectl top) or open a direct access (kubectl port-forward), see Read the cluster state: kubectl get, describe, logs and top.
A Pod blocked by a missing ConfigMap
You work in a dedicated namespace, deleted at the end:
kubectl config use-context kind-demo && kubectl create namespace debogageSwitched to context "kind-demo".namespace/debogage createdThe names and messages of this lesson are in French on purpose, so that the outputs stay those of the tested cluster: debogage means "debugging", appli "app", reglages "settings", plante "crashes", minimal "minimal", lecteur "reader", debogueur "debugger", and sans-shell.yaml "no shell". The echo messages "démarrage", "erreur fatale : base introuvable", "même Pod" and "code de sortie" mean "startup", "fatal error: database not found", "same Pod" and "exit code". The file names config-absente.yaml, inaccessible.yaml and uid-avant.txt mean "missing config", "unreachable" and "uid before".
The appli Pod reads an environment variable from the reglages ConfigMap, which does not exist yet:
mkdir -p ~/atelier-k8s/debogage && cd ~/atelier-k8s/debogagecat > config-absente.yaml <<'EOF'apiVersion: v1kind: Podmetadata: name: applispec: containers: - name: nginx image: nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb env: - name: MODE valueFrom: configMapKeyRef: name: reglages key: MODEEOFkubectl apply -f config-absente.yaml -n debogage && sleep 10 && kubectl get pod appli -n debogagepod/appli createdNAME READY STATUS RESTARTS AGEappli 0/1 CreateContainerConfigError 0 10sCreateContainerConfigError means that the kubelet could not prepare the configuration of the container, hence RESTARTS 0. The container was not even created, and the status does not say why. For the cause, read the Pod's events.
How do you read a Pod's events?
The kubectl events --for pod/<pod> command lists the events of an object in order. It is the first reflex when a Pod does not start, because the kubelet writes there what is blocking.
kubectl events -n debogage --for pod/appliLAST SEEN TYPE REASON OBJECT MESSAGE10s Normal Scheduled Pod/appli Successfully assigned debogage/appli to demo-control-plane9s (x2 over 9s) Normal Pulled Pod/appli Container image "nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb" already present on machine and can be accessed by the pod9s (x2 over 9s) Warning Failed Pod/appli Error: configmap "reglages" not foundThe Warning Failed line gives the cause: configmap "reglages" not found. Create the missing ConfigMap, without touching the Pod, after noting its unique identifier (uid) to prove afterwards that the Pod was not recreated:
kubectl get pod appli -n debogage -o jsonpath='{.metadata.uid}' > uid-avant.txt && kubectl create configmap reglages -n debogage --from-literal=MODE=demo && kubectl wait --for=condition=Ready pod/appli -n debogage --timeout=180sconfigmap/reglages createdpod/appli condition metThe Pod became ready on its own: the kubelet retries to create the container, and as soon as the ConfigMap exists, it succeeds. Here, the Pod is ready one second after the creation; the delay can be longer, because the wait between two attempts grows. Check that it is the same Pod:
test "$(cat uid-avant.txt)" = "$(kubectl get pod appli -n debogage -o jsonpath='{.metadata.uid}')" && echo "même Pod" && kubectl get pod appli -n debogagemême PodNAME READY STATUS RESTARTS AGEappli 1/1 Running 0 11sThe uid did not change: fixing the cause is enough, there is nothing to recreate.
The same missing ConfigMap can also be mounted as a volume instead of being read in an environment variable. The symptom changes: the Pod stays in ContainerCreating, because the kubelet has to prepare the volume before creating the container. The Pod's events then carry a Warning with the reason FailedMount, whose message names the ConfigMap that cannot be found. The approach does not change: you read the events, you note the cause, then you create the ConfigMap with the expected content. The kubelet retries to mount the volume, and the Pod starts on its own, without being recreated.
Read a Pod: exec, logs and state
How do you open a shell in a container?
kubectl exec runs a command in a running container. To open an interactive shell, add -it and run sh: kubectl exec -it appli -n debogage -- sh. Here, the command is a one-off, to observe what exec returns. The exit code of the command is relayed: the command terminated with exit code N message repeats the code of the command that was run, not a state of the Pod.
kubectl exec appli -n debogage -- sh -c 'echo "MODE=$MODE"; exit 137'; echo "code de sortie : $?"MODE=democommand terminated with exit code 137code de sortie : 137The MODE variable comes from the ConfigMap created in the previous step. The code 137 is the one that the command chose, simulated here: a code 137 relayed by exec does not prove that the container was killed. To run a command in a specific container of a multi-container Pod, add -c <name>.
How do you read a container's logs?
kubectl logs <pod> displays the output of a container. On a multi-container Pod, the command picks the default container and announces it with the Defaulted container ... out of: ... line; to read another one, use -c, as shown in Init containers and sidecars.
The plante Pod prints a startup line, an error on the error output, waits 15 seconds then stops with a failure. The wait leaves time to read the logs of the previous attempt while the new attempt runs:
cat > plante.yaml <<'EOF'apiVersion: v1kind: Podmetadata: name: plantespec: containers: - name: app image: busybox:1.38.0@sha256:a46257e06c82ad85242d0d38077d0a52d9896389ca60a1a59d3463db586e5fa4 command: - sh - -c - | echo "démarrage" echo "erreur fatale : base introuvable" >&2 sleep 15 exit 1EOFkubectl apply -f plante.yaml -n debogage && sleep 17 && kubectl get pod plante -n debogage && kubectl logs plante -n debogage --previouspod/plante createdNAME READY STATUS RESTARTS AGEplante 1/1 Running 1 (1s ago) 17serreur fatale : base introuvabledémarrageRESTARTS 1: the first attempt failed and a second one is running. The --previous option reads the logs of the attempt that has just ended, which gives the cause of the crash: base introuvable (database not found). The order of the two lines is not guaranteed, because the standard output and the error output are two distinct streams: you may see démarrage before or after the error.
One limit is worth knowing: the kubelet only keeps the last stopped container. When the current attempt has also ended, the "previous" one has already been deleted and kubectl logs --previous answers unable to retrieve container logs for containerd://…. In that case, kubectl logs <pod> without --previous returns the logs of the last attempt.
For the state, kubectl describe pod plante -n debogage displays State (the current state of the container) and Last State (that of the previous attempt), with the reason (Reason: Error), the exit code (Exit Code: 1) and the start and end times. Just after the restart, the container's section looks like this (the times are those of your run):
State: Running Started: Wed, 11 Nov 2026 16:09:43 +0000 Last State: Terminated Reason: Error Exit Code: 1 Started: Wed, 11 Nov 2026 16:09:28 +0000 Finished: Wed, 11 Nov 2026 16:09:43 +0000 Ready: TrueLast State keeps the first attempt: it lasted 15 seconds, between Started and Finished, then failed with code 1.
Fifteen seconds later, the second attempt has also ended: the Pod is 0/1 Error and State becomes Terminated with Reason: Error and Exit Code: 1, while Last State stays the same. The current state therefore depends on the moment: Terminated just after a failure, Waiting with CrashLoopBackOff during the wait between two attempts, Running during an attempt.
When kubectl exec fails: read the message
A failing exec gives a message that guides the diagnosis: the image has no shell, the container does not exist (or not yet), or you do not have the right. The minimal Pod uses the pause image, which contains a single executable:
cat > sans-shell.yaml <<'EOF'apiVersion: v1kind: Podmetadata: name: minimalspec: containers: - name: pause image: registry.k8s.io/pause:3.10@sha256:ee6521f290b2168b6e0935a181d4cff9be1ac3f505666ef0e3c98fae8199917aEOFkubectl apply -f sans-shell.yaml -n debogage && kubectl wait --for=condition=Ready pod/minimal -n debogage --timeout=90s && kubectl exec minimal -n debogage -- shpod/minimal createdpod/minimal condition meterror: Internal error occurred: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec "95e2f199a9ac1b2e5512fb51243849f5d8b384f65f25374c5b9875556d4beaff": OCI runtime exec failed: exec failed: unable to start container process: exec: "sh": executable file not found in $PATHThe useful message is at the end: exec: "sh": executable file not found in $PATH. The image has no shell at all, and this is not a permissions problem. The solution is an ephemeral container, seen further down.
The other possible cause remains, the permission. Entering a container is a permission distinct from reading Pods: kubectl exec first reads the Pod, then asks for the create right on the pods/exec subresource. The kubectl auth can-i create pods/exec command tests this second right. With your administrator account on the demo cluster, the answer is positive:
kubectl auth can-i create pods/exec -n debogageyesTo see a refusal, create a service account (ServiceAccount, the identity that a program uses in the cluster) that has received no rights. The --as option then runs the check under its identity:
kubectl create serviceaccount lecteur -n debogage && kubectl auth can-i create pods/exec -n debogage --as=system:serviceaccount:debogage:lecteurserviceaccount/lecteur creatednoThe no explains a refused exec: this account does not have the create right on pods/exec. A refused exec answers with a Forbidden error, which must not be confused with the missing shell message. An account with no rights at all can already be refused when reading the Pod, before reaching pods/exec. To grant this right, see RBAC.
How do you examine a Pod with an ephemeral container?
An ephemeral container is a container added to a running Pod, without recreating it, to examine it with tools that the original image does not have. The --target option makes it share the process space of a container of the Pod. Add a busybox container to the minimal Pod:
kubectl debug minimal -n debogage -c debogueur --image=busybox:1.38.0@sha256:a46257e06c82ad85242d0d38077d0a52d9896389ca60a1a59d3463db586e5fa4 --target=pause -- sleep 600Targeting container "pause". If you don't see processes from this container it may be because the container runtime doesn't support this feature.The command returns immediately (without -it, it does not attach a terminal). The message says that the ephemeral container targets pause. An immediate exec in this new container fails, because it has not started yet:
kubectl exec minimal -n debogage -c debogueur -- pserror: unable to upgrade connection: container not found ("debogueur")container not found does not mean that the container is absent, but that the container is not yet ready to receive an exec. Wait for it with a short loop: it starts in a few seconds when the image is already on the node, and the loop displays the same message at each failed attempt before ps answers.
for i in $(seq 1 20); do kubectl exec minimal -n debogage -c debogueur -- ps && break; sleep 1; doneerror: unable to upgrade connection: container not found ("debogueur")error: unable to upgrade connection: container not found ("debogueur")PID USER TIME COMMAND 1 65535 0:00 /pause 21 root 0:00 sleep 600 34 root 0:00 psThe first two lines are the attempts where the container did not exist yet; the number of error lines varies with the startup time. The successful ps is displayed next and the loop stops (break).
The /pause process, PID 1, belongs to the target container: this is the effect of --target. Without this option, only the processes of the ephemeral container are visible. Finally, check that the original Pod was not touched:
kubectl get pod minimal -n debogage -o jsonpath='{.spec.ephemeralContainers[*].name}{"\n"}{.status.containerStatuses[0].restartCount}{"\n"}'debogueur0The ephemeral container is recorded in the Pod's spec and the original container did not restart.
Pod Running but application unreachable
The Pod is Running and the application does not answer: the fault lies between the client and the application, and three steps locate it. The inaccessible.yaml file creates a healthy web Deployment, a web-svc Service whose selector contains a typo, and a client Pod for the tests:
cat > inaccessible.yaml <<'EOF'apiVersion: apps/v1kind: Deploymentmetadata: name: webspec: replicas: 1 selector: matchLabels: app: web template: metadata: labels: app: web spec: containers: - name: nginx image: nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb ports: - containerPort: 80---apiVersion: v1kind: Servicemetadata: name: web-svcspec: selector: app: wev # ❌ faute de frappe : aucun Pod ne porte ce label ports: - port: 80 targetPort: 80---apiVersion: v1kind: Podmetadata: name: clientspec: containers: - name: busybox image: busybox:1.38.0@sha256:a46257e06c82ad85242d0d38077d0a52d9896389ca60a1a59d3463db586e5fa4 command: ["sh", "-c", "sleep 3600"]EOFkubectl apply -f inaccessible.yaml -n debogage && kubectl rollout status deployment/web -n debogage --timeout=90s && kubectl wait --for=condition=Ready pod/client -n debogage --timeout=90sdeployment.apps/web createdservice/web-svc createdpod/client createdWaiting for deployment "web" rollout to finish: 0 of 1 updated replicas are available...deployment "web" successfully rolled outpod/client condition metThe comment in the manifest means "typo: no Pod carries this label".
Step 1: Check that the Pod answers directly
The client Pod calls the IP address of the web Pod, bypassing the Service. If the Pod answers, the application is not at fault:
POD_IP=$(kubectl get pod -n debogage -l app=web -o jsonpath='{.items[0].status.podIP}'); kubectl exec client -n debogage -- wget -qO- -T 3 "http://$POD_IP" | head -n 4<!DOCTYPE html><html><head><title>Welcome to nginx!</title>The nginx welcome page is displayed: the Pod and the application work. The problem is therefore at the Service level.
Step 2: Check the Service and the Endpoints
Read the selector of the Service, the list of its addresses (Endpoints) and the real labels of the Pods:
kubectl describe service web-svc -n debogage | grep -E "Selector|Endpoints"; kubectl get pods -n debogage --show-labels -l app=webSelector: app=wevEndpoints:NAME READY STATUS RESTARTS AGE LABELSweb-5f784567c7-sfkb7 1/1 Running 0 1s app=web,pod-template-hash=5f784567c7The selector of the Service is app=wev, the Pod carries app=web: the typo jumps out. The Endpoints: line is empty, which confirms that no Pod is selected.
Step 3: Test connectivity from another Pod
Reproduce what a real client sees: call the Service by its name, from another Pod.
kubectl exec client -n debogage -- wget -qO- -T 3 http://web-svc; echo "code de sortie : $?"wget: can't connect to remote host (10.96.146.126): Connection refusedcommand terminated with exit code 1code de sortie : 1The Service has an address, but it has no Pod behind it: the connection is refused. Fix the selector and run the test again:
kubectl patch service web-svc -n debogage -p '{"spec":{"selector":{"app":"web"}}}' && sleep 3 && kubectl exec client -n debogage -- wget -qO- -T 3 http://web-svc | head -n 4service/web-svc patched<!DOCTYPE html><html><head><title>Welcome to nginx!</title>By the name of the Service, the page is displayed. The three steps located the fault: the Pod answers, the Service does not select it, and once the selector is fixed the client reaches the application.
The other symptoms, summarized
Four symptoms have their own lesson. This page summarizes them in a few lines and points to the lesson concerned.
CrashLoopBackOff
The container starts, stops with an error, and the kubelet waits longer and longer before restarting it. Read kubectl logs <pod> and --previous as in this lesson, then kubectl describe for the exit code. The detail is in CrashLoopBackOff.
ImagePullBackOff
The kubelet cannot download the image: wrong name or tag, unreachable registry, missing credentials. The Pod's events give the cause. The detail is in ImagePullBackOff (in French).
Pending
The scheduler found no node for the Pod: requests too large, taint, missing volume. The FailedScheduling event says why, as in Requests and limits. The detail is in Pending Pods.
OOMKilled
The container exceeded its memory limit: exit code 137 and reason OOMKilled in Last State. The case, the fix and the resize are in Requests and limits.
Clean up the cluster
Delete the namespace: it takes away the Pods, the Service, the ConfigMap, the service account and the Deployment of the demonstration. Also remove the file that contained the uid of the first Pod.
kubectl delete namespace debogage && rm -f uid-avant.txtnamespace "debogage" deletedPut it into practice
Two guided labs reproduce the gestures of the demonstration, for about 25 minutes in total.
In the first one, A Pod blocked by a ConfigMap that does not exist, a Pod does not start and says nothing in its logs. You read the cause in its events, you record it in a file before any repair, then you create the ConfigMap with the expected content. The validation checks that the cause is recorded before the repair, that the ConfigMap carries the right configuration, that the Pod runs without having been recreated and that the application serves its configuration. Here, the ConfigMap is mounted as a volume, the case described at the end of the section on the missing ConfigMap.
In the second one, Get inside a container with no shell using kubectl debug, you attach an ephemeral container to a Pod with no shell. The validation checks that this container shares the processes of the application and that the process list read from it shows those of the application.
Challenge: ship a robust manifest from a set of requirements
The challenge is the unaided proof of the module: it gives a set of requirements, without the commands, and it is graded on the state of the cluster that you leave behind, not on the commands typed. You deliver an application in two variants, dev and prod, from a single Kustomize base, with an init container, a native sidecar, requests and limits and three probes. You schedule a CronJob, you bring a Job to completion, and you repair a failing Pod, after recording its cause. The challenge builds on this lesson, on Init containers and sidecars, Application volumes, Requests and limits, Probes, Jobs and CronJobs and Manage images for the digest.
The set of requirements has ten requirements of equal weight, each checked by a test, and passing requires 70 points out of 100. Three requirements are blocking: they must pass whatever the total. They are the production application available on its image by digest, the three probes set with Pods ready and no restart, and the cause of the fault recorded before the repair. No hint is offered: you have only the specification and what the lesson taught you. Each requirement is worth 10 points, so at least seven requirements met, blocking ones included, reach the threshold of 70.
Key takeaways
- The status of
kubectl get podsgives the first command:kubectl events --for pod/<pod>for a blocked Pod,kubectl logsfor a Pod that crashes. CreateContainerConfigErrormeans that the container was not created. A missing ConfigMap is read in the events:Failedwhen it is read in a variable,FailedMount(Pod inContainerCreating) when it is mounted as a volume. In both cases, the Pod starts on its own as soon as it exists.kubectl logs --previousreads the previous attempt; the kubelet only keeps the last stopped container.- Without
-c,kubectl logschooses the default container and announces it withDefaulted container. command terminated with exit code Nafterkubectl execrepeats the code of the command that was run, not a state of the Pod.exec: "sh": executable file not foundmeans that the image has no shell;kubectl debugadds an ephemeral container, with--targetto see its processes.- The right to use
execis distinct from reading Pods: anexecrequires reading the Pod andcreateonpods/exec, andkubectl auth can-i create pods/exectests this second right. - For an unreachable application, three steps: test the Pod directly, read the selector and the
Endpointsof the Service, test the Service from another Pod.
Test your knowledge
This quiz covers the choice of the first command, the reading of the kubectl exec and kubectl logs messages, and the location of a Service fault. Allow five minutes and aim for 80%; below that, reread "Pod Running but application unreachable".
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Pod Networking: how Pods talk to each other across nodes
A new full quiz with random questions
Work again only on the questions you missed