Kustomize describes an application once, in a base, and adapts it for each environment with overlays that contain only the differences. It is built into kubectl: kubectl apply -k and kubectl kustomize are enough, with no other binary to install. In this lesson you write a base (a Deployment, a Service and a generated ConfigMap), then two overlays, dev and prod, that you deploy into two namespaces. You then see what happens when the configuration changes, when the image changes, and a trap in the environment label that blocks the update of a Deployment.
What you will learn
- Structure one base and two overlays in which only the namespace, the name prefix, the environment label and the number of replicas differ
- Preview an overlay with
kubectl kustomizeandkubectl diff -k, then apply it withkubectl apply -k - Check that both environments come from the same base
- Generate a ConfigMap with
configMapGeneratorand explain the suffix of its name - Change the image of a Deployment from an overlay, without modifying the base
- Explain why a Secret is not written in clear text in a versioned Kustomize file
- Tell Kustomize and Helm apart, and say when to prefer one or the other
Prerequisites
You have read kubectl diff and wait: you read a diff and its exit code, and you follow a rollout with rollout status. You know the Deployment and the Service (lessons of the previous module), as well as ConfigMaps, whose envFrom syntax the base uses. The demo cluster is running on the kind-demo context.
Why use Kustomize rather than copy the folder?
When the same application has to run in dev and in prod, the spontaneous solution is to copy the folder of manifests. A few months later, a fix made in production is missing in dev, and nobody can tell what differs on purpose from what differs by oversight. Kustomize answers with composition: a base holds what is common, and each overlay declares only what it changes.
Kustomize needs no template. It reads valid YAML manifests and transforms them, whereas Helm (in French) generates the manifests from templates and values. Kustomize fits when your manifests already exist and the differences between environments stay simple (number of replicas, image, label, configuration). Helm fits better when you distribute an application to other teams, with many parameters and published versions.
The Kustomize version is the one of kubectl. You read it on the Kustomize Version line of the following command, and it can differ from the one of a kustomize binary installed separately.
kubectl versionClient Version: v1.37.0Kustomize Version: v5.8.1Server Version: v1.37.0In this lesson, kubectl v1.37.0 embeds Kustomize v5.8.1 and the Kubernetes server is v1.37.0.
How do you write the base?
A base is a folder that contains the common manifests and a kustomization.yaml file, the list of what Kustomize must assemble. Create the working tree and the three files of the base.
mkdir -p ~/atelier-k8s/kustomize/base ~/atelier-k8s/kustomize/overlays/dev ~/atelier-k8s/kustomize/overlays/prodcd ~/atelier-k8s/kustomizeThe web Deployment reads its configuration from a ConfigMap named web-config, which the base does not contain as a file: it will be generated.
cat > base/deployment.yaml <<'EOF'apiVersion: apps/v1kind: Deploymentmetadata: name: webspec: replicas: 2 selector: matchLabels: app: web template: metadata: labels: app: web spec: containers: - name: nginx image: nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb ports: - containerPort: 80 envFrom: - configMapRef: name: web-configEOFcat > base/service.yaml <<'EOF'apiVersion: v1kind: Servicemetadata: name: webspec: selector: app: web ports: - port: 80 targetPort: 80EOFcat > base/kustomization.yaml <<'EOF'apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - deployment.yaml - service.yamlconfigMapGenerator: - name: web-config literals: - ENV=baseEOFThe resources field lists the files of the base. The configMapGenerator field asks Kustomize to build a web-config ConfigMap that contains ENV=base. These commands print nothing.
How do you write the dev and prod overlays?
An overlay is a folder that starts from the base (resources: - ../../base) and declares its differences. The two overlays differ only by their values: namespace, name prefix, number of replicas, environment label and ConfigMap content.
cat > overlays/dev/kustomization.yaml <<'EOF'apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - ../../basenamespace: boutique-devnamePrefix: dev-replicas: - name: web count: 1labels: - pairs: env: dev includeSelectors: true includeTemplates: trueconfigMapGenerator: - name: web-config behavior: merge literals: - ENV=devEOFcat > overlays/prod/kustomization.yaml <<'EOF'apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - ../../basenamespace: boutique-prodnamePrefix: prod-replicas: - name: web count: 3labels: - pairs: env: prod includeSelectors: true includeTemplates: trueconfigMapGenerator: - name: web-config behavior: merge literals: - ENV=prodEOFEach field of the overlay has a precise effect.
| Field | Effect |
|---|---|
namespace | Puts all objects in this namespace (Kustomize does not create the namespace) |
namePrefix | Adds a prefix to the name of every object and fixes the internal references |
replicas | Replaces the number of replicas of the named Deployment |
labels | Adds a label to all objects; includeSelectors also pushes it into the selectors |
configMapGenerator with behavior: merge | Modifies the ConfigMap generated by the base instead of creating a second one |
How do you preview an overlay before applying it?
kubectl kustomize <folder> prints the YAML that Kustomize produces, without touching the cluster. It is the reflex to adopt before any apply. Start with dev.
kubectl kustomize overlays/devapiVersion: v1data: ENV: devkind: ConfigMapmetadata: labels: env: dev name: dev-web-config-cbc6h26dgf namespace: boutique-dev---apiVersion: v1kind: Servicemetadata: labels: env: dev name: dev-web namespace: boutique-devspec: ports: - port: 80 targetPort: 80 selector: app: web env: dev---apiVersion: apps/v1kind: Deploymentmetadata: labels: env: dev name: dev-web namespace: boutique-devspec: replicas: 1 selector: matchLabels: app: web env: dev template: metadata: labels: app: web env: dev spec: containers: - envFrom: - configMapRef: name: dev-web-config-cbc6h26dgf image: nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb name: nginx ports: - containerPort: 80Look at four things. The names carry the dev- prefix and the namespace is boutique-dev. The number of replicas went from 2 to 1. The env: dev label is present on the three objects, but also in the Service selector, in the Deployment selector and in the Pod labels, which is intended here: the Service keeps finding its Pods. Finally, the ConfigMap is called dev-web-config-cbc6h26dgf, with a suffix you did not write, and the Deployment references it under that full name: Kustomize rewrote the reference itself.
Do the same for prod.
kubectl kustomize overlays/prodapiVersion: v1data: ENV: prodkind: ConfigMapmetadata: labels: env: prod name: prod-web-config-679b259276 namespace: boutique-prod---apiVersion: v1kind: Servicemetadata: labels: env: prod name: prod-web namespace: boutique-prodspec: ports: - port: 80 targetPort: 80 selector: app: web env: prod---apiVersion: apps/v1kind: Deploymentmetadata: labels: env: prod name: prod-web namespace: boutique-prodspec: replicas: 3 selector: matchLabels: app: web env: prod template: metadata: labels: app: web env: prod spec: containers: - envFrom: - configMapRef: name: prod-web-config-679b259276 image: nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb name: nginx ports: - containerPort: 80The structure is identical: only the prefix, the namespace, the replicas (3), the label, the ConfigMap content and its suffix differ. The container, the port and the image come from the same base.
What does labels really do with includeSelectors?
The labels field always adds the label to the metadata of the objects. Its two options decide whether it also goes into the selectors and into the Pod labels. includeTemplates only changes the labels of the template. includeSelectors changes the selectors, and it also changes those same template labels. The table shows the kubectl kustomize output depending on the options.
| Options | metadata.labels | Service selector | Deployment selector | Template labels |
|---|---|---|---|---|
| none | yes | no | no | no |
includeTemplates: true alone | yes | no | no | yes |
includeSelectors: true alone | yes | yes | yes | yes |
| both (this lesson) | yes | yes | yes | yes |
The last two rows give the same result: includeSelectors: true already pushes the label into the template labels, so includeTemplates: true adds nothing here. Above all, remember that includeSelectors: true modifies the selector of the Deployment, a field that can no longer change after creation. The section "What trap does includeSelectors hide when the label changes?" shows what this costs.
How do you apply the overlays in two namespaces?
The Kustomize namespace field puts objects in a namespace, but does not create it. So create both namespaces, then read what kubectl diff -k announces for dev.
kubectl create namespace boutique-dev && kubectl create namespace boutique-prodnamespace/boutique-dev creatednamespace/boutique-prod createdkubectl diff -k overlays/dev; echo "code de sortie : $?"The echo messages of this lesson are in French on purpose, so that the outputs stay those of the tested cluster: "code de sortie" means "exit code".
diff -u -N /tmp/LIVE-110928176/apps.v1.Deployment.boutique-dev.dev-web /tmp/MERGED-2977931757/apps.v1.Deployment.boutique-dev.dev-web@@ -0,0 +1,48 @@diff -u -N /tmp/LIVE-110928176/v1.ConfigMap.boutique-dev.dev-web-config-cbc6h26dgf /tmp/MERGED-2977931757/v1.ConfigMap.boutique-dev.dev-web-config-cbc6h26dgf@@ -0,0 +1,11 @@diff -u -N /tmp/LIVE-110928176/v1.Service.boutique-dev.dev-web /tmp/MERGED-2977931757/v1.Service.boutique-dev.dev-web@@ -0,0 +1,28 @@code de sortie : 1The diff announces three entirely new objects (@@ -0,0 +1,N @@) and the code 1, which means "differences", as in the previous lesson: nothing exists yet. Now apply both overlays, then wait for both deployments.
kubectl apply -k overlays/dev && kubectl apply -k overlays/prodconfigmap/dev-web-config-cbc6h26dgf createdservice/dev-web createddeployment.apps/dev-web createdconfigmap/prod-web-config-679b259276 createdservice/prod-web createddeployment.apps/prod-web createdkubectl rollout status deployment/dev-web -n boutique-dev --timeout=90s && kubectl rollout status deployment/prod-web -n boutique-prod --timeout=90sWaiting for deployment "dev-web" rollout to finish: 0 of 1 updated replicas are available...deployment "dev-web" successfully rolled outWaiting for deployment "prod-web" rollout to finish: 0 of 3 updated replicas are available...Waiting for deployment "prod-web" rollout to finish: 1 of 3 updated replicas are available...Waiting for deployment "prod-web" rollout to finish: 2 of 3 updated replicas are available...deployment "prod-web" successfully rolled outBoth environments are running. It remains to compare what they contain.
How do you compare the two environments?
The env label acts as a filter: the -l 'env in (dev,prod)' option keeps only the objects created by the overlays, and -L env adds a column with the value of the label.
kubectl get deployments,services,configmaps -A -l 'env in (dev,prod)' -L envNAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE ENVboutique-dev deployment.apps/dev-web 1/1 1 1 1s devboutique-prod deployment.apps/prod-web 3/3 3 3 1s prod
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ENVboutique-dev service/dev-web ClusterIP 10.96.17.165 <none> 80/TCP 1s devboutique-prod service/prod-web ClusterIP 10.96.37.29 <none> 80/TCP 1s prod
NAMESPACE NAME DATA AGE ENVboutique-dev configmap/dev-web-config-cbc6h26dgf 1 1s devboutique-prod configmap/prod-web-config-679b259276 1 1s prodThere is one Deployment, one Service and one ConfigMap per namespace. The kube-root-ca.crt ConfigMap, which Kubernetes creates in every namespace, does not appear: it does not carry the label. Now compare what must stay common, the number of replicas and the image. The column titles are in French in the command (NOM means NAME), so that the output stays the one of the tested cluster.
kubectl get deployments -A -l 'env in (dev,prod)' -o 'custom-columns=NS:.metadata.namespace,NOM:.metadata.name,REPLICAS:.spec.replicas,IMAGE:.spec.template.spec.containers[0].image'NS NOM REPLICAS IMAGEboutique-dev dev-web 1 nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edbboutique-prod prod-web 3 nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edbThe replicas differ (1 and 3), but the image is the same, since it comes from the base. It remains to check that each Deployment reads its own configuration.
kubectl exec -n boutique-dev deployment/dev-web -- printenv ENVkubectl exec -n boutique-prod deployment/prod-web -- printenv ENVdevprodEach Deployment reads the ConfigMap of its environment, under a name that Kustomize rewrote on its own.
Why does the ConfigMap name change with its content?
The suffix of the name of a generated ConfigMap is a hash of its content. When you change a value, the name changes, so the Deployment references a new name, and Kubernetes replaces its Pods: the new configuration is taken into account without you restarting anything. A ConfigMap modified in place would not trigger this restart. Change the dev value and apply.
sed -i 's/ENV=dev/ENV=dev-v2/' overlays/dev/kustomization.yaml && kubectl apply -k overlays/devconfigmap/dev-web-config-fc86gm9t59 createdservice/dev-web unchangeddeployment.apps/dev-web configuredLook at the first line: a new dev-web-config-fc86gm9t59 ConfigMap is created, with a different suffix, and the Deployment is configured because its Pod template now references that name. The Service is unchanged. Check the result.
kubectl rollout status deployment/dev-web -n boutique-dev --timeout=90s && kubectl get configmap -n boutique-dev && kubectl exec -n boutique-dev deployment/dev-web -- printenv ENVWaiting for deployment "dev-web" rollout to finish: 1 old replicas are pending termination...Waiting for deployment "dev-web" rollout to finish: 1 old replicas are pending termination...deployment "dev-web" successfully rolled outNAME DATA AGEdev-web-config-cbc6h26dgf 1 2sdev-web-config-fc86gm9t59 1 1skube-root-ca.crt 1 2sdev-v2The Pod restarted and printenv ENV returns dev-v2. Note that the old dev-web-config-cbc6h26dgf ConfigMap still exists: apply does not delete an object that has disappeared from the output. To clean up, delete the namespace, as at the end of the lesson.
How do you change the image of an environment without touching the base?
The images field of an overlay replaces the image of a container, by its image name, without modifying the Deployment of the base. Add it to the prod overlay, with a digest, the fingerprint that designates a precise version of an image, whereas a tag can be moved to another version. This digest is the one of the nginx:1.29 image, which differs from the base image, so that the change is visible.
cat >> overlays/prod/kustomization.yaml <<'EOF'images: - name: nginx digest: sha256:1881968aff6f7cdcc4b888c00a11f4ce241ad7ec957e0cb4a9e19e93a3ff87eaEOFThe command prints nothing. Look at what Kustomize produces for the image.
kubectl kustomize overlays/prod | grep 'image:' image: nginx@sha256:1881968aff6f7cdcc4b888c00a11f4ce241ad7ec957e0cb4a9e19e93a3ff87eaWith digest: alone, the tag of the base disappears: the reference becomes nginx@sha256:..., with no readable version. Keep this in mind if a team reads the manifests: the version no longer appears in them. Now preview the change before applying it.
kubectl diff -k overlays/prod; echo "code de sortie : $?"diff -u -N /tmp/LIVE-1108288391/apps.v1.Deployment.boutique-prod.prod-web /tmp/MERGED-1808291598/apps.v1.Deployment.boutique-prod.prod-web--- /tmp/LIVE-1108288391/apps.v1.Deployment.boutique-prod.prod-web 2026-11-11 14:53:29.943080641 +0000+++ /tmp/MERGED-1808291598/apps.v1.Deployment.boutique-prod.prod-web 2026-11-11 14:53:29.947080963 +0000@@ -6,7 +6,7 @@ kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"env":"prod"},"name":"prod-web","namespace":"boutique-prod"},"spec":{"replicas":3,"selector":{"matchLabels":{"app":"web","env":"prod"}},"template":{"metadata":{"labels":{"app":"web","env":"prod"}},"spec":{"containers":[{"envFrom":[{"configMapRef":{"name":"prod-web-config-679b259276"}}],"image":"nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb","name":"nginx","ports":[{"containerPort":80}]}]}}}} creationTimestamp: "2026-11-11T14:53:27Z"- generation: 1+ generation: 2 labels: env: prod name: prod-web@@ -36,7 +36,7 @@ - envFrom: - configMapRef: name: prod-web-config-679b259276- image: nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edb+ image: nginx@sha256:1881968aff6f7cdcc4b888c00a11f4ce241ad7ec957e0cb4a9e19e93a3ff87ea imagePullPolicy: IfNotPresent name: nginx ports:code de sortie : 1The only change that comes from you is the image line, with the generation counter going from 1 to 2 as usual. Apply and wait for the end of the rollout.
kubectl apply -k overlays/prod && kubectl rollout status deployment/prod-web -n boutique-prod --timeout=90sconfigmap/prod-web-config-679b259276 unchangedservice/prod-web unchangeddeployment.apps/prod-web configuredWaiting for deployment "prod-web" rollout to finish: 0 out of 3 new replicas have been updated...deployment "prod-web" successfully rolled outThe ConfigMap and the Service are unchanged: only the Deployment changed. Check that the base did not move and that dev keeps its image.
kubectl get deployments -A -l 'env in (dev,prod)' -o 'custom-columns=NS:.metadata.namespace,NOM:.metadata.name,IMAGE:.spec.template.spec.containers[0].image'NS NOM IMAGEboutique-dev dev-web nginx:1.30.5@sha256:9bf97bd7714f5e24c1ccd545ecb9eb5435cb6d109c97cebb15e7e455e0239edbboutique-prod prod-web nginx@sha256:1881968aff6f7cdcc4b888c00a11f4ce241ad7ec957e0cb4a9e19e93a3ff87eaprod-web runs on the image of the overlay, dev-web on the one of the base, and the base/deployment.yaml file was never modified. For a finer change to a field, Kustomize also offers patches, which applies a targeted fix to an object of the base; images, replicas and labels already cover the most common differences.
What trap does includeSelectors hide when the label changes?
The env label entered the spec.selector of the Deployment through includeSelectors: true. Yet this selector can no longer be modified after creation. Rename the label value in the dev overlay, from dev to developpement, and ask for a server-side dry-run apply.
sed -i 's/env: dev/env: developpement/' overlays/dev/kustomization.yaml && kubectl apply -k overlays/dev --dry-run=serverconfigmap/dev-web-config-fc86gm9t59 configured (server dry run)service/dev-web configured (server dry run)The Deployment "dev-web" is invalid: spec.selector: Invalid value: {"matchLabels":{"app":"web","env":"developpement"}}: field is immutableRead the three lines in order. The Deployment is rejected, because its spec.selector is immutable, but it is not the first one cited: the ConfigMap and the Service are accepted before it.
The selector of a Service, on the other hand, can change. Without --dry-run=server, the Service would therefore be rewritten with the new label while the Deployment would stay rejected: the Service would no longer find any Pod. So always preview with --dry-run=server or kubectl diff -k before applying an overlay in which a selector label changes. To change a selector label, you have to recreate the Deployment.
Go back to the original value and check that the file matches the cluster again.
sed -i 's/env: developpement/env: dev/' overlays/dev/kustomization.yaml && kubectl diff -k overlays/dev; echo "code de sortie : $?"code de sortie : 0The code 0 confirms that the folder and the cluster are identical again.
Why not write a Secret in clear text in Kustomize?
Kustomize offers secretGenerator, which builds a Secret the way configMapGenerator builds a ConfigMap. But if you write the value of the Secret in the kustomization.yaml, it ends up in clear text in a file that you version: anyone who reads the repository, or its history, reads the password. A Kubernetes Secret itself is only a base64 encoding, not encryption. So keep sensitive values out of the repository, or encrypted. The options (External Secrets, Sealed Secrets, SOPS) are described in Protect the Secrets of a Kubernetes cluster (in French), and Flux CD and SOPS (in French) shows a complete case.
What about a GitOps tool?
GitOps tools read a Kustomize folder directly: you push the base and the overlays to Git, and the tool applies the overlay of each environment in place of your kubectl apply -k. The pages GitOps: principles, ArgoCD vs Flux, ArgoCD multi-environment with Kustomize (in French) and Flux CD, structuring your repositories (in French) detail this organization.
Clean up the cluster
Delete both namespaces: they take with them the Deployments, the Services and all the ConfigMaps, old ones included. A kubectl delete -k, on the other hand, would leave the old dev-web-config-cbc6h26dgf ConfigMap, since it is no longer in the output of the overlay.
kubectl delete namespace boutique-dev boutique-prod && rm -rf ~/atelier-k8s/kustomizenamespace "boutique-dev" deletednamespace "boutique-prod" deletedTroubleshooting
Kustomize errors are almost always read in the output of kubectl kustomize, before any apply. When an object does not have the expected effect, read this output again first.
| Symptom | Likely cause | Solution |
|---|---|---|
field is immutable on spec.selector | A selector label changed while includeSelectors: true pushed it into the Deployment | Keep the label stable, or recreate the Deployment; preview with --dry-run=server |
The Service no longer finds any Pod after an apply | The Service was rewritten while the Deployment was rejected | Go back to the old label and apply again; preview before applying |
apply -k fails because the namespace is not found | The namespace field does not create the namespace | kubectl create namespace before the first apply |
| An old ConfigMap stays in the namespace | apply does not delete an object that left the output | Delete it by hand, or delete the namespace |
| The image version is no longer readable | images: with digest: alone removes the tag | Also add newTag: if you want to keep the version in the manifest |
Put it into practice
The lab has you describe an application once in a Kustomize base with a generated ConfigMap, then deploy it into two namespaces with two overlays: the number of replicas, the name prefix and the environment label differ, and only one of the environments changes image version, through its overlay and without modifying the base. The validation covers the state of the cluster: replicas available in each environment, label on the Pods, Services that find their Pods, same container and same port, generated ConfigMap, shared base, and cluster matching the folder.
Key takeaways
- A base holds the common manifests, an overlay declares only its differences (
namespace,namePrefix,replicas,labels,images, generators). - Kustomize is built into
kubectl:kubectl kustomizeprints the output,kubectl diff -kcompares it with the cluster,kubectl apply -kapplies it. - The Kustomize
namespacefield does not create the namespace. - The name of a generated ConfigMap carries a hash of its content: changing the value renames the ConfigMap and redeploys the Pods, but leaves the old ConfigMap in place.
images:changes the image from the overlay without touching the base, and withdigest:alone the tag disappears.includeSelectors: truepushes the label into the selectors: changing it afterwards gets the Deployment rejected, while the Service is already rewritten.- A Secret is not written in clear text in a versioned file.
- Kustomize fits simple differences between environments, Helm fits applications distributed with many parameters.
Test your knowledge
These questions check that you can read the output of an overlay, predict the effect of a ConfigMap or image change, and diagnose the rejection of a selector.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Manage container images in Kubernetes
A new full quiz with random questions
Work again only on the questions you missed