cert-manager automates the issuing and renewal of your applications' TLS certificates in Kubernetes. You describe the certificate you want in a Certificate object, cert-manager requests it from an authority (an internal CA, Let's Encrypt through the ACME protocol, Vault...), stores the result in a Secret and renews it before it expires. This lesson covers the certificates your services present to their clients; the next one, Cluster PKI, covers those of the control plane. It is optional and is meant for whoever administers a cluster and wants HTTPS without watching expiry dates.
What you will learn
- Say what cert-manager adds to the cluster PKI and read its three objects.
- Install cert-manager and issue a first certificate with a self-signed Issuer.
- Create an internal CA and sign a service certificate with it.
- Get a Let's Encrypt certificate through ACME and secure an Ingress with an annotation.
- Diagnose a certificate that stays
not ready.
Prerequisites
- A working Kubernetes cluster and a configured
kubectl. The demonstration runs on the course's kind cluster, the lab on the catalog's kubeadm cluster. - Having followed the Ingress lesson, which has you create a TLS Secret and a certificate with
openssl. The notions of authority and chain are recalled in the first section. - For the Let's Encrypt part only: a public domain name that points to the cluster. The lab does not need one.
What does cert-manager add to the cluster PKI?
A PKI (public key infrastructure) is the set of authorities and certificates that let programs trust each other. A certificate authority (CA) signs certificates with its private key; a client that trusts this authority accepts any certificate it has signed. The chain of trust is the verifiable link from a certificate up to the authority that signed it. TLS diagnostics with openssl goes deeper into these notions.
The cluster PKI, managed by kubeadm, serves authentication inside the cluster: it signs the certificates of the Kubernetes components, and those of users through a signing request (CertificateSigningRequest). It does not sign the certificates your services present to their clients. cert-manager does not replace it: it adds resources that issue these application certificates and renew them without human action.
You no longer handle openssl or .pem files: you declare what you want, and a controller (a program that keeps comparing the desired state with the actual state) does the work. Three objects are enough to get started.
| Object | Role | Scope |
|---|---|---|
Issuer | A source of certificates (CA, ACME, Vault...) | One namespace only |
ClusterIssuer | Same, but usable from the whole cluster | Whole cluster |
Certificate | The request for a concrete certificate (names, lifetime) | One namespace |
An Issuer only issues certificates to the Certificate objects of its own namespace. A ClusterIssuer serves every namespace: it is the usual choice for a company CA or a shared Let's Encrypt issuer.
When you create a Certificate, cert-manager runs through a chain of objects internally that you do not create yourself. The diagram shows it: the Certificate produces a CertificateRequest, which the issuer signs, and the result lands in a Secret of type kubernetes.io/tls. With ACME, the issuer also goes through Order and Challenge objects. These intermediate objects let you follow and diagnose an issuance, and the troubleshooting section works back through them one by one.
How do you install cert-manager?
Installation goes through a single manifest that deploys the three controllers (cert-manager, cert-manager-webhook and cert-manager-cainjector) and the CRDs, that is, the definitions of the new objects. Pin a precise version rather than a floating one. This lesson uses 1.21.3, the one the lab installs, with the server-side apply that the lab uses.
kubectl apply --server-side --force-conflicts \ -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.3/cert-manager.yamlThen wait for the deployments to be available, and read the controller image to confirm the version.
kubectl -n cert-manager wait --for=condition=Available deployment --all --timeout=300skubectl -n cert-manager get deploy cert-manager \ -o jsonpath='{.spec.template.spec.containers[0].image}'The first command returns once cert-manager, cert-manager-webhook and cert-manager-cainjector are available. The second must print an image whose tag is v1.21.3.
cert-manager validates every resource through an admission webhook. An Issuer created right after installation can therefore fail with failed calling webhook: this is why you wait for the webhook's Available condition before going on.
The compatibility matrix of version 1.21 lists Kubernetes 1.33 to 1.36, while the course teaches 1.37. The lab of this lesson was played on a 1.37.2 cluster with no discrepancy. Read the official matrix for the version you install before deploying it to a production cluster.
How do you issue a first certificate with a self-signed Issuer?
A self-signed issuer (selfSigned) depends on no authority: each certificate is signed with its own key. No browser trusts it, so it is not for production. It is used to validate the installation, and to bootstrap an internal CA in the next section.
Create a self-signed ClusterIssuer, then a test Certificate.
apiVersion: cert-manager.io/v1kind: ClusterIssuermetadata: name: selfsignedspec: selfSigned: {}---apiVersion: cert-manager.io/v1kind: Certificatemetadata: name: test-cert namespace: defaultspec: secretName: test-cert-tls issuerRef: name: selfsigned kind: ClusterIssuer commonName: test.example.com dnsNames: - test.example.comApply the file and read the certificate's Ready condition.
kubectl apply -f first-certificate.yamlkubectl get certificate test-cert \ -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'The expected result is True: the test-cert-tls Secret then holds a private key and a certificate. To read the certificate, decode the tls.crt field of the Secret and pass it to openssl:
kubectl get secret test-cert-tls -o jsonpath='{.data.tls\.crt}' \ | base64 -d | openssl x509 -noout -subject -issuer -datesThe output gives the subject, the issuer and the validity dates. Here the subject and the issuer are identical, since the certificate signed itself.
How do you create an internal CA to sign your services?
A private network does not need Let's Encrypt: a single internal certificate authority can sign all the services. The principle comes down to three steps. The self-signed issuer creates a root CA certificate, a second issuer uses that certificate as its authority, and the application certificates are then signed by this second issuer.
-
Create the root CA with
isCA: true, signed by the self-signed issuer:internal-ca.yaml apiVersion: cert-manager.io/v1kind: Certificatemetadata:name: ca-rootnamespace: cert-managerspec:isCA: truecommonName: mon-lab-casecretName: mon-lab-ca-secretissuerRef:name: selfsignedkind: ClusterIssuer -
Declare a ClusterIssuer of type
cathat signs with the Secret of this CA:apiVersion: cert-manager.io/v1kind: ClusterIssuermetadata:name: mon-lab-caspec:ca:secretName: mon-lab-ca-secret -
Issue a service certificate, with several names (SANs) and an early renewal:
apiVersion: cert-manager.io/v1kind: Certificatemetadata:name: app-certnamespace: defaultspec:secretName: app-cert-tlsissuerRef:name: mon-lab-cakind: ClusterIssuercommonName: app.interne.labdnsNames:- app.interne.lab- api.interne.labduration: 2160h # 90 daysrenewBefore: 360h # renews 15 days before expiry
The CA Secret of a ClusterIssuer lives in the namespace of cluster resources, which is cert-manager by default. The root CA is therefore created there. The lab of this lesson uses Issuer objects rather than ClusterIssuer objects: the CA and its Secret then live in the working namespace, and the reasoning is the same.
A Ready certificate is not necessarily signed by the right authority. Check the chain with openssl verify, against your CA's public certificate.
kubectl -n cert-manager get secret mon-lab-ca-secret \ -o jsonpath='{.data.tls\.crt}' | base64 -d > ca.pemkubectl get secret app-cert-tls -o jsonpath='{.data.tls\.crt}' \ | base64 -d > app.pemopenssl verify -CAfile ca.pem app.pemopenssl verify answers OK when app.pem leads to ca.pem. For browsers and curl to trust the signed services, distribute this public ca.pem to the client machines. Never distribute the private key, which stays in the Secret.
How do you get a Let's Encrypt certificate with ACME?
For a service exposed on the Internet, Let's Encrypt issues browser-trusted certificates for free. cert-manager speaks the ACME protocol and proves that you control the domain with a challenge. The ACME certificates guide details the protocol itself; here we only look at what it changes in cert-manager.
Two challenges exist. With HTTP-01, cert-manager publishes a temporary file under http://your-domain/.well-known/acme-challenge/: it is simple, but the domain must be publicly reachable over HTTP. With DNS-01, it creates a TXT record in your DNS zone: it is more complex, but it is the only way to get a wildcard certificate (*.example.com), and the service does not need to be exposed.
Let's Encrypt applies strict quotas in production. Tune your configuration on the staging endpoint (acme-staging-v02), which issues untrusted certificates but with no tight quota, then switch to production once issuance works.
Here is a Let's Encrypt ClusterIssuer with an HTTP-01 challenge, solved through the course's traefik Ingress class:
apiVersion: cert-manager.io/v1kind: ClusterIssuermetadata: name: letsencrypt-prodspec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: admin@example.com privateKeySecretRef: name: letsencrypt-prod-account solvers: - http01: ingress: ingressClassName: traefikFor a wildcard, replace the solver with a DNS-01 challenge tied to your DNS provider. The configuration depends on the provider; the cert-manager documentation lists the supported DNS-01 solvers. Neither this ClusterIssuer nor the Ingress of the next section is replayed in the lab, for lack of public access to the cluster.
How do you secure an Ingress automatically?
One annotation is enough for an Ingress to obtain and then renew its certificate. Add cert-manager.io/cluster-issuer with the issuer's name, and a tls block that names the Secret to fill.
apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: mon-app annotations: cert-manager.io/cluster-issuer: letsencrypt-prodspec: ingressClassName: traefik tls: - hosts: - app.example.com secretName: app-example-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: mon-app port: number: 80cert-manager spots the annotation, creates the matching Certificate, solves the challenge and fills the app-example-tls Secret. The Ingress then serves traffic over HTTPS. Annotations depend on the Ingress controller, but this one is read by cert-manager itself; the full Ingress configuration is in the Kubernetes Ingress lesson.
How does cert-manager renew certificates?
cert-manager watches every certificate and requests a new one before it expires, with no cron or script on your side. The renewBefore field sets the margin; without it, the default margin is one third of the lifetime. To read a certificate's expiry:
kubectl get certificate app-cert -o jsonpath='{.status.notAfter}'At renewal time, cert-manager updates the same Secret. An Ingress controller reloads the new certificate by itself. For a Pod that mounts the Secret as a volume, the kubelet updates the mounted file after a short delay, unless the mount goes through subPath, which is never updated; the application must then re-read the file, which many only do on restart: check the behavior of yours.
Why does a certificate stay not ready, and how do you diagnose it?
When a Certificate does not reach Ready, the cause is not always visible on it: you have to work back through the chain of objects in the diagram, from the nearest to the farthest. A simple case to provoke is a Certificate whose issuerRef names an issuer that does not exist: this is the failure the lab sets up. The certificate then stays without Ready, and the cause shows in its events and in its CertificateRequest.
# 1. the certificate itself: Ready condition and eventskubectl describe certificate app-cert# 2. the underlying requestkubectl describe certificaterequest# 3. for ACME: the order and the challengekubectl describe orderkubectl describe challenge# 4. the namespace eventskubectl get eventsIn each describe, read the Ready condition and the Events section at the end of the output: they say which object blocks and why. The table links the most common symptoms to their cause.
| Symptom | Likely cause | Fix |
|---|---|---|
failed calling webhook | Webhook not ready yet | Wait for the webhook's Available condition |
Certificate without Ready, issuer not found | issuerRef points to an issuer that does not exist | Fix issuerRef (name and kind) or create the issuer |
HTTP-01 challenge pending | Domain not publicly reachable over HTTP | Check the DNS and the exposure of the Ingress |
rateLimited on ACME | Let's Encrypt production quota reached | Go back to staging while you test |
| CA Secret missing | The root CA is not Ready | Fix the CA's Certificate first |
How do you protect cert-manager and the keys it handles?
cert-manager handles private keys: treat it as a sensitive component. The first reflex is to keep it up to date. Version 1.20.3 fixed a HIGH severity flaw (GHSA-8rvj-mm4h-c258): the aggregated cert-manager-edit ClusterRole wrongly granted the right to create ACME Challenge and Order resources, which made it possible to hijack the configuration of a solver. The 1.21.3 of this lesson is later than that fix.
A kubernetes.io/tls Secret holds the service's private key: restrict the Role objects that can read it. The internal CA's Secret needs even more care, since it can sign any certificate: minimal access and a backup outside the cluster. Finally, separate the issuers by use (staging, production, internal CA) to partition their scopes. An authority driven from outside the cluster is also possible: cert-manager can rely on Vault PKI (in French) as a provider.
Put it into practice
The lab replays the core of this lesson on the catalog's kubeadm cluster, where cert-manager 1.21.3 is installed for you. You create a self-signed issuer auto-signe, an internal CA (a ca-interne Certificate with isCA, then an Issuer of the same name), the certificate of a vitrine-tls service signed by it, then you repair the api-tls certificate, left not ready because it targets a non-existent issuer. The tests read the issuers and the certificates, then check each chain with openssl verify against the internal CA: a certificate that is ready but signed by the wrong issuer does not pass. Let's Encrypt stays out of the lab, for lack of public access.
Key takeaways
- cert-manager automates the issuing and renewal of your applications' TLS certificates; it complements the cluster PKI without replacing that of the control plane.
IssuerandClusterIssuerare the source of certificates,Certificateis the request: the result lands in akubernetes.io/tlsSecret.- The self-signed issuer validates the installation, the internal CA signs your private services and is checked with
openssl verify. - Let's Encrypt through ACME issues public certificates: HTTP-01 for a reachable domain, DNS-01 for a wildcard. Test on staging first.
- The
cert-manager.io/cluster-issuerannotation on an Ingress is enough to switch it to HTTPS. - A
not readycertificate is diagnosed by working back throughCertificate,CertificateRequest,OrderthenChallenge.
Knowledge check
Eight prediction and diagnosis questions: the role of each object, reading a signing chain, choosing an ACME challenge and working back through a not ready certificate.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Kubernetes cluster PKI: authorities and certificates
A new full quiz with random questions
Work again only on the questions you missed