Skip to content
Stéphane Robert
Français
Account: not signed in, sync available My learning
Select theme
Français
Lire en français

Verify client and backend identities with Traefik

By updated

Improve this page

What would you like to report?

The issue opens pre-filled with this page title and URL.

Traefik logo

For Traefik to know who it is talking to, you must verify an identity in each direction: require a certificate from the client with a TLS option clientAuth, and check the backend's with a serversTransport that names the expected certificate authority. This lesson is for anyone who already publishes a service behind Traefik 3.7 and wants a sensitive route to accept only identified callers, instead of trusting everything that arrives and everything that answers. It also teaches you not to confuse an identity (who is this caller) with an authorization (what it is allowed to do). The option names come from the Traefik 3.7 documentation; the behaviors described come from the training lab and from a reading of the code, on Traefik 3.7.15.

What you will learn

The lesson starts from a Traefik that encrypts without verifying anything and ends at a Traefik whose every refusal is proven.

  • Tell an identity from an authorization, and server authentication from client authentication.
  • Require a valid client certificate on a route with clientAuth.
  • Make Traefik verify a backend's certificate with a serversTransport.
  • Explain why insecureSkipVerify disables the proof instead of fixing the error.
  • Restrict the trust given to X-Forwarded-For headers to the upstream proxy alone.
  • Prove a refusal with the HTTP code and the curl exit code, and rotate an authority without cutting access.

Prerequisites

This lesson deals with identities, not with obtaining the server certificate or with password protection, both covered elsewhere.

What is an identity, and how does it differ from an authorization?

An identity answers the question "who are you?", an authorization answers "are you allowed to do this?", and a certificate only proves the first. A certificate is a digital document that binds a name to a cryptographic key. It is signed by a certificate authority (CA), whose signature guarantees that binding. Whoever receives a certificate checks the chain (was the certificate signed by an authority I chose as a reference?), the validity date, and that the peer really holds the associated private key.

If these checks pass, the receiver knows the identity of its peer. It does not yet know what it should allow, because authorization is another decision. Think of a building badge: a genuine badge proves you are an employee, it does not open the server room for you. With Traefik, a client certificate signed by the right authority gets the connection admitted; what its holder can then ask for is decided elsewhere, in the routing, in the route's protections and in the application.

Two distinct mechanisms hide behind the word "TLS". Server authentication is that of any HTTPS: the client verifies the server's certificate. Client authentication is the reverse: the server requires a certificate and verifies it. When both take place, it is called mTLS (mutual TLS). Each direction has its own certificate, its own authority and its own configuration: fixing one does not fix the other.

What are the two verification directions behind Traefik?

Traefik holds two independent TLS connections, one with the client and one with the backend, and each has its own verification, in a different direction. On the client side, Traefik is the server: it can require a certificate from the caller, with a TLS option. On the backend side, Traefik is the client: it can verify the certificate the backend presents to it, with a serversTransport, the configuration block that describes how Traefik connects to a service's servers.

The diagram below places the two directions side by side. Look at the two highlighted boxes: they are the two places in the configuration where the certificate verifications are decided. The dotted arrows show how a refusal shows up: during the TLS handshake (the initial exchange that establishes the encrypted connection) for the client, through an error response from Traefik for the backend.

Diagram of two verifications around Traefik: on the left, the client presents its certificate and the entry point's TLS option (clientAuth, caFiles, clientAuthType) checks it; on the right, Traefik connects to the backend and its ServersTransport (rootCAs, serverName) checks the backend's certificate. A refusal on the client side happens during the TLS handshake, a refusal on the backend side gives a 5xx response.Direction 2 · the backendproves who it isTraefikDirection 1 · the client proveswho it isrefused during the TLShandshakeupstream connection refused,5xx answerClientpresents its certificateEntry point TLS optionclientAuth: caFiles + clientAuthTypeTraefikServersTransportrootCAs, serverNameBackendpresents its certificate
Two independent verifications: the TLS option checks the client, the ServersTransport checks the backend. Source · SVG
Esc to close, + and − to zoom, scroll to move around. 100 %

A third question is not a matter of certificates but of trust: whom to believe when a request announces its origin address in an X-Forwarded-For header? It also concerns an identity, the one the caller announces, but through another mechanism: the connection's source address, not a certificate. The complete path of a request is described in Publish a web service.

How do you require a client certificate on a route?

You declare in the dynamic configuration a TLS option whose clientAuth section names the clients' authority (caFiles) and the level of requirement (clientAuthType: RequireAndVerifyClientCert), then you attach it to the router (the routing rule that directs requests to a service). The caFiles file must contain only the authorities that sign client certificates: neither the server certificate's authority, nor the backends'. It is a rule of prudence, not a Traefik constraint: mixing them would build a chain of trust that nobody decided.

Here is the shape of the option and of its attachment to the router, with neutral names. This excerpt was not replayed as is for this page: it is the lab that has you measure the behavior.

tls:
  options:
    clients-internes:
      clientAuth:
        caFiles:
          - /etc/traefik/pki/ca-clients.crt
        clientAuthType: RequireAndVerifyClientCert


http:
  routers:
    facturation:
      rule: Host(`facturation.corp.test`)
      entryPoints:
        - websecure
      tls:
        options: clients-internes@file
      service: facturation

The requirement level is the setting that misleads. The Traefik 3.7 documentation defines five values for clientAuthType, and only one meets all the conditions of a real check:

ValueWhat Traefik does
NoClientCertIgnores any client certificate
RequestClientCertRequests a certificate, continues if there is none
RequireAnyClientCertRequires a certificate, without verifying who signed it
VerifyClientCertIfGivenVerifies the certificate if there is one, continues otherwise
RequireAndVerifyClientCertRequires a certificate signed by an authority in caFiles

The trap is that caFiles can be present without requiring anything: with NoClientCert, the configuration looks like mTLS and lets everybody in. RequireAnyClientCert is barely better: it requires a certificate, but a certificate forged by anyone passes, so the identity is not proven. Only RequireAndVerifyClientCert refuses the three cases that must be refused: a missing certificate, a foreign authority and an expired certificate, which the lab test checks. VerifyClientCertIfGiven suits a route open to everyone that recognizes those who introduce themselves, without granting anything sensitive.

Why does a valid client certificate not grant every right?

Because the verification only answers "this connection comes from a recognized holder", and the access decision is made at another level. Two illustrations show it. The first comes from the lab: a reverse proxy (a server placed in front of Traefik, which relays requests) presents its own client certificate, signed by the same authority as a user's. Both are admitted, but Traefik believes the address announced by the proxy and not the one announced by a direct user. This difference is decided on the connection's source address (the trustedIPs setting, explained below), not on the certificate: two verified identities, but different authorizations.

The second is the scope. A TLS option applies to the routers that reference it, and any holder of a certificate from that authority reaches all those routes. If two populations must not mix, for example operations staff and an external partner, they share neither authority nor option: you declare one option per population. This choice is a design judgment, not a Traefik rule.

For the other levels of decision, such as address filtering, delegated authentication or rate limiting, Protecting the Traefik entry point describes them. The principle of mTLS between services is presented in mTLS pod-to-pod (in French).

How does Traefik verify a backend's certificate?

You declare a serversTransport that names the backend's authority in rootCAs, and you reference it in the service, whose servers are reached over https://. Without it, Traefik can speak TLS to its backend without ever checking who it is. Here is the shape of the transport and of its reference in the service, with neutral names: this excerpt was not replayed as is for this page.

http:
  serversTransports:
    verif-facturation:
      rootCAs:
        - /etc/traefik/pki/ca-backends.crt
      serverName: facturation.corp.test
  services:
    facturation:
      loadBalancer:
        serversTransport: verif-facturation
        servers:
          - url: https://facturation.interne:8443

The two options each have a role. According to the Traefik 3.7 documentation, rootCAs is the set of authorities to use to verify the servers' certificates, and serverName sets the name used as SNI (Server Name Indication, the host name the client announces at the start of the handshake so that the server picks the right certificate).

The name matters as much as the authority. When absent, serverName lets Traefik compare the certificate's name to the host in the backend's URL. When set, it replaces that host: on Traefik 3.7.15, a serverName that differs from the certificate's name makes the upstream connection fail. The option is therefore useful when the name in the backend's address is not the one its certificate carries; in the lab, the address already carries its certificate's name, which makes it superfluous.

Another precision comes from the Traefik 3.7.15 code: as soon as rootCAs is set, Traefik trusts only the listed authorities, and the system authorities are not added to them. Without rootCAs, those are the ones used: that is the default behavior.

The rootCAs file follows the same prudence as the clients' one: one authority per role, here the backends' and not the clients'. A serversTransport applies to all the services that reference it: one transport per family of backends avoids an authority accepted for one being accepted for the other too.

The verification is proven by a refusal. The lab places behind a second route a backend whose certificate is signed by an unrelated authority. Querying the right backend shows nothing, since the response is a success with or without the check: the test that counts is the wrong backend's, which must fail.

Why is insecureSkipVerify not a fix?

insecureSkipVerify: true removes the verification of the backend's chain and name: the link stays encrypted, but Traefik no longer knows who it is talking to. The Traefik 3.7 documentation describes this option as the one that controls whether the server's certificate chain and host name are verified; its default value is false. Switching it to true is the reflex when a connection to a backend fails on a certificate error, and it is a bad reflex: the error disappears because the proof disappears.

Here is the counter-example, as found in an inherited configuration:

# ❌ Avoid: the backend's certificate is never looked at again
http:
  serversTransports:
    verif-facturation:
      insecureSkipVerify: true

With this line, a backend that presents the wrong certificate receives the traffic and answers like the right one. No error is displayed, which is the real danger: a chain of trust that is unplugged looks like one that works. The same finding applies to HAProxy in Verifying the identity of the backend and of the source, where verify none plays the same role.

The correct approach is to look for why the verification fails. Three frequent causes can be checked with openssl: the rootCAs authority is not the one that signed the backend, the certificate's name does not match the targeted name, or the certificate is expired.

Terminal window
openssl verify -CAfile ca-backends.crt backend.crt
openssl x509 -in backend.crt -noout -subject -issuer -dates -ext subjectAltName

The first command must answer OK if the authority does sign the certificate. The second shows the issuer, the validity dates and the names the certificate declares (the subjectAltName field, the list of names it is valid for). The -ext option of openssl x509 requires OpenSSL 1.1.1 or later; both commands were replayed with OpenSSL 3.0.13. Fix the cause you find in the serversTransport, never by removing the check.

How do you stop a direct caller from forging its address?

You limit the entry point's forwardedHeaders.trustedIPs to the address of the upstream proxy alone, so that Traefik gives no credit to the X-Forwarded-* headers sent by any other caller. When a proxy sits in front of Traefik, the source address Traefik sees is the proxy's; the real client's travels in the X-Forwarded-For header. But a header is text: a caller that contacts Traefik directly can write any address in it, and the application behind, which reads this header to decide (logging, filtering, quotas), would believe an invented identity.

Both settings go on the entry point, so in the static configuration (read only once at startup, unlike the dynamic configuration, reloaded on the fly). According to the Traefik 3.7 documentation, trustedIPs lists the addresses or ranges from which Traefik trusts these headers, and insecure always believes them, which the documentation recommends reserving for tests.

entryPoints:
  websecure:
    address: ":8443"
    forwardedHeaders:
      trustedIPs:
        - "10.0.0.10/32"      # the upstream proxy, and only it

The classic mistake is 0.0.0.0/0 (all IPv4 addresses) in trustedIPs: it trusts any IPv4 caller, an effect close to insecure: true. That range does not cover native IPv6 addresses, which need ::/0 for the same effect; the nuance does not excuse the mistake. A range that is too wide, an entire subnet when a single host relays, is the same defect. The change requires a restart of Traefik, because the static configuration is only read at startup.

The verification has two sides, and the second is the one people forget. A direct request carrying a forged X-Forwarded-For must reach the backend without the forged address; a request relayed by the authorized proxy must still reach the backend. Without the second measurement, removing the proxy would pass for hardening. The lab checks both with a witness backend that returns what it received, not with Traefik's configuration.

How do you renew certificates without cutting access?

You replace an authority in steps, because caFiles accepts a list: the old and the new coexist while the certificates are reissued, then the old one is removed. A certificate expires (the lab's are issued for 30 days, apart from the one expired on purpose), and an authority can also be replaced. A renewal that locks everyone out is an outage; one that still honors the old authority once finished removed nothing. Both are measured.

The sequence I recommend is a method judgment, not a Traefik obligation: add the new authority next to the old one, reissue the client certificates, verify that a certificate signed by the new one is accepted, then remove the old one and verify that a certificate it had signed is refused. The lab test does a blunter variant: it replaces the authority in one go, observes that the new one's certificate passes and the old one's is refused, then restores the original state.

One implementation detail, verified in the code and by a test on Traefik 3.7.15: the file provider watches the dynamic configuration directory and the files it contains, not the authority file cited by its path. Replacing the authority file alone changes nothing: the old authority is still honored and the new one refused, as long as nothing is modified in the watched directory. To make Traefik reread the replaced authority, the test modifies the watched routes file, which forces the reread. Automating the issuance of internal certificates falls to a management authority such as Vault PKI (in French).

How do you prove a refusal is a real refusal?

A refusal is proven with two values: the HTTP code and the curl exit code, because several unrelated situations give "something other than 200". A code other than 200 can mean a TLS refusal (what you are looking for), a 404 returned by Traefik (the route does not exist, the check never took place) or 000 (no response: service down, unreadable certificate file). The lab test rules these causes out one by one.

Terminal window
curl -k -s -o /dev/null -w '%{http_code}\n' --max-time 5 \
  --cert client.crt --key client.key \
  https://facturation.corp.test:8443/ ; echo "curl : $?"

The -k option only concerns the self-signed certificate of Traefik's server, a different check from the ones in this lesson. Read the two lines. With a valid certificate, 200 and curl : 0: the check lets through what it must let through, and this measurement serves as an anchor, without it a stopped service would pass for a refusal everywhere. Without a certificate, with one from another authority or with an expired certificate, the refusal happens during the handshake: curl exits with a non-zero error code, with no application HTTP response (55 or 56 in our tests on Traefik 3.7.15: with TLS 1.3, the client only sees the server's alert when reading the response). If curl exits with 0 and a 403 or a 404, the TLS connection succeeded: the certificate was accepted, it is not a certificate refusal.

For the backend, the proof is different: the wrong backend must make Traefik answer with a 5xx error. A 404 proves nothing (the request did not reach the backend), and a 200 means the verification is still disabled. Reading Traefik's response to tell a 502 from a 504 is described in Troubleshooting a Traefik entry point.

Hands-on

The lab starts from a Traefik misconfigured in three ways: the TLS option references the clients' authority but requires no certificate, the transport to the backend has verification disabled, and the entry point believes the headers from any address. You fix the dynamic and static configurations, then the validation measures four things: a valid client certificate passes while a missing certificate, a foreign authority and an expired certificate are refused during the handshake; the backend with the wrong certificate makes Traefik answer with a 5xx while the right one answers 200; a forged X-Forwarded-For does not reach the backend while the authorized upstream proxy still passes; a rotation of the clients' authority accepts the new one and refuses the old one.

Key points

An identity verification is proven by its refusals, not by the presence of an option in a file.

  • A certificate proves an identity; authorization is decided elsewhere, and two valid holders can have different rights.
  • Server authentication and client authentication are two separate mechanisms, each with its own certificate and authority.
  • The client is verified with a TLS option: clientAuth, caFiles and clientAuthType: RequireAndVerifyClientCert. NoClientCert asks for nothing and RequireAnyClientCert does not verify the signer.
  • The backend is verified with a serversTransport: rootCAs names the expected authority, serverName the name announced in SNI.
  • insecureSkipVerify: true removes the proof instead of fixing the error: look for the authority, name or date at fault.
  • forwardedHeaders.trustedIPs is limited to the upstream proxy's address; the static configuration requires a restart.
  • A refusal is proven by the HTTP code and by the curl exit code, with a valid case as an anchor.
  • An authority rotation is measured in both directions: the new authority passes, the old one is refused.

Knowledge check

Eight questions check that you can choose the right verification for each direction, recognize a configuration that verifies nothing, and prove a refusal.

Knowledge check

Check what you have learned with this interactive quiz

8 questions
8 min
70% to pass

How it works

  • The timer starts when you click Start
  • Multiple choice, true/false and short answer questions
  • You can move between questions
  • Detailed results are shown at the end

Is this project useful to you?

The guides, courses, quizzes and labs stay free. If you use them regularly, you can contribute to their upkeep and development.

Become a supporting member (new tab) One-time support (new tab) Learn more

Follow the project on LinkedIn