To verify a backend's identity with HAProxy 3.4, declare four elements on the server line: ssl verify required, ca-file, sni and verifyhost. Without sni, verification fails, with a symptom that does not mention it at all. This lesson is for readers who already publish a service behind HAProxy and want the proxy to know who it is talking to (the backend) and who a request comes from (the source). It does not redo the TLS termination covered in Publish HTTPS with HAProxy 3.4: crt-store, SNI and built-in ACME certificates: it teaches how to establish trust, in both directions.
What you will learn
The lesson starts from a chain that answers without checking anything and ends with a chain where every identity is verified.
- Turn on the verification of a backend's certificate with
verify required. - Explain why
ca-fileandverifyhostare not enough withoutsni. - Recognize a certificate authority without
keyUsagewithopenssl. - Interpret
SC--andssl_bc_err_strto link a 503 to its real cause. - Tell apart a client with a valid certificate, without a certificate, or signed by another authority.
- Stop a client from choosing its own address through a header or the PROXY protocol.
Prerequisites
The gestures were played on HAProxy 3.4.7, the package from the trixie-backports-3.4 repository of Debian 13. You need a running HAProxy, an account able to use sudo and a backend that speaks TLS. The tables and the return codes reproduce the test bench measurements of the lesson. The configuration excerpts reproduce their shape, with neutral names: they were not replayed as they stand, and the page says so when an output comes from a specific run.
- Install HAProxy 3.4 and read its sections: Install HAProxy and read its configuration sections.
- Serve a site over HTTPS and understand SNI: Publish HTTPS with HAProxy 3.4: crt-store, SNI and built-in ACME certificates.
- Read a certificate error with
openssl s_client: TLS diagnostics: debugging SSL errors with openssl. - Follow the path of a request,
HostandX-Forwarded-Forincluded: Publish a web service: the path of a request to the backend.
Why can a proxy that encrypts to its backend verify nothing?
Encrypting a link says nothing about the identity of whoever is at the other end: with ssl verify none, HAProxy speaks TLS to any server, an impostor included. The link is confidential, but the certificate presented is never looked at: a server that carries the same name as the legitimate one, signed by another authority, receives the traffic and returns a response indistinguishable from a success. That is the trap of this lesson: no error is displayed. A fully disconnected chain of trust looks exactly like a working chain, and only a negative check (a backend that must be rejected and is) tells one from the other.
Verification covers two identities. The backend's: the proxy checks that the server's certificate is signed by the authority it expects, and that it carries the right name. The source's: the application behind the proxy only sees the proxy's address, so it trusts headers that the proxy must rewrite, and the client can prove who it is with a client certificate. The sections below take these three mechanisms in order.
How does HAProxy verify a backend's certificate?
Verifying a backend is done on the server line: ssl turns TLS on, verify required makes verification mandatory, ca-file designates the trusted authority, verifyhost imposes the name, and sni says which name is requested. The first three keywords say "verify, and against what"; verifyhost adds the check of the name carried by the certificate, without which a certificate valid for another service would pass; sni tells the backend which certificate to present. The ca-file file must contain only the authority of the backends: putting the clients' authority in it mixes two chains of trust that have nothing to do with each other.
Here is the shape measured on the test bench, with a neutral name. It is split over several lines for readability; HAProxy reads it as a single server line.
backend facturation server api1 10.0.0.21:8443 ssl verify required \ ca-file /etc/haproxy/pki/ca-backends.crt \ sni str(facturation.corp.test) \ verifyhost facturation.corp.testAfter a haproxy -c -f /etc/haproxy/haproxy.cfg and a reload, the expected result is a response from the legitimate backend. The real test is negative: temporarily point the server to a server whose certificate is signed by another authority, and the response must become an error. If it stays a success, verification is not strict.
Why does a verify required fail without sni?
Without sni, a verify required fails, even with the right ca-file and the right verifyhost: HAProxy 3.4.7 returns a 503 while the server is UP. A TLS server that carries several certificates cannot guess which one to present: it waits for the client to announce the requested name in the SNI (Server Name Indication, the host name placed in clear at the start of the TLS handshake). Without this announcement, the proxy does not receive the expected certificate, verification fails, and the traffic is not served. verifyhost sets what is checked; sni sets what is requested, and both are needed.
The measurement was made on a minimal test bench, HAProxy 3.4.7, with four configurations of the server line against the same backend:
server line | Result |
|---|---|
ssl verify required ca-file ca.crt verifyhost nom | 503 |
ssl verify required ca-file ca.crt | 503 |
ssl verify required ca-file ca.crt sni str(nom) verifyhost nom | 200 |
ssl verify none | 200 |
The result is the same with a Python backend and with an nginx backend: so it is not the server. The neighboring hypotheses were ruled out one by one: neither verifyhost, nor the extendedKeyUsage of the leaf, nor the TLS version, nor the name of the authority explains the failure. The only change that moves from 503 to 200 is adding sni. Note the last line: verify none answers 200 without verifying anything, which explains why the fault goes unnoticed as long as nobody checks.
A precision for 3.4: since 3.3, HAProxy derives by default the SNI sent to the server from the request's Host header (sni-auto). That value changes with each client, whereas the backend's name is fixed: declare sni explicitly, it is the form that was measured.
Why does a CA without keyUsage keep the server DOWN?
A certificate authority built without a keyUsage extension passes openssl verify but fails as soon as verification is strict, which is what a verify required demands: the server stays DOWN and the proxy returns a 503. The symptom blames the routing or the backend, while the fault is in the PKI (the infrastructure that builds and signs certificates). The keyUsage extension says what the authority's key is for: signing certificates. An authority that does not declare it is tolerated by light checks and rejected by strict ones.
The natural false lead is the extendedKeyUsage of the leaf (the server's certificate): it has nothing to do with it in this measurement. The right reflex is to replay the strict verification before touching HAProxy:
openssl verify -x509_strict -purpose sslserver -CAfile ca-backends.crt feuille.crtOn the lesson's test bench, this command fails for an authority without keyUsage and succeeds with it. With OpenSSL 3.0.13, the same verification, replayed for this page on an authority built without the extension, answers "error 92 at 1 depth lookup: CA cert does not include key usage extension", whereas openssl verify without options returns OK. To build a correct authority with openssl req -x509, declare the extension at creation time, for example with -addext "keyUsage=critical,keyCertSign,cRLSign", then replay the strict command above: it is the one that validates.
How do you read a 503 with SC-- and ssl_bc_err_str?
The SC-- termination code in the log says the failure happened while establishing the connection, on the server side; the ssl_bc_err_str field gives the exact reason supplied by OpenSSL. The 3.4 configuration manual, section 8.5, defines S as "the TCP session was unexpectedly aborted by the server, or the server explicitly refused it" and C as "the proxy was waiting for the CONNECTION to establish". Read together, they say the request never reached the application: the disagreement is in the TLS handshake with the backend, not in the routing.
The default log does not say why. Add the ssl_bc_err_str field to a log-format to get the cause. This line is given as an example of structure: only the field's result was measured.
backend facturation log-format "%ci:%cp %ft %b/%s %ST %tsc %[ssl_bc_err_str]"In the test bench measurement, the field returns "error:0A000086: certificate verify failed": the certificate verification failed, which points to sni, to the name or to the authority. This field saves hours of searching, because the 503 and the server state do not tell these causes apart. The diagram below orders the three questions to ask: is the server DOWN, does the log show SC--, what does ssl_bc_err_str say.
This reading is limited to the trust fault. The general method for diagnosing an HAProxy that does not answer is beyond the scope of this lesson, and is covered by Diagnose HAProxy without masking the cause.
How do you tell a client with a certificate, without one, and from another authority apart?
To recognize a client by its certificate without closing the whole site, put verify optional on the bind and decide route by route with ssl_c_used and ssl_c_verify. The bind carries ca-file: the clients' authority, which is not the backends' one. With verify optional, the handshake succeeds even without a certificate, which leaves a public route open; with verify required, any client without a certificate would be refused at connection time, the public route included. Authorization is then played out in the rules: ssl_c_used says whether a certificate was presented, and ssl_c_verify is 0 when the chain is valid.
frontend entree bind :443 ssl crt /etc/haproxy/certs/site.pem \ ca-file /etc/haproxy/pki/ca-clients.crt verify optional http-request deny deny_status 403 if { path_beg /reserve/ } !{ ssl_c_used } http-request deny deny_status 403 if { path_beg /reserve/ } !{ ssl_c_verify 0 }The two rules cover two distinct failures: the absence of a certificate, and a certificate whose chain is not valid. Test the three cases: without a certificate, with one from another authority, with one from the clients' authority. Only the last must reach the reserved route, and the public route must answer without a certificate. What you observe for the other-authority case (handshake refused or error code) is established on the test bench: it is the lab that has you measure it.
How do you stop a client from choosing its own address?
A client must never be able to choose the address the application will read: replace X-Forwarded-For instead of extending it, and accept the PROXY protocol only from a trusted address. option forwardfor appends to the received header: if the client sends its own X-Forwarded-For, its value reaches the application first, and the application almost always reads the first one. The gesture that settles it is to replace: delete the received header before option forwardfor, or rewrite it with the source address.
frontend entree http-request del-header X-Forwarded-For option forwardfor http-request set-header X-Forwarded-Proto https http-request set-header X-Client-DN %{+Q}[ssl_c_s_dn]The same rule holds for any header the application believes: the original protocol, and the name carried by the client certificate, which the last line transmits. That line applies to a route that requires a certificate. On a public route, delete the identity header with http-request del-header X-Client-DN instead of setting it, otherwise the client sends it itself. You can check it with a backend that returns what it received: it is the witness, not the proxy's configuration.
The other door is the PROXY protocol, which lets an upstream relay declare the real client's address. An accept-proxy listener trusts whoever connects to it, and the PROXY header comes before the TLS handshake: no identity check protects this listener, only its address does.
frontend relais bind 10.0.0.5:8444 accept-proxy ssl crt /etc/haproxy/certs/site.pemBind the listener to an address reachable only from the trusted source, not to all interfaces. Then check two things: from the network, the listener does not answer; from the trusted source, the declared address reaches the application. Without the second measurement, deleting the listener would be enough, and the configuration would have lost a capability instead of gaining a control. The lesson Install HAProxy and read its configuration sections presents send-proxy, the sending side.
Practice
The lab starts from an HAProxy where everything answers while trust is disconnected: the backend is reached with verification disabled, no client identity is requested, and the client's address can be chosen by header as well as by the PROXY protocol. You reconnect the chain from an acceptance criteria file that describes four authorities, routes and identities. Validation covers four measurements: the backend's identity is verified and an impostor with the same name is rejected, a valid client identity is told apart from an absence and from another authority, a client cannot choose its address, and the authorized route keeps the protocol and the identity all the way to the application.
Key points
A chain of trust is proven by its rejections: a backend that must be rejected, a client that must be refused.
ssl verify noneencrypts without verifying: an impostor with the same name answers like the legitimate one, without any error.- A backend is verified with
ssl verify required,ca-file,sniandverifyhost:ca-filesays against what,verifyhostwhich name to check,sniwhich name to request. - Without
sni, averify requiredfails: 503 on HAProxy 3.4.7 with anUPserver, even thoughca-fileandverifyhostare correct. - An authority without
keyUsagepassesopenssl verifyand fails with-x509_strict -purpose sslserver: the server staysDOWN, the fault is in the PKI, not in the routing. SC--places the failure at establishment, on the server side;ssl_bc_err_str, in thelog-format, gives the exact OpenSSL reason.- A client is recognized with
verify optionalon thebind, thenssl_c_usedandssl_c_verify 0route by route. option forwardforappends: delete or rewrite the received header, and do the same for any identity header the application believes.- An
accept-proxylistener is protected by its address, never by TLS, and is measured from both sides.
Knowledge check
Eight questions check that you can verify a backend, interpret the symptom of a trust failure and close the doors through which a source chooses its identity, once the lab has been played.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Protect and observe HAProxy: ACLs, stick-tables, headers, logs and Prometheus
A new full quiz with random questions
Work again only on the questions you missed