Publish HTTPS with HAProxy 3.4: crt-store, SNI and built-in ACME certificates

With HAProxy, a frontend that listens over HTTPS presents a certificate chosen by the requested name (SNI), decrypts the request and sends it in clear to a backend; since version 3.2, a built-in ACME client can obtain and renew this certificate without an external tool, but it remains experimental in 3.4. This lesson is the optional HAProxy module of the training: it covers publishing over HTTPS, and leaves load balancing to the Linux lesson devoted to HAProxy.

What you will learn

The lesson follows the path of an encrypted connection, then that of the certificate that makes it possible.

• Terminate TLS in a frontend and redirect HTTP to HTTPS.
• Understand the choice of certificate by SNI.
• Describe the acme and crt-store sections.
• Measure what "experimental" means for the ACME client.
• Check a renewal and a reload with no cut.

Prerequisites

This lesson assumes you know the basics of HAProxy and what an ACME certificate is.

• The frontend and backend concepts and the validation of a configuration with haproxy -c: HAProxy.
• The path of a request, SNI and TLS termination: Publish a web service.
• The ACME protocol and its challenges: Automatic certificates.

How does HAProxy terminate TLS?

The frontend declares a bind with ssl, then a rule redirects port 80 to HTTPS. HAProxy decrypts the connection, applies its rules to the clear request, then passes it on to the backend. An X-Forwarded-Proto header tells the application that the client arrived over HTTPS.

frontend https_in
    bind :80
    bind :443 ssl crt /etc/haproxy/certs/ alpn h2,http/1.1
    http-request redirect scheme https unless { ssl_fc }
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    default_backend app

When crt designates a directory, HAProxy loads all the certificates it contains. For each connection, it compares the name announced by the client (SNI) with the certificates' names and presents the right one; if nothing matches, it presents a default certificate. It is the same mechanism as Traefik's, described in the lesson on the path of a request.

What are crt-store and the acme section for?

The crt-store section groups the certificates and their keys in a named location, and the acme section describes the authority that issues them. A certificate of the store is tied to an authority by the acme keyword, and the frontend references it by its name rather than by a file path.

global
    expose-experimental-directives

acme pebble
    directory https://pebble.lab.test:14000/dir
    contact   admin@lab.test
    account-key /etc/haproxy/acme/account.key
    challenge http-01

crt-store certs
    crt-base /etc/haproxy/certs
    key-base /etc/haproxy/certs
    load crt "app.lab.test.pem" acme pebble domains "app.lab.test"

frontend https_in
    bind :443 ssl alpn h2,http/1.1
    ssl-f-use crt "@certs/app.lab.test"

This file illustrates the structure, not a ready-to-use solution: the authority's address, the paths and the names are to be adapted. The principle to keep is that the certificate is requested and renewed by HAProxy itself, and that the frontend reads it from the store. If the certificate does not exist yet at startup, HAProxy can present a temporary one while it obtains the real one.

The certificates obtained by this client are not written to disk. HAProxy avoids touching the filesystem once its configuration is loaded: the certificate and its key live in memory, inside the store. The consequence catches people out at the first renewal, because the file they expected in the store's directory never appears, and a backup that copies that directory backs up nothing. To get a copy, you have to ask the process for it, with the dump ssl cert command on the admin socket, or with the haproxy-dump-certs utility that the Debian package provides.

An experimental ACME client

HAProxy's ACME client requires the global directive expose-experimental-directives: the project ranks it among the features whose syntax may still change. This does not mean it does not work, but that a version upgrade must be preceded by a reading of the release notes. For a critical service, weigh this risk against that of an external tool such as certbot or lego, older and more widespread.

How do you protect the ACME account key?

The ACME account key proves your identity to the authority: it must be readable by HAProxy alone. Whoever holds it can request certificates in the account's name. Place it in a dedicated directory, with permissions restricted to the account that runs HAProxy, outside any versioned repository.

The same principle applies to the private keys of the store's certificates: a file readable by everyone cancels the point of encryption. After a configuration, list the files' permissions and check that only the owner and, if needed, its group can access them.

How do you reload without cutting connections?

Reload HAProxy rather than restarting it: the master process starts new worker processes with the new configuration, and the old ones finish their connections in progress. With the systemd service, the reload command does this sequence, once haproxy -c -f has validated the configuration.

haproxy -c -f /etc/haproxy/haproxy.cfg
systemctl reload haproxy

To prove that a renewed certificate is served without interrupting the process, compare two things before and after the renewal: the serial number or the end date of the presented certificate (openssl s_client, with the requested name as SNI), and the identifier of the master process, which must stay the same. A certificate that changes and a process that does not prove the live renewal.

Practice: HTTPS and ACME with HAProxy

The lab starts from a small application listening on the host and an HAProxy deployment that does not publish it properly over HTTPS. The test ACME authority and its HTTP-01 challenge responder are already prepared. It asks you to route app.lab.test to the local backend, to redirect HTTP to HTTPS, to obtain the certificate with the native ACME client, to protect the account key and to prove that a renewed certificate is served without interrupting the process.

Guided lab Practice

Learn how to do it

Publish HTTPS with HAProxy

$ dsoxlab run pub-haproxy-tls

Environment: VM·Duration: ~40 min

How it is checked: the state of the system is checked after a reboot, not the command you typed.

First time? What DSOXLab is and how to install it

DSOXLab is the command line that prepares a disposable environment (an isolated shell or a VM), runs the lab scenario, then checks the real state of the system and computes a score. Nothing is installed or broken on your workstation.

It needs Python 3.11 or later, uv or pipx, and git. The installation guide details these prerequisites; on Windows, or on a Mac without Python, use the appliance instead.

This lab starts virtual machines: it also needs ansible-playbook, a hypervisor (KVM or Incus), Terraform and hardware virtualization (/dev/kvm), which inside a VirtualBox VM means nested virtualization. The first time, dsoxlab start pub-haproxy-tls checks all of it before creating the machines. Prepare your machine for vm labs.

Install the CLI and add the catalogue, once:

uv tool install dsoxlab   # or: pipx install dsoxlab
dsoxlab catalog add https://github.com/stephrobert/publier-service-dsoxlab-training

Before you run the lab, check:

• DSOXLab installed
• catalogue available
• compatible environment (isolated shell or VM)

To check, nothing is probed from this page: run dsoxlab doctor. Read the diagnosis

Run this lab, then validate your work:

dsoxlab run pub-haproxy-tls
dsoxlab check pub-haproxy-tls

See every lab: dsoxlab list-labs. Track your progress: dsoxlab progress. The full dsoxlab guide covers scoring and the pitfalls.

Key points

HAProxy serves HTTPS through a frontend, a certificate store and, as an option, its ACME client.

• The SNI chooses the certificate presented.
• crt-store groups the certificates, acme describes the authority.
• The ACME client is experimental: expose-experimental-directives is required.
• The ACME account key stays readable by HAProxy alone.
• The certificates stay in memory: dump ssl cert or haproxy-dump-certs gets a copy out.
• reload after haproxy -c replaces the processes without cutting connections.

Knowledge check

Knowledge check

Check what you have learned with this interactive quiz

8 questions

8 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →