Skip to content
Français
Account: not signed in, sync available My learning
Select theme
Français

Limit abuse with HAProxy stick-tables

By updated

Improve this page

HAProxy logo

To limit abuse with HAProxy, count requests in a stick-table whose key is the client's address, never a value the client sends, and reject above a threshold. This lesson is for readers who already publish a service behind HAProxy 3.4 and want a client that talks too much to get rejections without bothering the others. It does not repeat the ten-line recipe of Protect and observe HAProxy: ACLs, stick-tables, headers, logs and Prometheus: it teaches what makes it silently wrong, that is, the key, the window, the expiration and the scope of the table.

What you will learn

The lesson goes from the table that fills up without protecting anything to the table whose scope you know.

  • Declare a stick-table with its type, its size, its expiration and what it stores.
  • Track a client with track-sc0 and reject above a threshold with a 429.
  • Choose a key the client cannot modify, and prove it.
  • Tell apart the window of the rate and the expiration of the entry, and read it in show table.
  • Choose between counting requests, connections or sessions.
  • Say what the table does not protect: another instance, a shared address, memory.

Prerequisites

The lesson assumes a running HAProxy and an account able to use sudo. The gestures were played on HAProxy 3.4.7, the package from the trixie-backports-3.4 repository of Debian 13, with a client distinct from the machine that carries the proxy. The commands to lift a block in the dedicated section were not replayed, and the page says so where it gives them.

What is an HAProxy stick-table and what does it store?

A stick-table is an in-memory table, specific to one HAProxy process, that associates a key (an address, a string) with counters you choose. It is declared with stick-table and four settings. The type says the nature of the key: ip for an IPv4 address, string for a string. A table of type string therefore cannot be keyed on an address, that is, have an address as its key. The size (size) bounds the number of entries, hence the memory. The expiration (expire) says how long an entry survives without activity. Finally store lists the data kept per entry, for example http_req_rate(1m), the request rate measured over one minute.

A table counts nothing by itself: it waits for a rule to hand it a key to track. That is the role of track-sc, presented right after. The table is then read through the admin socket, with show table <name>, which lets you see what HAProxy has actually retained.

How do you track a client and reject above a threshold?

You track the key with http-request track-sc0 <key> table <name>, then reject when sc_http_req_rate(0) exceeds the threshold. The 0 designates the first tracking slot: track-sc0 feeds it, sc_http_req_rate(0) reads its counter. The example below puts the table in a backend that serves nothing else, which names it: it is read by its name and several frontends can share it. The values (1m, 2m, 100, 50k) are those of the example, not replayed as they stand and to be adapted to your traffic. The lab uses other values: do not mix them up.

frontend public
bind :80
http-request track-sc0 src table limites
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
default_backend applicatif
backend limites
stick-table type ip size 50k expire 2m store http_req_rate(1m)

Read the threshold with its exact meaning. gt 100 means "strictly more than a hundred": a burst of a hundred requests goes through, the hundred-and-first is rejected. The reason lies in the order of operations: track-sc0 counts the current request before the comparison, and gt is strict. This count holds for an entry at rest; if the rate has not gone back down, fewer than a hundred requests go through, and the rejected requests count too. A control stricter than what was planned rejects traffic the service was sized to absorb, and that is a fault too. The rejection code is chosen with deny_status; without it, deny returns a 403, which does not tell the client to slow down. The 429 (Too Many Requests) does.

One last condition makes the difference between a limit and a decoration: the rejection must apply to the key, not to the service. As long as one client is rejected, another client must still be served. Otherwise the limit does not protect the service, it replaces it.

Why must the key never be a value sent by the client?

A counter keyed on a header is reset by the very client it was meant to limit. A header is chosen by whoever sends it: a client that changes the value of X-Client-Id on every request starts from a new counter each time, and never reaches the threshold. The table fills up, show table displays entries, and nothing is protected. It is the first fault of this lesson's lab, and it is not visible when reading the configuration.

The lab makes you prove it, not just fix it: once the source is blocked, the validation replays the request with five different header values, and all of them must stay rejected. The fix comes down to one word, src, the address of the TCP connection, the only value of the request that the client does not choose. It also requires a suitable type: a table of type string cannot be keyed on an address, and changing the key without changing the type does not load.

This principle is not specific to HAProxy. The guide Nginx limits reaches the same rule with limit_req: the grouping key must be impossible to forge.

One case moves the problem without removing it: behind another proxy, src is the relay's address, and all the clients end up under the same key. Reading the address that the relay announced in a header is acceptable only if this relay is trusted, if it overwrites the header received from the client and if HAProxy accepts this header only from it. This variant is not played in the lab; treat it as a point to validate on your architecture.

What is the difference between the window and expire?

The window says over how long the rate is computed, the expiration says how long the entry survives in the table: they are two distinct bounds, and an entry whose rate has dropped to a low value still exists. The values in this section are those of the lab, not those of the previous example. In store http_req_rate(10s), the ten seconds are the window; in expire 30s, the thirty seconds are the lifetime of the entry after the last request. The rate is a sliding average that HAProxy makes decay: the 3.4 manual writes that http_req_rate(<period>) "reports the AVERAGE HTTP request rate over that period". It is not a bucket that empties at the deadline. Measured over a ten-second window after a burst of 5 requests, under the threshold, the rate was still 3 thirteen seconds after the last one, with no rejection at all. It goes down, then falls back under the threshold: that is when the source gets the service back.

The diagram follows the life of an entry. Look at the highlighted node: the key is served again, and yet the entry is still in the table. Only the expiration removes it.

An entry is created at the first request, the rate rises with the requests, crossing the threshold causes a 429 rejection for this key only, then the rate goes down when the requests stop, the key is served again while the entry still exists, and the entry disappears at expiration.noyesFirst request: the entry is createdRequests keep coming: the rate climbsRate above the threshold?429 refusal for this key onlyRequests stop: the rate decays, this is the windowRate below the threshold: the key is served again, the entry stillexistsNo request until the expiry: the entry disappears, this is expire
The window sets the decay of the rate, expire sets the disappearance of the entry. Source · SVG
Esc to close, + and − to zoom, scroll to move around. 100 %

Two practical consequences. The first: do not wait for an exact zero. To decide that your block does not lift, wait up to two windows and look whether the rate falls back under the threshold. Concluding earlier amounts to blaming a correct configuration. This trap cost the lab itself two tests, which demanded an exact zero and claimed a property the tool does not promise. The second: set expire at least as long as the window. With a shorter expiration, the entry disappears before the rate has finished being computed, and the counter starts again from zero without saying anything.

This slowness has an effect on blocks, and it is measured on a different burst from the previous one. With a 10-second window, a threshold of 20 and a burst of 25 requests, above the threshold, one control request out of two was still rejected thirteen seconds later. A rate therefore stays above the threshold longer than a window after an overrun: a block lifts later than you think. Do not confuse it with the previous measurement, where the rate of 3 stayed under the threshold and rejected nothing.

To observe it, read the table through the socket during and after a burst:

Terminal window
echo 'show table limites' | sudo socat -t2 stdio /run/haproxy/admin.sock

The first line reports the type and the size of the table. Each entry carries its key (key=) and its rate. After the burst, the rate goes down while the key stays listed; it disappears only after expire without a request. The disappearance of the entry, and not a zero rate, is the proof that the memory is freed.

Should you count requests, connections or sessions?

Count what you want to bound: http_req_rate for HTTP requests, conn_rate for connections, sess_rate for sessions. The three are not interchangeable. A persistent HTTP connection often carries dozens of requests: a client that opens only one connection can saturate a backend without conn_rate moving. Conversely, counting requests does not protect against the client that opens a thousand slow connections and completes none; a concurrent-connections datum such as conn_cur answers that risk better.

The lab counts requests, with a client that opens one connection per request, which makes the two numbers equal and hides the difference. These three data are quoted from the manual; their comparison is not replayed in the lab. Choose the datum according to the abuse you fear, then check it with show table.

How do you lift a block, and who can do it?

A rate block lifts by itself when the average falls back under the threshold; to lift it by hand, you erase the entry from the table through the socket. The expected command is clear table <name> key <key>, for example clear table limites key 192.0.2.10, which this lesson did not replay. It deletes the entry, so the counter starts again from zero.

The point to remember is the right. An operator-level socket can run set table and clear table, and therefore erase a ban: handing out "read" access to a metrics collector is not enough to prevent it from modifying your limits. The lesson Change HAProxy's state live with limited rights covers the socket levels and how to separate them.

This right also sheds light on a design choice. A block that does not lift is not a rate limit, it is a ban, and an address banned by mistake reopens by hand. The lab checks both halves: the rejection is reproducible, and the source gets the service back once the rate has gone down under the threshold.

What does a stick-table not protect?

A stick-table protects only within one HAProxy instance and does not tell apart clients that share an address. These are the two limits that the lab's decision record requires, because the configuration does not say them. Neither is fixed with a directive. The last two entries of the list complete the inventory.

  • Scope of an instance. Two HAProxy behind a load balancer have two tables: a threshold of 100 per instance lets 200 through. The peers section synchronizes the tables, at the price of a convergence that is not immediate. Replication therefore does not turn a local limit into an exact global quota.
  • Shared address. Behind a NAT or a mobile network, thousands of clients share an address, and the limit punishes them together. Counting by address is the best the connection offers, and it is also its limit.
  • Bounded memory. size caps the number of entries, hence the memory: an oversized value lets the traffic decide what the proxy consumes. The more varied the key (high cardinality), the faster the table fills. When it is full, HAProxy "will flush a few of the oldest entries" by default (3.4 manual); it is the nopurge option that makes it reject new entries. This behavior is read in the manual, not replayed.
  • Not a log. The table lives in memory, it is neither a history nor durable evidence. To find out who was rejected last week, rely on the logs, not on show table.

An honest decision record states these limits in black and white. That is what the lab's last measurement asks for, read by a person and not by a test.

Why does an IPv6 client escape a table of type ip?

An IPv6 client whose address reaches a table of type ip is not limited at all: track-sc0 tracks nothing for it. The 3.4 manual writes "In 3.2, "ip" is an alias for "ipv4"… In a future version, "ip" will instead correspond to "ipv6"", and it advises against ip in favor of explicit ipv4 or ipv6. Converting an IPv6 address into an IPv4 key succeeds only for an IPv4-mapped, IPv4-compatible or 6to4 address (manual, section 7.1); otherwise the key is null. This point comes from reading the code (smp_to_stkey in stick_table.c, v3.4.0) and the manual: it was not replayed.

The flaw is a protection hole: you believe your service is protected, and an IPv6 client goes through without being counted. The remedy is to declare type ipv6 for IPv6 clients, or to provide for both families with one table per type. This lesson's example and the lab both use type ip, a form that will change meaning: read them again with this reservation.

The lab does not let you observe this hole: its network carries no global IPv6 address, only the loopback and the link-local address, and its scope is IPv4. The flaw is observed in production, where a service is reachable over IPv6. The lesson is that of the scope of a key: an address family that is not covered is an open door, and a green test only measures what it goes through.

Practice

The lab starts from a stick-table that fills up and can be read, but protects nothing: its key is a header the client sends, its threshold is never reached, and its window and expiration are mixed up. You fix it from an acceptance criteria file of imposed values, then you write a decision record. Validation covers four measurements: the table counts the client's address over the announced window, an overrun rejects without blocking another source, the key cannot be chosen on the client side and the block lifts, and the size and expiration bounds are the announced ones. The decision record, read by a person, states what the table does not protect.

Key points

A stick-table protects if the key is reliable, if the threshold is reachable and if you know its scope.

  • A stick-table is set by its type, its size, its expire and what it stores with store.
  • track-sc0 tracks a key, sc_http_req_rate(0) reads it, deny_status 429 rejects; gt 100 rejects as soon as the average exceeds a hundred, that is the hundred-and-first request of a burst that starts from an entry at rest.
  • The key must never be a value the client sends: a header can be changed, and the counter starts again from zero. src is the reliable key.
  • The window says over how long the rate is computed, expire how long the entry survives, and show table shows both.
  • http_req_rate(<period>) is a sliding average: it goes down and falls back under the threshold, it does not drop to zero at the deadline. Wait up to two windows before concluding.
  • Requests, connections and sessions are three different counters: choose according to the abuse you fear.
  • An operator socket can run clear table: it can lift a block.
  • A table of type ip does not track IPv6 clients: declare ipv4 and ipv6 explicitly.
  • The table lives in one instance, does not tell apart clients behind the same address, and is not a durable log.

Knowledge check

Eight questions check that you can choose the key, read the window and the expiration, and know the limits of the table, once the lab has been played.

Knowledge check

Check what you have learned with this interactive quiz

8 questions
8 min
70% to pass

How it works

  • The timer starts when you click Start
  • Multiple choice, true/false and short answer questions
  • You can move between questions
  • Detailed results are shown at the end

Starts the quiz and the timer

Is this project useful to you?

The guides, courses, quizzes and labs stay free. If you use them regularly, you can contribute to their upkeep and development.

Become a supporting member (new tab) One-time support (new tab) Learn more

Follow the project on LinkedIn