A gateway in front of a web service can make it faster in three ways, waiting less, retrying and reusing responses, and each one becomes dangerous as soon as it is tuned without discernment. This lesson is for someone who can publish a service behind a reverse proxy and wants to understand what these three gestures can break: an order placed twice, Alice's data served to Bob, a caller stuck for ten minutes on a dead backend. It lays down the concepts, valid for any tool, then has you fix a policy that is provided.
What you will learn
The lesson goes from durations to reused responses, then to the proof that nothing is altered.
- Distinguish the phases of a request that have a duration, and those a timeout must cover.
- Choose a read timeout by measuring the slowest legitimate response, without picking a round number.
- Decide whether to replay a request from its method, and never from what you observe.
- Explain why a replay hides a failure from the client and can double a write.
- State the conditions for a response to be cached without leaking between users.
- Prove a cache and its isolation by a measurement independent of its own headers.
Prerequisites
The lesson assumes you can read an HTTP exchange and send a request with curl.
- Read a request, a response, their headers, the methods and the status codes: Read HTTP before configuring a proxy.
- Place the gateway on the path of a request: Publish a web service.
- Know how to edit a simple YAML file. The lab provides the machine and the services, you install nothing.
What does this lesson cover, and what does it leave to the tool pages?
This lesson lays down the concepts common to all proxies; the syntax of a specific tool lives in its own page. It answers three questions: which phases of a request have a duration, what a replay can break on a write, and under what conditions a response is cached without leaking. A gateway here is the server placed in front of the application, the one that receives the callers and relays their requests to the backend, the application that actually produces the response.
The tool pages take these concepts up with their directives. For Nginx, Nginx cache: prove that no private response leaks gives proxy_cache, the cache key, proxy_no_cache and compression. For HAProxy, Bound time, queues and long-lived flows with HAProxy gives the per-phase timeouts and retry-on. Here, the lab uses a teaching gateway whose policy fits in a YAML file: the vocabulary is deliberately neutral, so the reasoning transposes.
Which phases of a request have a duration, and which one should you bound?
A request going through a gateway waits in at least three places, and each deserves its own timeout: establishing the connection, waiting for the response, and the total duration seen by the caller. A timeout is the maximum time granted to a phase before giving up. Without one, an element that never answers holds the request indefinitely, and the request in turn holds a connection, a process, a caller.
The first phase is the connection: the gateway opens a TCP channel to the backend. Two backend failures are told apart there. A host that refuses the connection (application stopped, port closed) is detected almost immediately, since the failure comes back right away. An unreachable host, whose packets are dropped without any answer, is much slower to settle: the gateway waits for the connect timeout, which is 60 seconds by default in Nginx (proxy_connect_timeout, which generally cannot exceed 75 seconds). This timeout is therefore tuned like the others.
The second phase is reading: the connection is open, the request is sent, and the gateway waits for the response. This is the phase that drags on with a live but blocked backend, for example on a saturated database. In Nginx (proxy_read_timeout) and in HAProxy (timeout server), this read timeout runs between two successive receptions of bytes, not over the total duration of the response: a backend that sends one byte per second never triggers it as long as that interval stays under the timeout.
The third duration is not in the gateway, it is at the caller: a browser, a curl --max-time script, another service. The caller has its own patience, and the rule that follows is simple. The gateway's timeout must be shorter than the caller's, otherwise the caller gives up first, never receives the gateway's clean error message, and no longer knows whether the request succeeded.
How do you choose a read timeout that does not cut what works?
You choose a read timeout by measuring the slowest legitimate response of the service, then adding a margin; you do not pick a round number. A timeout that is too long lets the failure spread: each caller waits, connections pile up, and a single dead backend ends up immobilizing the whole gateway. A timeout that is too short produces the opposite effect, a quieter one: it rejects slow but correct responses, and it is an outage you inflict on yourself.
The two halves therefore count together. In the lab, the application exposes a /lent route that answers correctly after two seconds, and a /panne route that never answers. An acceptable read timeout lets the first one through and cuts the second early enough that the caller does not stay blocked indefinitely. Neither condition is enough alone: a one-second timeout cuts /panne but breaks /lent, and a ten-minute timeout serves /lent but leaves the caller hanging on /panne.
The method comes down to three gestures. Measure first, on the real service, the duration of the slowest legitimate response. Add then a margin that absorbs a reasonable peak, without inflating it. Check finally both sides, the legitimate slowness that passes and the failure that is cut. A timeout is tuned on a measurement; the exercise's value is not to be copied onto your traffic.
Why do you decide on a replay from the request's method?
You decide whether to replay a request from its HTTP method, because only the method says whether executing it twice has the same effect as executing it once. A replay (retry) sends the same request again after a failure. A method is idempotent when sending it once or several times has the same intended effect for the client. RFC 9110 puts PUT, DELETE and the so-called safe methods, that is read-only ones, among the idempotent methods: GET, HEAD, OPTIONS and TRACE. POST is not, since each POST /commande can create one more order. The RFC speaks of intended effect, not server state: a server remains free to log each request, and two GETs leave two log lines without the method ceasing to be idempotent.
The difficulty comes from the client not seeing what the server sees. Imagine an application that increments its order counter, then loses the connection before sending its response. For the gateway, it is a connection cut without a response. For the caller, it is the same thing. But in one case the order never happened, and in the other it happened and it is the response that was lost. Nothing distinguishes them from the outside: only the business counter, on the application side, knows the truth.
Look at the diagram from top to bottom. The application counts the order, then cuts without answering. If the gateway replays the POST, the counter goes to two, and the client only sees a failure. If it does not replay, the counter stays at one, and the client sees the same failure. The observation is identical on both sides of the branch, which is precisely the reason not to base the decision on it.
What does a replay hide from the client, and what can it break?
A replay hides the failure from the client: the failure did happen and was counted on the gateway side, but the caller sees nothing, which is an advantage for a read and a trap for a write. When the gateway replays a GET and the second attempt succeeds, the caller receives its page without ever knowing a first attempt failed. That is useful: a passing failure disappears from the user's point of view. It is also an observation risk, since absorbed failures must be counted somewhere, otherwise a failing backend stays invisible as long as the second attempt passes.
For a write, the same hiding changes nature. If the gateway replays a POST whose response was lost, the write happens twice, and the client, who possibly receives a success on the second attempt, believes it placed only one. The tools draw the line at comparable places. With HAProxy, retry-on all-retryable-errors replays a write whose response was lost, whereas retry-on conn-failure only replays what did not reach the server; the lesson Bound time, queues and long-lived flows with HAProxy details the directives.
The rule that follows is to replay what is idempotent, and not to replay a write whose outcome is unknown. A POST you do not know was processed is not replayed. RFC 9110 is even stricter: "A proxy MUST NOT automatically retry non-idempotent requests", with no exception provided for a proxy. HAProxy and Nginx depart from it in only one case: a connection that fails before the request could be sent is replayed whatever the method, since nothing happened on the server side. This is a practice of these two tools, consistent with the spirit of the RFC (knowing the request was never applied), not a permission it grants: do not assume it of another proxy. In the lab, the starting policy replays GET and POST; the fix is to remove what must not be replayed, without forbidding every replay.
Restricting replays is not removing them. A gateway that no longer replayed anything would gain safety while losing its tolerance to passing failures: a read whose connection fails must be able to succeed on the second attempt. It is therefore the selective replay, by method, that holds both requirements.
Under what conditions can a response be cached?
A response can be cached if it is identical for everyone who asks for it, or if the key that files it distinguishes precisely what makes it different. A cache keeps the response to a request in order to hand it to the next one without bothering the backend. It is useful because it saves work, and dangerous for the same reason: the response filed for one user can be handed to another.
Two families of responses are told apart. A public response is the same for everyone: a home page, a catalog, a static file. It is cached without reservation. A private response depends on who asks: an account's content, an invoice, a basket. The application server knows this nature, and declares it with the Cache-Control header. Three directives matter here, and they do not say the same thing (RFC 9111). private forbids storage only to shared caches, such as the gateway's: the user's browser cache may keep the response. no-store goes further and forbids it to all caches, browser included. max-age, finally, authorizes nothing: it sets the freshness lifetime of an already storable response, whereas public explicitly authorizes a shared cache to store it. The general principle, described by RFC 9111, is that the cache applies the origin's contract instead of contradicting it.
In the lab, the policy is more rudimentary: it lists the routes to cache under cache.cacher. The starting flaw is having put /prive in it, a route whose response depends on the user. It is the typical case of an optimization that seems to work: responses arrive faster, the cache looks effective, and nobody sees that each caller receives the first response filed, whatever their identity.
How does variation prevent serving the wrong version?
Variation tells the cache which elements of the request distinguish two responses: anything that changes the content and is not declared makes it serve the wrong version. A cache key is the string that designates a stored entry: two requests that produce the same key receive the same response. By default, it is limited to the address requested. Yet two requests to the same address can expect different content, for example a page in French for one and in English for the other.
The standardized mechanism is called Vary: the server lists in it the request headers its response depends on, for example Vary: Accept-Language. The cache then stores one entry per value of that header. In the lab, the same idea is written under cache.varier_sur, a list of headers that distinguish two entries. A header absent from this list distinguishes nothing: without Accept-Language in the list, the first version filed, for example French, is served to whoever asks for English.
For a private route, two remedies exist, and they do not cost the same. The first is to not cache it: remove it from cache.cacher, and isolation becomes obvious. The second is to vary on identity, for example the Authorization header: each user gets their own entry. For a private response, we recommend the first, and it is a choice of caution, not a rule of the protocol: varying on identity assumes never forgetting the header, and since each entry has a single reader, the cache's gain depends only on how often the same user asks for the same page again, often rarely.
Two traps deserve a name. A key that is too poor mixes responses, and a key that is too rich scatters the cache: varying on the raw value of Accept-Language creates one entry per wording, while the content does not change. And removing the cache everywhere satisfies isolation while failing the promise of speed: the public page must stay served from the cache.
How do you prove a cache works without trusting its headers?
You prove a cache by what the backend produces, never by what the gateway declares. A HIT is a response served from the cache, a MISS a request that goes to the backend. Many gateways add a diagnostic header that shows HIT or MISS, but it is only a declaration by the proxy that you are precisely trying to verify: a header written by hand would show HIT everywhere, including on a response that has just gone through the backend.
The reliable measurement relies on a response that changes at every pass. In the lab, /public carries a randomly drawn number on each request that reaches the application. Two identical requests that receive the same number prove there was a cache, since the backend produced the number only once. Two different numbers prove there was none. No header is taken on trust.
The same logic proves variation and isolation. For language, a request in English must not receive the stored French response. For identity, Alice and Bob each present a token in the Authorization header, and each one's private response carries their name: Bob must never see Alice's, nor the reverse. What makes these findings conclusive is that they read the response body, that is, what the user would actually receive.
The full proof always pairs with a counter-test. Isolation alone: caching nothing satisfies it. Reuse alone: caching everything satisfies it. The work lies in their meeting, a public page really served from the cache and a private page that never leaves its owner. For the Nginx implementation, with the measurement of a HIT by an origin counter, read Nginx cache: prove that no private response leaks.
Which flaws should you look for in a fast policy, and how is each one measured?
A fast but dangerous policy is recognized by four flaws, and each is measured by an effect the client does not see with the naked eye. The table crosses the flaw, its hidden symptom and the measurement that demonstrates it. It reads from left to right: you start from what is written in the policy, move to what really happens, then look at how to establish it.
| Flaw in the policy | What really happens | How to demonstrate it |
|---|---|---|
| A private route is cached | A user receives another user's response | Two accounts request the same route, each must read their own name |
| The cache varies on nothing | One language is served to someone asking for another | A request in French then in English must return two different bodies |
The POST is replayed | An order is counted twice | A single POST: the business counter must go up by one unit only |
| No realistic read timeout | A dead backend holds the caller | /lent answers, and /panne is cut before the caller gives up |
Two of these measurements are critical because they concern an irreparable fault: the leak of private data between users and the doubled order. The other two, the language variation and the timeout, degrade the service without exposing anyone. This ranking guides the order of correction: fix first what cannot be recovered.
The third flaw is the one that cannot be guessed. The client observes exactly the same thing, a failure, whether the order is counted once or twice. Only reading the application's counter, before then after a single attempt, reveals it. That is why the decision to replay is made on the method, and why the verification is done on the system's state, not on the response received.
Practice
The lab gives you a provided cache and timeout policy, which speeds up responses well and carries the four flaws of the table above. You fix it in its YAML file, without touching the application: the cache must keep speeding up what can be sped up, without ever serving one user's data to another, without doubling an order, and without leaving a caller waiting indefinitely.
The validation reads no diagnostic header. It checks that two identical requests on /public receive the same body and that another language receives a different body, that Alice and Bob never share a private response, that an order sent once is counted once in the business counter, and that /lent answers while /panne is cut before twenty seconds.
Key points
- A request waits in several places: connection, read and the caller's patience. The gateway's timeout must stay shorter than the caller's.
- A read timeout is chosen by measuring the slowest legitimate response: too long, it spreads the failure; too short, it creates one.
- An idempotent method (
GET,HEAD,OPTIONS,TRACE,PUT,DELETE) can be replayed; aPOSTwhose response was lost must not be, because the caller cannot tell a lost response from an action that did not happen. - A replay hides the failure from the client. The failure must be counted on the gateway side, otherwise a failing backend stays invisible.
- Selective replay holds both requirements: replay what is idempotent, do not replay a write whose outcome is unknown. Replaying a request that could not be sent is a practice of HAProxy and Nginx, whereas RFC 9110 forbids a proxy to retry a non-idempotent request.
- A private response is not cached under a key that does not distinguish the user. The simplest is not to cache it; the origin declares it through
Cache-Control:privateforbids it to shared caches,no-storeto all caches, andmax-ageauthorizes nothing. - Variation (
Vary, orcache.varier_surin the lab) declares what distinguishes two entries. An undeclared header distinguishes nothing. - A cache is proven by what the backend produces, with a response that changes at every pass, and never by a diagnostic header. The isolation proof comes with its counter-test: the public page stays served from the cache.
Knowledge check
These questions check that you can decide on a timeout, a replay and a caching, and choose the measurement that proves it.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Recognize the needs of long-lived streams and application protocols
A new full quiz with random questions
Work again only on the questions you missed