An entry point tuned for web pages cuts, delays or refuses long-lived streams (WebSocket, SSE, gRPC) while the service behind it works perfectly. This lesson is for someone who already knows how to publish an application behind a proxy and discovers that a notification arrives late, that a real-time conversation drops after a minute or that a gRPC call fails. It explains what a long-lived stream is, why a proxy cuts it without any failure, and what each protocol asks for in particular, without configuring any tool. The directives live in the pages for Nginx and HAProxy.
What you will learn
The lesson goes from the definition of a long-lived stream to how to measure it.
- Distinguish a long-lived stream from an ordinary request, and name WebSocket, SSE and gRPC.
- Explain why an idle timeout cuts a stream that is not broken.
- Describe what each of the three protocols requires from a proxy.
- Choose an idle timeout from the application's heartbeat (its small regular "I'm alive" message) and not from the stream's duration, and recognize the limits a heartbeat does not protect against.
- Prove that a stream flows by timing the arrival of each message.
Prerequisites
The lesson assumes you can read an HTTP exchange and place the proxy on the path of a request.
- Read a request, a response, their headers and their status codes: Read HTTP before configuring a proxy.
- Place the entry point between the caller and the application: Publish a web service.
- Know the idea of a timeout and of a read timeout: Master timeouts, retries and caching without altering responses.
The lab provides the machine, the service and the measurement client: you install nothing.
What is a long-lived stream, and how does it differ from a page?
A long-lived stream is a connection that stays open while data flows through it in bursts, whereas an ordinary request opens, receives its response and closes. An entry point (or reverse proxy, the server placed in front of the application that relays requests to a backend) is designed for the second case. It assumes a response arrives quickly, ends, and that an idle connection is a dead connection.
Three protocols produce long-lived streams. A WebSocket is a connection that starts as an HTTP request, then switches, after a handshake (an initial exchange where the client asks to change protocol), into a channel where client and server send messages whenever they want. SSE (Server-Sent Events) is an HTTP response that does not end: the server writes events into it over time, and the client receives them one by one. gRPC is a remote procedure call protocol that runs over HTTP/2, and that can also keep an exchange open over time.
An application protocol is the convention that the application and its client follow on top of the transport, here WebSocket, SSE or gRPC. The proxy does not always understand it, but it must let it through. That is why long-lived streams ask something different of the proxy than pages do: the same entry point can serve the site perfectly and get these three routes wrong, each for its own reason.
| What changes | A page | A long-lived stream |
|---|---|---|
| Connection duration | A few moments | Minutes, hours |
| End of the response | The response ends | The response does not end, or the connection is a channel |
| Data | Sent in one block | Sent in bursts, with silences |
| Silent connection | Sign of a failure | Normal situation |
The last row is the one that traps most. It prepares the next question.
Why does an entry point cut a stream when nothing is broken?
A proxy cuts a stream when no data has flowed for the delay it set itself, not because the connection lasts a long time. This delay is an idle timeout. In Nginx, proxy_read_timeout is an example; in HAProxy, it is timeout server. The timeout restarts from zero at every byte received: a stream that emits a message every thirty seconds never falls under a sixty-second timeout, whereas a stream silent for a minute is closed.
A total-duration limit, an absolute deadline that some products offer in addition (it is one of the three timeouts of a request in the lesson Master timeouts, retries and caching without altering responses), is of a different nature: it cuts even an active stream, and the client must then reconnect. It is not what explains a silent stream cut after a minute.
The trap comes from the fact that, for a page, this behavior is exactly what you want. A response that gives no sign of life is probably blocked, and cutting it frees the connection. For a stream, silence is legitimate: a notification application may have nothing to say for five minutes. The proxy, which cannot tell a quiet stream from a blocked backend, applies the same rule to both.
The result is baffling. The service works, the network works, no error appears on the application side, and yet the connection drops, often at regular intervals, after the same number of seconds. This regularity is the clue: a cut that falls at exactly sixty seconds suggests a timeout set to that value rather than an incident. Read it as a clue, to be confirmed by reading the configuration.
The WebSocket case adds a subtlety. After the 101 Switching Protocols response (the code by which the server agrees to change protocol), the connection becomes a tunnel: the proxy now relays bytes without interpreting HTTP any more. In HAProxy, this phase falls under a timeout distinct from the one for reading the response, timeout tunnel: when defined, it replaces timeout client and timeout server as soon as the connection has become a tunnel, and like them it measures idleness, in both directions. In that tool, it is therefore the one to set for a WebSocket, and the other tools each have their own settings. The principle is common: every phase of a conversation has its timeout, and the tunnel is one more phase.
Why does an SSE stream arrive in a block instead of as it goes?
A proxy that buffers the response holds back the events of an SSE stream until it has enough, and delivers them together, too late. Buffering means reading the backend's response as fast as possible, storing it in memory, then handing it to the client at its own pace. For a page, this is generally the right setting: the backend is freed right away and a slow client does not hold it up.
For a stream, it is the opposite of what is needed. The server emits one event per second, but the proxy waits until it has filled its buffer before writing to the client. The client receives all the events, in the right order, without loss, but in a burst, when the buffer empties. The rhythm, which is the very point of a stream, is gone, and a dashboard meant to show live notifications shows them in batches.
This flaw passes naive tests. A test that counts the events received at the end of the stream gives a correct result with a proxy that holds everything back, since everything ends up arriving. What distinguishes a stream from a batch is the moment of each arrival, not their number. Hence a measurement rule that comes back throughout the lesson: you check a stream by timing each message, never by reading a final body.
Note also that the buffer may be somewhere other than in the proxy you are tuning. A client that buffers its own reading produces the same delay in front of a perfectly configured proxy: before blaming the entry point, make sure the measurement tool holds nothing back on its side.
What does each application protocol ask of a proxy?
Each protocol requires something different: WebSocket wants the handshake passed on, SSE wants nothing held back, gRPC wants HTTP/2 spoken all the way to the backend. The table summarizes these requirements, then each row is explained.
| Protocol | What it requires from the proxy | Symptom when it is missing |
|---|---|---|
| WebSocket | Let the protocol-change request through, then relay the tunnel | The connection does not establish, or establishes without anything flowing |
| SSE | Hold nothing back, tolerate silences | Events arrive together, or the connection drops silently |
| gRPC | Speak HTTP/2 and pass on the final statuses | The call fails at the transport level |
For WebSocket, the request to switch goes through headers (Upgrade in particular) that describe the connection between two neighboring links and not the end-to-end request. They are called hop-by-hop, and a proxy does not forward them on its own: you have to ask it to. Without them, the backend answers as to an ordinary request and the handshake fails. Even a successful handshake proves only half the path: a 101 shows that the backend accepted, not that messages then flow. You need a message that comes back.
For SSE, the requirement is negative: do not buffer anything, and do not take silence for an end. The event stream is a long HTTP response with no planned end. Since it can stay quiet for a long time between two state changes, it depends on the idle timeout seen above.
For gRPC, two new notions. HTTP/2 is the version of the protocol that multiplexes several exchanges over a single connection, that is, interleaves them without mixing them. It can also send trailers, headers placed after the response body. gRPC puts its final status there. gRPC requires HTTP/2 end to end, all the way to the backend: a proxy limited to HTTP/1 toward the backend can carry neither interleaved exchanges nor trailers, and the call fails.
This last point has a diagnostic consequence: a gRPC call refused by the application carries an HTTP 200 code, and it is the gRPC status, in the trailers, that says what happened. A proxy that confuses the two makes a definitive refusal pass for a passing failure, and a client configured to replay failures can then replay in vain. The exact gestures are in Nginx: publish WebSocket, SSE and gRPC without cuts. The page Bound time, queues and long-lived flows with HAProxy covers WebSocket and SSE, and leaves gRPC aside.
How do you choose a timeout: against the heartbeat or against the duration?
An idle timeout is set against the application's heartbeat, that is, the interval between its regular messages, and not against the duration of the stream. The heartbeat (or ping) is a small message that the application sends at regular intervals to say it is alive, even when it has nothing useful to transmit. In SSE, it often takes the form of a dedicated event or a comment line that the client ignores.
The reasoning is as follows. The timeout restarts at every byte: as long as the heartbeat falls before it, the connection is never judged idle, even if the stream lasts hours. Setting the idle timeout to the expected duration of the session, for example one hour "to be safe", is therefore a bad setting. It brings nothing if the heartbeat is well placed, and it turns a protection into a risk: a backend that disappears without warning can then keep its connection open for a whole hour.
The practical rule is deduced in three steps. Note the application's heartbeat interval, or failing that its longest normal silence. Set the timeout a little above it, with a margin that absorbs a delayed heartbeat. Apply it to the stream's route only, because ordinary pages keep a short timeout that frees their dead connections.
This rule holds for an idle timeout, and only for it. A heartbeat does not protect against an absolute deadline like the one described above, since the deadline counts elapsed time without checking whether bytes are flowing. The measurement shows it: on Traefik 3.7.15, a three-second write deadline cuts an SSE stream that nevertheless emits one event per second, so never silent, after three events; the same deadline raised to twenty seconds lets twenty through. Before setting a timeout against the heartbeat, check which of the two your product offers, which Master health, timeouts and long-lived flows with Traefik details for that product.
Two precisions of judgment, so they are not taken for facts. First, the exact margin has no universal value: it depends on the application's tolerance for a lost heartbeat. Second, a heartbeat is not mandatory for a stream to survive: without it, you must accept a longer timeout, at the price of slower failure detection. It is a trade-off, not a rule.
How does a client know the stream is dead and not just silent?
A silent stream and a dead stream look alike as long as nothing distinguishes the absence of messages from a failure, hence two tools: the heartbeat and the explicit signaling of the failure. On the client side, a stream with no news for ten seconds can mean the application has nothing to say, or that the service feeding it has gone down. No byte settles it.
The heartbeat settles the first half. If the server emits a heartbeat every few seconds, the client knows that a silence longer than that interval is no longer normal. A heartbeat whose interval is set to thousands of seconds never triggers in practice: it amounts to no heartbeat, and the client remains unable to tell a silent stream from a dead stream.
The explicit signaling settles the second half. When the data source goes down, the stream server can send a dedicated failure event, then close the connection cleanly. The client thus learns right away what is happening, instead of waiting for a silence that never ends. Leaving the stream open and mute forever is what informs the client least: it receives neither data nor explanation, and cannot decide to reconnect.
These two mechanisms belong to the service, not to the proxy. The entry point passes on what the application writes: a failure event must therefore come from it. It is a good reason to read an incident in two steps: did the proxy cut, or did the service stop emitting? The answer directs the fix to the right place.
How do you prove a stream really works?
You prove a stream by timing the arrival of each message, not by reading the body at the end, because only the moment of arrival distinguishes a stream from a batch. A connection that opens, a 200 or 101 code and a complete body at close say nothing about the rhythm. Four measurements cover the problems of this lesson.
- A WebSocket round trip: you send a message and measure the time before its echo. An abnormal delay betrays a delay injected somewhere.
- The gap between two SSE events: if the producer emits one event per second, the gaps measured must stay close to that second. Gaps of tens of seconds followed by a burst point to a buffer.
- The first heartbeat: it must arrive within a few seconds, not thousands.
- The failure: you stop the source, and the stream must signal a failure event then close, instead of hanging.
Each of these measurements fails for a precise reason, which makes the diagnosis readable. An echo that comes back but late does not have the same cause as an echo that never comes back. Events that all arrive, but in a block, point to the buffer and not to the network.
A word on reading the results. A measurement that passes once does not prove robustness: a stream can hold for five seconds and drop at the sixtieth. To judge an idle timeout, you must therefore leave the stream silent for longer than the timeout, and see whether it survives. This is the counter-test of the timeout and heartbeat rule.
Practice
The lab provides a notification service that exposes a WebSocket and an SSE stream fed by a producer that emits one notification per second. Its configuration carries four mistakes at once: a delayed echo, events held in a buffer, a heartbeat that never triggers in practice, and a producer failure that the stream does not signal. You fix them in a settings file (called a policy in the lab), with the provided measurement client that times each stream. The validation runs this client and checks that the WebSocket echo comes back in under one second, that SSE events arrive one by one (at least two in four seconds, with less than 2.5 seconds between two), that a first heartbeat arrives before four seconds in an observation window of five seconds, and that a failure event and a close event are emitted when the producer is stopped.
The lab does not measure the exact duration of an idle timeout, nor the directives of a specific proxy: those points remain carried by the tool pages.
Key points
- A long-lived stream is a connection that stays open with data in bursts; an entry point tuned for pages treats it as suspect.
- A read timeout is an idle timeout: it restarts at every byte received and does not limit the total duration.
- After a 101 response, the connection becomes a tunnel; in HAProxy, it falls under a timeout distinct from reading, and each tool has its own settings.
- A buffer loses no SSE event but destroys their rhythm: the stream becomes a batch.
- WebSocket requires the handshake to be passed on, SSE that nothing be held back, gRPC that HTTP/2 be spoken all the way to the backend.
- An idle timeout is set against the application's heartbeat, not against the duration of the stream; but a heartbeat does not protect against an absolute deadline, measured on Traefik 3.7.15.
- A silent stream is only told from a dead stream by a heartbeat and an explicit signaling of the failure, two things that belong to the service.
- A stream is proven by timing each message, never by reading a final body.
Knowledge check
These questions check that you can explain a stream cut, choose a timeout and decide on the measurement that proves a stream.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Load balance a service and check its health
A new full quiz with random questions
Work again only on the questions you missed