Proxy buffering and streaming responses

Buffering means the proxy accumulates a message (or part of it) in memory or on disk before passing it on, instead of relaying bytes as they arrive. nginx buffers both directions by default: proxy_buffering on for responses and proxy_request_buffering on for request bodies. That is why a Server-Sent Events stream, an LLM token stream or a chunked progress log arrives at the browser in one burst at the end rather than incrementally. The two-line fix is proxy_buffering off; in the location, or, better, have the upstream return X-Accel-Buffering: no on exactly the responses that need to stream.

HAProxy, Envoy, Caddy and Traefik all stream by default and have the opposite problem: you have to opt in to buffering when you want it.

The four buffering decisions on one request#

Header buffering is never optional in any HTTP/1.1 proxy: routing decisions require the full header block, so it must be read before anything else happens. Body buffering is the part that is configurable, and it is what breaks streaming. Header size limits are a different subject entirely, covered in header and body size limits at the proxy.

Why response buffering is the default, and what it buys#

Consider an upstream with a fixed worker pool: gunicorn with 8 sync workers, PHP-FPM with 20 children, a servlet container with a bounded thread pool. Each worker is occupied for the whole duration of writing its response, so a mobile client that takes 40 seconds to drain a 2 MB response blocks that worker for 40 seconds doing nothing but waiting on TCP backpressure.

With proxy_buffering on, nginx reads the response as fast as the upstream can produce it, frees the worker, then dribbles bytes out at whatever rate the client manages. An event-driven proxy holding a socket costs kilobytes; a blocked application worker costs a process. That is the entire rationale, and the reason nginx ships with it enabled.

The corollary defines when you should turn it off: disable response buffering only when time-to-first-byte for partial content is part of the contract. If the client cares when byte 0 arrives relative to byte N, buffering is wrong. If the client only cares when the last byte arrives, buffering is right and disabling it hurts your upstream.

The nginx response buffer chain#

Four directives interact, and getting them wrong produces a config that will not load.

The flow: headers go into one proxy_buffer_size buffer, body bytes fill the proxy_buffers set, and overflow that the client has not drained is written to a temp file under proxy_temp_path up to proxy_max_temp_file_size. Beyond that, nginx stops reading and the upstream blocks, which is normal backpressure rather than an error.

proxy_max_temp_file_size 0; is the common hardening move: the response stays entirely in memory and full buffers apply backpressure to the upstream instead of touching disk. That converts a disk-space risk into an upstream-worker-occupancy risk, which is the trade-off you are really choosing.

A misconfiguration that fails at startup rather than at runtime:

nginx: [emerg] "proxy_busy_buffers_size" must be less than the size of all
"proxy_buffers" minus one buffer in /etc/nginx/conf.d/api.conf:14

proxy_busy_buffers_size must be smaller than proxy_buffers total minus one buffer, because nginx needs at least one free buffer to keep reading from the upstream while the busy ones are being flushed to the client.

Request body buffering, and the two gotchas#

proxy_request_buffering on (the default since nginx 1.7.11) means nginx reads the entire request body before using an upstream connection. For a 500 MB upload, the client's whole transfer completes against nginx, on disk, before the application sees a byte. The warning everyone eventually greps for:

2026/09/08 09:12:44 [warn] 1123#1123: *5581 a client request body is buffered
to a temporary file /var/lib/nginx/body/0000000012, client: 10.1.0.7,
server: api.example.com, request: "POST /v1/upload HTTP/1.1", host: "api.example.com"

This is emitted whenever the body exceeds client_body_buffer_size, which defaults to two memory pages: nginx documents that as 8k on x86, other 32-bit platforms and x86-64, and usually 16k on other 64-bit platforms. The common case, x86-64, is therefore 8k, not 16k. It is a warn, not an error, and the request succeeds. If it fires on every request and the bodies are small, raise client_body_buffer_size to cover your p99 body size. If the bodies are genuinely large, the warning is telling you the truth and the fix is to stream instead.

Two gotchas when disabling it:

1. proxy_request_buffering off requires proxy_http_version 1.1. nginx documents that a request body sent by the client with chunked transfer encoding is buffered regardless of the directive unless HTTP/1.1 is enabled for proxying, because HTTP/1.0 upstreams cannot receive a chunked body and nginx must compute a Content-Length.
2. Unbuffered requests cannot be retried. With the body streamed straight through, nginx no longer holds a copy, so it cannot pass the request to the next server in the upstream group on failure. You lose proxy_next_upstream for those routes. That interacts directly with the pooled-connection race described in keep-alive and upstream connection pooling: the retry that normally papers over an idle-close 502 is not available on a streaming upload.

location /v1/upload {
    proxy_pass http://app;
    proxy_http_version 1.1;
    proxy_request_buffering off;   # stream the upload to the app
    proxy_buffering off;           # stream the progress response back
    client_max_body_size 0;        # 0 disables the size check entirely
    proxy_read_timeout 300s;
}

X-Accel-Buffering: the answer most people want#

Rather than a config change per streaming route, the upstream application can disable buffering for one response:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
X-Accel-Buffering: no

nginx honours this on the proxy, FastCGI, uwsgi, SCGI and gRPC paths. It is the right mechanism because the application, not the proxy operator, knows whether a given response is a stream. A single route that returns JSON for one query and SSE for another can decide per request.

Two properties that catch people out:

• nginx consumes the header. X-Accel-Buffering is in nginx's default hidden-headers list, alongside X-Accel-Redirect, X-Accel-Expires, X-Accel-Limit-Rate and X-Accel-Charset. It is stripped from the response sent to the client. In a two-tier nginx chain (edge nginx in front of a per-service nginx), the inner nginx acts on it and the outer nginx never sees it, so the outer tier buffers anyway. The fix on the inner tier is proxy_pass_header X-Accel-Buffering;.
• It is nginx-specific. HAProxy, Envoy, Caddy, Traefik and every CDN ignore it. It is not in any RFC. Any non-nginx hop in the chain needs its own configuration.

How the major proxies compare#

The pattern is clear: nginx is the outlier, because it was designed around protecting fork-per-request and worker-pool application servers from slow clients. The others were designed later, for upstreams that handle their own concurrency.

HAProxy's model is genuinely different#

HAProxy does not have a "buffer the response" mode to turn off. Each stream owns a buffer per direction, sized by tune.bufsize (default 16384 bytes), and data is forwarded as soon as it is available. There is no accumulation of a whole message and no disk spill, which is why SSE and streaming JSON work through HAProxy with no configuration at all.

The consequences run in the other direction:

• The complete request header block must fit in one buffer, minus tune.maxrewrite reserved for header insertion. Oversized headers are rejected rather than spilled.
• Body inspection is opt-in and bounded. option http-buffer-request makes HAProxy wait for the full body (up to one buffer) before forwarding, which is what ACLs on req.body require. It converts that frontend into a buffering proxy, so do not enable it globally on a config that also carries uploads.
• Long-lived streams are governed by timeout tunnel after an upgrade and timeout server before it. Getting these wrong truncates a working SSE stream at exactly the timeout value, which is the subject of timeout budgets across a proxy chain.

For Envoy, the equivalent trap is not the buffer filter itself but any filter that needs a whole body: ext_authz with with_request_body, a Lua or WASM filter that calls for the full body, or a rate limiter keyed on body content. Enabling one of these silently converts a streaming path into a buffered one. Envoy's per_connection_buffer_limit_bytes (default 1 MiB) is a flow-control watermark rather than an accumulation target, so it does not cause buffering by itself.

Buffering you cannot turn off#

Some layers buffer because their feature set requires it, and no header will change that:

• Any body-scanning WAF or DLP appliance. It cannot decide to allow a body it has not read. If a rule inspects the response body, the response is buffered by definition.
• Edge compression and response transformation. Rewriting HTML or compressing a response requires enough input to work with. See compression through proxies for the interaction between gzip buffers and streaming.
• Caching. A response being written into a cache entry is being accumulated. nginx will not stream and store the same response, and neither will a CDN. If you need both behaviours, you need two routes.
• Amazon API Gateway REST APIs. They buffer the full integration response and enforce a 10 MB integration response payload quota that cannot be raised. Streaming responses through a REST API is not a configuration problem, it is unsupported. Lambda response streaming works through Function URLs, not through API Gateway REST APIs.

So when a stream works locally and not in production, the diagnostic question is not "which proxy setting did I miss" but "which hop has a feature enabled that requires the whole body".

Worked example: proving where the buffering is#

# -N disables curl's own output buffering. --trace-time stamps each read.
$ curl -N --trace-time -sS https://api.example.com/v1/events

13:04:11.002214 == Info: Connected to api.example.com
13:04:11.310442 <= Recv header, 17 bytes (0x11) HTTP/1.1 200 OK
13:04:19.884120 <= Recv data, 4096 bytes
13:04:19.884390 <= Recv data, 4096 bytes
13:04:19.884511 <= Recv data, 2210 bytes

Headers at 11.31s, then nothing for eight seconds, then everything at once in buffer-sized reads. That timing signature (a long silence followed by reads that are multiples of 4096) is response buffering, not a slow upstream. Contrast with the same endpoint hit directly:

$ curl -N --trace-time -sS http://10.0.2.11:8080/v1/events
13:06:02.114201 <= Recv data, 63 bytes
13:06:03.118776 <= Recv data, 61 bytes
13:06:04.121003 <= Recv data, 58 bytes

Small reads once per second: the application streams correctly and the proxy is holding it. Repeat at each hop (origin, inner proxy, edge, CDN); the first hop showing the clumped pattern is the culprit.

Failure modes#

SSE arrives in one burst at the end. Symptom: EventSource fires no message events until the connection closes, or fires them all at once. Cause: proxy_buffering on with a response smaller than the buffer set, so nothing is flushed until completion. Fix: proxy_buffering off on that location or X-Accel-Buffering: no from the upstream. If events still clump after that, check gzip: nginx's gzip filter accumulates into gzip_buffers, so set gzip off on streaming locations.

SSE dies after exactly 60 seconds. Symptom: the stream works then the connection drops on a round number. Cause: proxy_read_timeout (default 60s) elapsed between events. Fix: raise it for the location and emit a comment line (: keep-alive) from the application at a shorter interval.

Upload fails at exactly the buffer limit. Symptom: 413 Request Entity Too Large, with client intended to send too large body: 5242880 bytes in the error log. Cause: client_max_body_size, default 1m. This is a size policy check, not buffering. Raising client_body_buffer_size will not fix it and raising client_max_body_size will not stop the temp-file warning. They are independent controls that are constantly confused.

Disk fills with proxy temp files. Symptom: [crit] ... writev() "/var/lib/nginx/proxy/3/07/0000000073" failed (28: No space left on device) while reading upstream, and 502s once the partition is full. Cause: large buffered responses spilling to proxy_temp_path faster than clients drain them. Fix: proxy_max_temp_file_size 0; to force memory-only buffering with upstream backpressure, or move proxy_temp_path to a dedicated volume and monitor it.

Streaming works through nginx and breaks at the CDN. Symptom: curl -N against the origin streams, against the public hostname does not. Cause: X-Accel-Buffering is nginx-specific and was consumed by nginx anyway; the CDN has its own policy. Fix: configure the CDN, and check whether a body-scanning or transformation feature is enabled on that path.

Response buffering off, then intermittent 502s. Symptom: unbuffered routes fail more often than buffered ones. Cause: with buffering disabled nginx cannot retry to another upstream once response bytes have been forwarded, so transient upstream failures that were previously invisible now surface. See 502 vs 503 vs 504 for classifying them.

Frequently asked questions#

How do I disable buffering in nginx for Server-Sent Events?#

Set proxy_buffering off; in the SSE location, along with proxy_http_version 1.1; and proxy_read_timeout long enough to cover the gap between events. The cleaner alternative is for the application to return X-Accel-Buffering: no on SSE responses, which requires no nginx change and only affects the responses that need it.

What does X-Accel-Buffering: no actually do?#

It tells nginx to disable response buffering for that single response, overriding proxy_buffering on. nginx strips the header before sending the response to the client, because it is in nginx's default hidden-headers list. Only nginx understands it; other proxies and CDNs ignore it entirely.

Does proxy_buffering off also stream the request body?#

No. proxy_buffering controls the upstream-to-client direction only. Request bodies are governed by the separate proxy_request_buffering directive, which also defaults to on. Streaming an upload requires proxy_request_buffering off; plus proxy_http_version 1.1;, and disables proxy_next_upstream retries for that request.

Why does nginx log "a client request body is buffered to a temporary file"?#

Because the request body exceeded client_body_buffer_size (two memory pages: 8k on x86, other 32-bit platforms and x86-64, usually 16k on other 64-bit platforms) and nginx spilled it to disk. It is a warning, not a failure, and the request still succeeds. Raise client_body_buffer_size if the bodies are legitimately small, or switch that route to unbuffered request streaming if they are large.

Is a 413 caused by buffering?#

No. A 413 comes from client_max_body_size (default 1m), which is a size policy applied before and independently of any buffering decision. The log line is client intended to send too large body. Buffering controls where the body is held, not whether it is allowed.

Does HAProxy buffer responses like nginx does?#

No. HAProxy uses a fixed buffer per stream direction (tune.bufsize, default 16384 bytes) and forwards data as it arrives rather than accumulating the whole message, so streaming works with no configuration. Request-body buffering is opt-in via option http-buffer-request. The configuration and trade-offs are covered in HAProxy configuration for HTTP reverse proxying.

Why does LLM token streaming work locally but not in production?#

Almost always because a proxy in the production path buffers and the local setup has no proxy. Test each hop with curl -N --trace-time and look for the first one where reads arrive in clumps at buffer-sized boundaries. On nginx it is proxy_buffering; at a CDN it is usually a body-scanning or transformation feature that cannot be selectively disabled per response.