Topic cluster
Performance and protocols
Keep-alive, connection pooling, buffering, timeout budgets, HTTP/2 and HTTP/3, compression and caching, and what each one costs when it is set wrong.
Proxy performance work is rarely about throughput. It is almost always about four things that are configured once, forgotten, and then produce failures that look like something else entirely: connection reuse, buffering, timeouts and caching.
Each of them has a default that was chosen for a workload that is probably not yours, and several of those defaults are older than the workload you are running. Buffering is on, which is right for slow clients and wrong for streaming. Timeouts default to sixty seconds at several layers at once, which guarantees they are not in a sensible order. And on nginx builds before 1.29.7, upstream connections are not reused at all unless you configure three separate directives correctly.
The four defaults worth changing first#
- Upstream keep-alive. On nginx before 1.29.7 this is off unless you configure it, which means a new TCP connection per request and, at volume, ephemeral port exhaustion. From 1.29.7 nginx caches upstream connections by default, so the first job is finding out which behaviour your build has. Keep-alive and upstream connection pooling.
- Response buffering. On by default, which breaks server-sent events, streaming responses and progress output. Proxy buffering and streaming responses.
- The timeout ladder. Almost never in the right order out of the box. Timeout budgets across a proxy chain, and the timeout ladder checker to test yours.
gzip_proxied. Defaults tooffin nginx, so proxied responses are not compressed at all no matter what else you set. Compression through proxies.
Shedding load rather than queueing it#
Rate limiting at the proxy is the other half of capacity management: timeouts decide how long you wait, rate limits decide how much you accept. It depends on getting the client IP right first, which is why it sits downstream of configuring trusted proxies.
Then the protocol layer#
HTTP/2 and HTTP/3 through proxies covers the part people find counter-intuitive: the protocol version is negotiated per hop, so HTTP/2 at the edge and HTTP/1.1 upstream is normal, and every bug in that translation lives at the boundary. Caching in reverse proxies is the one change that reduces upstream load rather than merely reorganising it.
Every guide in this cluster#
- Caching in reverse proxiesRFC 9111 semantics at a shared cache, nginx proxy_cache defaults, cache key design, invalidation strategies and the misconfiguration that leaks user data.
- Compression through proxiesWhere to compress in a proxy chain, why nginx gzip_proxied defaults to off, how Vary Accept-Encoding keeps shared caches correct, and BREACH-safe choices.
- HTTP/2 and HTTP/3 through proxiesProtocol version is negotiated per hop. How ALPN, h2c, connection coalescing and HTTP/2 to HTTP/1.1 downgrades behave across nginx, HAProxy and Envoy.
- Keep-alive and upstream connection poolingnginx reuses upstream connections only if three directives are set together. Defaults, the idle-close race behind 502s, and port exhaustion maths.
- Proxy buffering and streaming responsesWhy nginx holds your SSE stream to the end, what proxy_buffering does at each leg, and the X-Accel-Buffering header that fixes it per response.
- Rate limiting at the proxyFixed window, sliding window and token bucket compared, nginx limit_req in depth, HAProxy stick tables, Envoy local vs global, and how to choose a limiter key.
- Timeout budgets across a proxy chainThe client-facing timeout must be longest and each inner hop shorter. Defaults for nginx, HAProxy, Envoy and ALB, a worked ladder, retry maths.