Compression through proxies

Compress at exactly one hop: the outermost proxy that terminates TLS, sees the client's real Accept-Encoding, and still has uncompressed bytes to work with. Every hop upstream of that should send plain text, and every hop downstream should pass the encoded body through untouched. Compressing at the origin and at the edge forces the edge to decompress before recompressing, so you pay for three transforms and get the same wire bytes as doing it once. The exception is static assets, which should be compressed offline at maximum effort and served from disk: nothing on the request path matches a brotli level 11 pass, and it costs no CPU per request.

Content codings and where they are supported#

The choice is a ratio-versus-CPU decision that turns on whether compression happens on the request path. Precompressed static files should use brotli at level 11 with a gzip copy as fallback, since that CPU is spent once at build time. Dynamic HTML and JSON should use gzip or brotli at a low level: the ngx_brotli default is 6, and levels beyond roughly 5 on the request path spend disproportionate CPU for a small size gain. Already-compressed formats get nothing, as the decision table below sets out.

The nginx defaults that surprise people#

text/html is not merely the default value of gzip_types, it is always compressed and cannot be removed from the list. That is why gzip_types application/json; still gzips HTML.

There is a second defaults interaction worth internalising, and it applies to nginx before 1.29.7. On those versions proxy_pass speaks HTTP/1.0 unless you set proxy_http_version 1.1, while gzip_http_version defaults to 1.1. In a two-tier nginx deployment (edge nginx proxying to an inner nginx that serves the application), the inner nginx receives an HTTP/1.0 request and therefore refuses to gzip, no matter what gzip_types says. Neither directive is wrong on its own; the pair produces silent non-compression. nginx 1.29.7 changed the proxy_http_version default to 1.1, which removes the interaction on current builds but not on the older packages most distributions ship. The trailing-slash and version subtleties of that directive are covered in nginx proxy_pass and the trailing slash.

Vary, cache correctness, and the encoding mismatch#

A shared cache keys responses on method and URI. If one URI can produce gzip bytes for one client and brotli or identity bytes for another, the cache needs Vary: Accept-Encoding to know that, and it then keeps a variant per distinct Accept-Encoding value. Two things go wrong.

Missing Vary. With gzip_vary off (the nginx default) a downstream cache stores whichever encoding it fetched first and serves it to everybody. A client that sent Accept-Encoding: identity then receives gzip bytes, which surfaces as ERR_CONTENT_DECODING_FAILED in Chrome and "Content Encoding Error" in Firefox. The response is not corrupt; it is correctly encoded for a different client.

Too much Vary. Accept-Encoding values are near-arbitrary strings, so a cache keying on the raw value stores separate entries for gzip, deflate, br, br, gzip, gzip;q=1.0, identity;q=0.5, *;q=0 and so on, fragmenting the object set for no benefit. The fix is normalisation: rewrite the request's Accept-Encoding to one of a few canonical values before it reaches the cache key, so at most three variants exist per URI. The same cardinality problem in its more damaging form is discussed under caching in reverse proxies.

map $http_accept_encoding $normalised_ae {
    default        "";
    "~*\bbr\b"     "br";
    "~*\bgzip\b"   "gzip";
}

location / {
    proxy_set_header Accept-Encoding $normalised_ae;
    proxy_cache      content_cache;
    proxy_cache_key  "$scheme$host$request_uri$normalised_ae";
    proxy_pass       http://app_pool;
}

The explicit $normalised_ae in proxy_cache_key is belt and braces alongside Vary: the cache stays correct even if the upstream forgets to send it.

Decompressing at the proxy for inspection or rewriting#

Anything that reads or edits a response body needs plaintext: WAF response inspection, DLP scanning, sub_filter substitution, injection of a nonce or script tag. nginx's sub_filter operates on the body as received and does nothing to a gzip-encoded body, silently, with no warning in the error log.

Two mechanisms exist. proxy_set_header Accept-Encoding ""; clears the field before forwarding so the upstream returns identity, which is the standard prerequisite for sub_filter; the cost is an uncompressed upstream leg, which matters over a WAN. ngx_http_gunzip_module (gunzip on;) instead inflates a gzip response when the client did not advertise gzip, but it is designed for serving stored-gzip content to legacy clients and does nothing for brotli.

The composition that works is: clear Accept-Encoding upstream, rewrite the body, compress on the way out.

location / {
    proxy_http_version 1.1;
    proxy_set_header   Connection "";
    proxy_set_header   Accept-Encoding "";   # required for sub_filter

    sub_filter '</head>' '<script nonce="$request_id"></script></head>';
    sub_filter_once on;

    gzip            on;
    gzip_vary       on;
    gzip_proxied    any;
    gzip_min_length 1024;
    gzip_comp_level 5;
    gzip_types      text/plain text/css application/json
                    application/javascript text/xml application/xml
                    image/svg+xml application/wasm;

    proxy_pass http://app_pool;
}

For static assets, precompression removes the request-path cost entirely:

location /assets/ {
    brotli_static on;   # serves file.js.br when the client accepts br
    gzip_static   on;   # falls back to file.js.gz
    gzip          off;  # no on-the-fly work here
    add_header    Cache-Control "public, max-age=31536000, immutable";
}

gzip_static always serves the .gz file even to clients that did not ask for gzip, relying on gunzip to inflate for them, which lets you store only the compressed copy.

What to compress and what to leave alone#

The "under roughly 1 KiB" row is why gzip_min_length 20 is a poor default to inherit: below a single network segment, compressing changes nothing observable at the client and costs CPU on every request. 1024 is a defensible starting point.

Compression as a side channel: BREACH and CRIME#

Compression ratio leaks information about plaintext. If an attacker can inject a guess into a compressed message that also contains a secret, a shorter output means the guess matched existing text.

CRIME (2012) exploited this at the TLS layer and against SPDY header compression. It is closed: TLS-level compression is not negotiated in practice and TLS 1.3 removed the mechanism entirely. HTTP/2 replaced SPDY's shared compressor with HPACK, which is designed to bound this class of leak (never compressing a value against attacker-controlled data in an unrestricted way), though sensitive header values should still be marked never-indexed.

BREACH (2013, CVE-2013-3587) moved the same idea to HTTP response bodies, which are compressed regardless of TLS version. Three conditions must all hold: the body is compressed, it reflects attacker-controlled input, and it contains a secret such as a CSRF token. Break any one. Ranked by practicality: keep secrets out of responses that reflect input; mask CSRF tokens with a per-response random pad so the bytes differ every time; and only as a last resort disable compression on the specific endpoints meeting all three conditions. Disabling compression site-wide is the wrong trade for almost every site, so this belongs on your reverse proxy security checklist as a per-endpoint decision rather than a global switch.

Decompression bombs on the request side#

Content-Encoding is legal on requests, and a small compressed body can expand enormously. nginx does not decompress request bodies, so client_max_body_size (default 1m) limits only the compressed size, and the expansion happens in your application or WAF where no such limit exists.

Defences, applied together: reject Content-Encoding on requests unless an endpoint genuinely needs it (most APIs do not); where it is needed, decompress through a streaming reader with a hard output cap and abort at the cap rather than after buffering; enforce a maximum expansion ratio as well as an absolute cap; and apply the same limits to every body-inspecting component in the chain, since each decompresses independently. Limits across a chain are covered in header and body size limits at the proxy.

Failure modes#

Double compression. An upstream sends Content-Encoding: gzip and a proxy compresses again, producing Content-Encoding: gzip, gzip. A correct client applies the codings in reverse order and copes; a sloppy one shows ERR_CONTENT_DECODING_FAILED. nginx skips its gzip filter when Content-Encoding is already set, but a chain of CDN, WAF appliance and application framework all configured to compress can still produce it. Check with curl -H 'Accept-Encoding: gzip' -sI and look for a comma in the header.

sub_filter silently does nothing. The upstream returned a gzip body, so there was no plaintext to substitute in. There is no error and no warning: the page simply lacks the injected content. Confirm by checking whether the upstream response carries Content-Encoding, then add proxy_set_header Accept-Encoding "";. The same class of failure affects sub_filter when the target string spans a buffer boundary, which is a separate issue covered under proxy buffering.

Server-Sent Events stop being incremental. gzip on text/event-stream makes the compressor hold data until it has enough to emit, so events arrive in bursts or only at connection close. Turn compression off for that content type and disable response buffering (proxy_buffering off; or an X-Accel-Buffering: no response header). Long-poll and streaming JSON endpoints behave the same way.

A cache serves brotli to a gzip-only client. Missing Vary: Accept-Encoding, or a cache key omitting the encoding. The failure is client-dependent so it looks intermittent. Set gzip_vary on and include the normalised encoding in the cache key.

Compression disappears behind a CDN. gzip_proxied off plus a Via header from the CDN. Response sizes jump and no nginx configuration changed. Set gzip_proxied any;.

Nothing is compressed except HTML. gzip_types was never set, so it is still text/html. JSON APIs in particular go uncompressed for years because the HTML compresses fine and nobody checks.

Content-Length disagreement after compression. A proxy that compresses a body but forwards the original Content-Length truncates the response. nginx drops Content-Length and switches to chunked encoding when it compresses; a hand-written proxy that does not is the usual culprit. Symptom: curl: (18) transfer closed with N bytes remaining.

Frequently asked questions#

Should I compress at the origin or at the edge proxy?#

At the edge, at the hop that terminates TLS and sees the client's real Accept-Encoding. That hop is usually horizontally scalable, knows what the client supports, and is the last place the bytes need to be plaintext. The origin should send identity so any hop in between can inspect, rewrite or cache the body without a decompress-recompress cycle.

Why does nginx not gzip my JSON responses?#

Because gzip_types defaults to text/html only. Add the types you serve, for example gzip_types application/json application/javascript text/css;. If JSON is still uncompressed, check that the response exceeds gzip_min_length, that the request is HTTP/1.1 (gzip_http_version defaults to 1.1), and that gzip_proxied is set if a Via header is present.

Does gzip_proxied control compression of proxy_pass responses?#

Not directly. nginx determines that a request is "proxied" by the presence of a Via request header field, which is inserted by a proxy in front of nginx, not by nginx's own proxy_pass. A standalone nginx compresses upstream responses with the default gzip_proxied off; the same setup stops compressing once a CDN adds Via.

Is brotli always better than gzip?#

Better ratio for text, but not universally usable. Browsers advertise br only over HTTPS, and high brotli levels are too slow for per-request use. The practical rule is brotli level 11 for precompressed static files and gzip or low-level brotli for dynamic responses, with gzip always kept as the fallback.

How do I make sub_filter work when the upstream sends gzip?#

Add proxy_set_header Accept-Encoding ""; in the location so the upstream returns an uncompressed body, then re-enable gzip on the nginx side so the client still gets a compressed response. Without clearing Accept-Encoding, sub_filter sees compressed bytes, matches nothing, and reports no error.

Is it safe to compress pages containing CSRF tokens?#

Only if the page does not also reflect attacker-controlled input, which is the BREACH precondition. The robust mitigation is to mask the token with fresh per-response randomness so its compressed representation changes every time, rather than disabling compression across the site. Endpoints that reflect a query parameter into a page containing a secret are the ones to examine first.

How do I stop a gzip bomb in a request body?#

Reject Content-Encoding on requests unless an endpoint needs it, and where it is needed, stream the decompression with both an absolute output cap and a maximum expansion ratio, aborting as soon as either is exceeded. client_max_body_size does not help on its own because it bounds the compressed size only.