nginx as a reverse proxy: a complete configuration

A working nginx reverse proxy is four things: an upstream block naming the backends, a server block that owns a listen socket and a server_name, a location that calls proxy_pass, and a header block that repairs what nginx's defaults would otherwise send. The defaults are the part that surprises people, because they describe an origin server sitting on the same LAN in 2004: Host set to the upstream's own name, a 1 MB body limit, 60 second timeouts everywhere and, on builds before 1.29.7, HTTP/1.0 to the upstream with no connection reuse. None of those are wrong; most of them are wrong for a modern application backend.

One thing is deliberately not here. The rule that decides what URI the upstream receives, including the trailing slash on proxy_pass, lives in nginx proxy_pass and the trailing slash, and the proxy_pass URI simulator answers the same question interactively.

How a request finds a server block#

Contexts nest: http contains server and upstream; server contains location; location can contain location. Inheritance flows downward, with a critical exception covered below.

Selection happens in two stages. First socket: nginx collects every server block whose listen matches the address and port the connection arrived on (a listen with no address means all addresses). Then name: the request's Host is matched against server_name in a fixed order, exact name, then the longest wildcard beginning with an asterisk (*.example.com), then the longest wildcard ending with one (mail.*), then the first matching regular expression in order of appearance.

If nothing matches, nginx uses the default_server for that socket. If no block is marked default_server, the first block with a matching listen becomes it, so an unrelated site silently answers for every unknown hostname. Make it explicit:

server {
    listen      80  default_server;
    listen      [::]:80 default_server;
    listen      443 ssl default_server;
    listen      [::]:443 ssl default_server;
    ssl_reject_handshake on;   # nginx 1.19.4+, no certificate needed here
    server_name _;             # "_" is just an invalid name, not a wildcard
    return      444;
}

444 is nginx's own non-standard code: close the connection with no response at all. It is the right answer for a Host you do not serve, because it costs one packet, produces no error page to fingerprint, and makes host-header scanning obvious in the log. ssl_reject_handshake on lets the same block cover TLS without a certificate; nginx aborts the handshake with unrecognized_name rather than presenting some other site's certificate. An empty server_name "" matches requests carrying no Host at all.

Location matching precedence in full#

This is the ranking, applied in order:

1. location = /path (exact). If it matches, the search stops immediately.
2. All prefix locations are compared; the longest matching prefix is remembered. If that longest match was declared with ^~, the search stops and it is used.
3. Regular expression locations (~ case sensitive, ~* case insensitive) are tested in the order they appear in the file. The first one that matches wins, and it beats the remembered prefix.
4. If no regex matches, the remembered longest prefix is used.

Two consequences do the damage: regexes beat longer prefixes, and regexes are ordered by file position rather than specificity.

server {
    location /static/          { root /srv/www; }
    location ~* \.(js|css)$    { expires 30d; }
}

Request: GET /static/app.js. The intuitive answer is /static/, because it is longer, more specific and written first. The actual answer is the regex. Step 2 remembers /static/ but does not stop, step 3 finds \.(js|css)$, and the regex wins. That block has no root, so it inherits the server-level one and serves from the wrong directory or 404s. Had the prefix block been a proxy_pass, the request would stop being proxied at all.

The fix is one character pair:

location ^~ /static/ { root /srv/www; expires 30d; }

^~ means "if this is the longest prefix match, do not consider regexes at all". Use it on every prefix that must own its whole subtree. Two smaller traps: prefix matching is string matching, not path segment matching, so location /api also matches /apiary (write location /api/ plus location = /api); and a regex nested inside a prefix location is only reached if that prefix was selected, which is a safe way to scope one.

The proxy_set_header block, line by line#

nginx sends the upstream a deliberately minimal request. Everything a backend needs to know about the client has to be added explicitly.

proxy_set_header Host              $host;
proxy_set_header X-Real-IP         $remote_addr;
proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host  $http_host;

• Host $host replaces the default $proxy_host, the literal host from proxy_pass (for an upstream group, the group name). Without it, name-based vhosts miss, absolute URLs point at the group name and cookies get the wrong domain. $host is the Host header lowercased with the port stripped, falling back to server_name.
• X-Real-IP $remote_addr is the immediate peer address. An nginx convention rather than a standard, and the value most frameworks consume most easily.
• X-Forwarded-For $proxy_add_x_forwarded_for appends $remote_addr to whatever the client sent, which is why the header means nothing without a trust boundary; see configuring trusted proxies for which entry to believe.
• X-Forwarded-Proto $scheme stops a TLS-terminating proxy from producing http:// redirect loops in every framework that builds absolute URLs.
• X-Forwarded-Host $http_host preserves the original host including the port, which $host discards. Use $http_host here and $host for Host, deliberately.

nginx defaults that surprise people#

The consequential row is proxy_read_timeout. Because it measures the gap between reads rather than total duration, a backend dribbling one byte every 59 seconds never times out, while a backend that thinks for 61 seconds and then answers instantly produces a 504. Size the whole ladder with timeout budgets across a proxy chain and check it with the timeout ladder checker.

upstream blocks: balancing, failure counting and DNS#

upstream api {
    zone api 64k;              # shared memory: state visible to all workers
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=10s weight=2;
    server 10.0.1.13:8080 backup;
    server 10.0.1.14:8080 down;
    keepalive 64;
    keepalive_timeout 60s;
    keepalive_requests 1000;
}

weight (default 1) sets the relative share; backup takes traffic only when every non-backup server is unavailable; down removes a server while keeping it in the hash ring. max_fails (default 1) and fail_timeout (default 10s) drive passive health tracking, and fail_timeout does two jobs at once: it is both the window in which max_fails failures must occur and the period the server then sits out. max_fails=3 fail_timeout=10s means "three failures within ten seconds, then unavailable for ten seconds".

Load balancing methods, and when each is the right answer:

backup cannot be combined with hash, ip_hash or random. For affinity that survives pool changes, prefer hash ... consistent or a real cookie, as in sticky sessions.

Before 1.29.7, keep-alive needs three directives, not one. keepalive 64; sizes the idle cache per worker but does nothing alone on those builds, because HTTP/1.0 has no persistent connections and nginx sends Connection: close. Writing all three stays the portable choice on 1.29.7 and later:

proxy_http_version 1.1;
proxy_set_header   Connection "";
# plus keepalive N; inside the upstream block

An empty header value means "do not send this header", which lets the upstream default to persistent. Sizing, the idle-close race and ephemeral port arithmetic are in keep-alive and upstream connection pooling.

DNS. Names written literally in an upstream block are resolved once, at configuration load and pinned for the worker's lifetime; multiple A records become multiple servers. A name that does not resolve refuses the start or reload with host not found in upstream, and an address that changes later is never noticed. nginx 1.27.3 brought the resolve parameter of server to open source builds; it re-resolves on TTL expiry and needs a resolver in http plus a zone in the upstream block. Older builds have only a variable in proxy_pass, which trades the group's failure tracking for request-time resolution.

TLS on both sides#

Client-side TLS is listen 443 ssl, ssl_certificate, ssl_certificate_key and http2 on; (a separate directive since 1.25.1, replacing the http2 parameter on listen). Cipher policy, passthrough and SNI routing belong to TLS termination, passthrough and re-encryption.

The upstream side is a separate set of directives with weaker defaults, because nginx assumes that network is yours:

An HTTPS backend therefore needs at minimum proxy_pass https://... and proxy_ssl_server_name on;, plus proxy_ssl_verify on; with a CA file if the tunnel is to be authenticated rather than merely encrypted.

Logging that makes upstream problems visible#

The default combined format tells you nothing about the backend. This one does:

log_format upstream_detail
    '$remote_addr - $remote_user [$time_local] "$request" '
    '$status $body_bytes_sent "$http_referer" "$http_user_agent" '
    'rt=$request_time uct=$upstream_connect_time uht=$upstream_header_time '
    'urt=$upstream_response_time ua=$upstream_addr us=$upstream_status '
    'ucs=$upstream_cache_status';

• $request_time is the full client-visible duration. Compared with $upstream_response_time, a large gap means the time went on the client link or on buffering, not on the backend.
• $upstream_response_time is time to the last upstream byte; $upstream_header_time is time to the last header byte, so the difference separates a slow body from a slow decision.
• $upstream_connect_time separates "saturated at the TCP level" from "thinking".
• $upstream_addr names the server used. After a proxy_next_upstream retry it holds several addresses separated by commas, with $upstream_status holding the matching list, so a comma in these fields is the cheapest retry detector there is.
• $upstream_cache_status is MISS, BYPASS, EXPIRED, STALE, UPDATING, REVALIDATED or HIT, and - with no cache configured. It is the only proof that a caching proxy is caching.

Reload semantics, and why long connections survive#

nginx -t && nginx -s reload      # or: systemctl reload nginx, or kill -HUP <master pid>

nginx -t parses the configuration and reports the file and line of the first error, turning a failed deploy into a no-op. On reload the master reads the new configuration, starts a new generation of workers with it, and asks the old ones to shut down gracefully: they stop accepting immediately and keep serving the requests they already hold. Both generations are visible in ps, the old ones labelled nginx: worker process is shutting down.

That overlap has three consequences. Memory roughly doubles during the transition. Old workers keep running the old configuration, old TLS certificates and old upstream addresses included, until their last connection closes. And a WebSocket, an SSE stream or a long poll pins an old worker indefinitely, because worker_shutdown_timeout defaults to no limit at all. Where reloads are frequent, bound it:

worker_shutdown_timeout 30s;

Old workers then force their remaining connections closed after 30 seconds. That is a visible disconnect for streaming clients, which is why it is a policy decision rather than a default: pick a number your clients can reconnect from. A few changes still need a restart, notably user and listening-socket changes the master cannot inherit.

The complete server block#

http {
    log_format upstream_detail
        '$remote_addr - $remote_user [$time_local] "$request" '
        '$status $body_bytes_sent rt=$request_time '
        'uct=$upstream_connect_time uht=$upstream_header_time '
        'urt=$upstream_response_time ua=$upstream_addr '
        'us=$upstream_status ucs=$upstream_cache_status';

    worker_shutdown_timeout 30s;

    # An empty value removes the header entirely, which is what upstream
    # keep-alive needs. Mapping the empty case to "close" is the common
    # copy-paste bug: it disables the keepalive pool for every non-WebSocket
    # request while looking correct.
    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      "";
    }

    upstream api {
        zone       api 64k;
        least_conn;
        server     10.0.1.11:8080 max_fails=3 fail_timeout=10s;
        server     10.0.1.12:8080 max_fails=3 fail_timeout=10s;
        server     10.0.1.13:8080 backup;
        keepalive  64;
    }

    server {
        listen 80;
        listen [::]:80;
        server_name app.example.com;
        return 301 https://$host$request_uri;
    }

    server {
        listen      443 ssl;
        listen      [::]:443 ssl;
        http2       on;                       # separate directive since 1.25.1
        server_name app.example.com;

        ssl_certificate     /etc/ssl/app/fullchain.pem;
        ssl_certificate_key /etc/ssl/app/privkey.pem;

        access_log /var/log/nginx/app.access.log upstream_detail;
        error_log  /var/log/nginx/app.error.log warn;

        client_max_body_size 1m;              # raised per location below

        # Defined once here. No location below may add a proxy_set_header
        # without repeating this whole set: the directive replaces, not merges.
        proxy_http_version 1.1;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host  $http_host;
        proxy_set_header Connection        $connection_upgrade;

        proxy_connect_timeout 3s;
        proxy_send_timeout    30s;
        proxy_read_timeout    30s;
        proxy_buffer_size     16k;            # room for large Set-Cookie / JWT
        proxy_buffers         8 16k;
        proxy_next_upstream   error timeout;

        gzip          on;
        gzip_proxied  any;                    # nginx may sit behind a CDN
        gzip_types    application/json application/javascript text/css text/plain;

        # 1. Exact match: wins outright, search stops, never proxied.
        location = /healthz { access_log off; return 200 "ok\n"; }

        # 2. ^~ stops regex evaluation, so no asset regex can steal these
        #    paths and drop the header block above.
        location ^~ /static/ {
            root       /srv/www;
            expires    1y;
            access_log off;
        }

        # 3. Streaming endpoint: buffering off, long read gap allowed.
        #    No proxy_set_header here, so the server-level set is inherited.
        location = /api/events {
            proxy_pass         http://api;
            proxy_buffering    off;
            proxy_read_timeout 1h;
        }

        # 4. Uploads: only this path gets the larger body limit.
        location = /api/upload {
            proxy_pass           http://api;
            client_max_body_size 200m;
            proxy_request_buffering off;      # stream the body upstream
        }

        # 5. Everything else under /api. No URI part on proxy_pass, so the
        #    request path is forwarded unchanged, prefix included.
        location /api/ {
            proxy_pass http://api;
        }

        # 6. SPA shell, last because it is the shortest prefix.
        location / {
            root      /srv/www;
            try_files $uri /index.html;
        }
    }

    server {
        listen 80  default_server;
        listen 443 ssl default_server;
        ssl_reject_handshake on;
        server_name _;
        return 444;
    }
}

Observable behaviour: GET /api/users/7 reaches a backend as GET /api/users/7 HTTP/1.1 with Host: app.example.com over a pooled connection. GET /static/app.js is served from disk. GET /nope on an unknown Host gets no bytes back at all. POST /api/upload with a 50 MB body streams straight through instead of landing in /var/lib/nginx/body, at the cost of not being retryable on another server, as explained in proxy buffering and streaming responses. WebSocket upgrades work in any proxied location because $connection_upgrade is set at server level; see WebSockets through a reverse proxy.

Failure modes#

Mapping each status to the layer that produced it is the subject of 502 vs 503 vs 504, which is the faster path when the error log is not conclusive.

Frequently asked questions#

What is the correct proxy_set_header block for nginx?#

Set Host $host, X-Real-IP $remote_addr, X-Forwarded-For $proxy_add_x_forwarded_for, X-Forwarded-Proto $scheme and X-Forwarded-Host $http_host, plus proxy_http_version 1.1 and proxy_set_header Connection "" for upstream keep-alive. Define them at one level only: a proxy_set_header in a nested block replaces the inherited set instead of adding to it.

Why is nginx matching the wrong location block?#

Because a regular expression location beats a longer prefix location. nginx checks exact = matches first, remembers the longest prefix, then tries every regex in file order and takes the first match, falling back to the remembered prefix only if none matched. Add ^~ to the prefix location to stop regex evaluation for that subtree.

Does nginx reuse connections to upstream servers by default?#

It depends on the version. From nginx 1.29.7 upstream connections are cached by default (keepalive 32 local) and the default proxy protocol is HTTP/1.1. On any earlier build nginx opens a new TCP connection per upstream request unless you add keepalive N; to the upstream block, set proxy_http_version 1.1; and clear the Connection header with proxy_set_header Connection "";, and all three are required. Write all three regardless, so the configuration behaves the same on both.

What is the default proxy timeout in nginx?#

proxy_connect_timeout, proxy_send_timeout and proxy_read_timeout all default to 60 seconds, and proxy_connect_timeout cannot exceed 75 seconds. The send and read timeouts measure the gap between two successive operations, not the total request duration, so a response that trickles indefinitely will never trigger them.

How do I fix 413 Request Entity Too Large in nginx?#

Raise client_max_body_size above the largest legitimate body; the default is 1m. Set it in the upload location rather than globally, and confirm the error log line client intended to send too large body came from this nginx and not a second proxy further along the chain.

How do I serve static files and proxy an API from the same server block?#

Put the API on its own prefix location with proxy_pass, the assets on a ^~ prefix location with root, and let location / serve the SPA shell with try_files $uri /index.html. The ^~ matters: without it any location ~* \.(js|css)$ elsewhere in the file captures asset requests and bypasses the settings you meant to apply.

What is the difference between nginx reload and restart?#

A reload (nginx -s reload or SIGHUP) keeps the master process, starts new workers with the new configuration and lets old workers finish their existing connections, so no request is dropped. A restart kills everything and rebinds the sockets, dropping connections, but is required for changes the master cannot apply in place such as user.

Why does an nginx reload not pick up my new TLS certificate?#

It does, for new connections, but old workers keep serving their existing connections with the old certificate until they exit. With worker_shutdown_timeout at its default of no limit, one WebSocket or SSE client can keep an old worker, and the old certificate, alive indefinitely. Set worker_shutdown_timeout 30s; to bound it.