Why does nginx return 502 after I restart a Docker container?

nginx returns 502 after a backend container is recreated because it looked up the container's name once, when it loaded its configuration, and it is still connecting to the old address. nginx never asks DNS again for a name written literally in proxy_pass or in an upstream block. When the new container comes up on a different address, every request fails with connect() failed (111: Connection refused) or connect() failed (113: Host is unreachable) until nginx is reloaded. The fix is to make nginx re-resolve the name at run time. On nginx 1.27.3 and later, do that with server api:8000 resolve; in an upstream block that has a zone, plus a resolver pointed at Docker's DNS on 127.0.0.11 with valid=10s.

The popular alternative, a variable in proxy_pass, also works and is the only option on older open source builds. It costs more than most answers admit, and the costs are covered below.

Pick the fix by nginx version#

Run nginx -v inside the proxy container first. The official image tag is not the version: in testing, nginx:1.27-alpine was 1.27.5 and nginx:1.26-alpine was 1.26.3.

nginx 1.27.3 (26 November 2024) added both halves: the resolve parameter on server in upstream, and resolver and resolver_timeout inside the upstream block. The 1.28.0 stable branch followed on 23 April 2025 with both.

Confirm the diagnosis in two commands#

Before changing configuration, prove that nginx is connecting to an address the backend no longer holds. Compare the address in the error log with the address Docker currently gives the container:

# The address nginx is using: the upstream field of the error line
docker compose logs nginx | grep 'connect() failed' | tail -1

# The address the backend has now
docker inspect -f '{{range $n, $c := .NetworkSettings.Networks}}{{$n}} {{$c.IPAddress}}{{"\n"}}{{end}}' myproject-api-1

If the upstream: address in the log is not one the container has on any network, this page is your problem. If they match, the container is reachable at the right address and is refusing or dropping connections itself, so go back to reading 502, 503 and 504 by cause.

The two error strings mean different things, and both appeared in a test run on Docker Engine 28.1.1 with nginx 1.27.5 holding a literal upstream:

connect() failed (111: Connection refused) while connecting to upstream, ...
    upstream: "http://192.168.160.2:80/"
connect() failed (113: Host is unreachable) while connecting to upstream, ...
    upstream: "http://192.168.160.2:80/"

111 came back immediately, while another container held the old address. 113 came back after a few seconds, once nothing held the old address. The Alpine image's C library words errno 113 as Host is unreachable; glibc words the same errno as No route to host, so search your logs for both. A third variant is worse: when a second nginx container took the old address in the same test, the proxy returned 200s from the wrong service and logged nothing.

Does a restarted container get a new IP?#

Docker does not promise either answer, so do not design around one. The Compose networking documentation says Docker "assigns container IP addresses dynamically from the network's subnet each time a container starts so they are not persisted across restarts or recreations", and that a container updated by docker compose up "joins the network under a different IP address but the same name". It also says plainly whose job it is to cope: "It is each container's responsibility to detect this condition, look up the name again, and reconnect." nginx with a literal upstream does not.

What actually happened on one host, Docker Engine 28.1.1, one user-defined bridge network:

This is why the fault looks random: most restarts reuse the address, then a deploy or a one-off container takes it and an identical-looking restart breaks nginx. Use docker inspect when debugging, and make nginx re-resolve so the answer stops mattering.

The 1.27.3+ fix: resolve in the upstream block#

upstream api {
    zone api 64k;                          # required: group state in shared memory
    resolver 127.0.0.11 valid=10s;         # upstream context needs 1.27.3+
    server api:8000 resolve;
    keepalive 16;
}

server {
    listen 80;
    location / {
        proxy_pass http://api;
        proxy_http_version 1.1;            # default from 1.29.7, harmless before
        proxy_set_header Connection "";
    }
}

The documentation describes resolve as monitoring "changes of the IP addresses that correspond to a domain name of the server" and modifying the upstream configuration "without the need of restarting nginx". It sets two preconditions, and nginx refuses to start if either is missing. These are the exact messages from 1.27.5:

Placing resolver inside the upstream block keeps Docker's DNS address next to the name it resolves; putting it in http works too.

In the test, a resolve upstream with valid=10s still returned 502 one second after the backend came back on a new address, and 200 twelve seconds later: the old answer stays cached until valid= expires, so valid= sets the worst-case outage after a recreate.

One more property matters in Compose. With a literal name, nginx will not start at all if the backend's name does not resolve: host not found in upstream "api:8000", and the proxy container exits. With resolve, nginx starts, logs api could not be resolved (3: Host not found), and returns 502 (no live upstreams while connecting to upstream) until the backend appears.

The circulating fix: a variable in proxy_pass#

resolver 127.0.0.11 valid=10s;

server {
    listen 80;
    location / {
        set $api_backend http://api:8000;
        proxy_pass $api_backend;
    }
}

This is the commonly circulated answer, and it works: with a variable, nginx resolves the name per request (subject to the valid= cache) instead of once at load. It is the right choice on open source builds older than 1.27.3. On 1.27.3 and later it is the worse of the two, for four reasons the usual answers leave out:

1. There is no upstream block, so there is nowhere to put keepalive. The keepalive directive is valid only in upstream context. You cannot size or tune an idle connection pool for a name resolved this way. See keep-alive and upstream connection pooling for what the pool buys you.
2. No max_fails, fail_timeout, backup, weight or balancing method. Those are parameters of server inside upstream. Passive failure tracking, covered in health checks and upstream failover, stops applying.
3. URI handling changes. The proxy module documentation states that when variables are used and a URI is specified in the directive, "it is passed to the server as is, replacing the original request URI". location /api/ { proxy_pass http://$api_backend/; } therefore sends every request upstream as /, not as the stripped path. Leave the URI out, as above, or build it yourself with $request_uri. The full rules, including what happens to proxy_redirect default, are in proxy_pass with variables, and the proxy_pass simulator shows the URI nginx will send.
4. The variable must not match an upstream block's name. nginx searches a variable's host "among the described server groups" first and only uses the resolver if none matches. set $b api; with an upstream api {} defined elsewhere in the configuration silently uses the group's addresses from load time, and the stale address comes back.

Why 127.0.0.11, and why it fails on the default bridge#

127.0.0.11 is Docker's embedded DNS server, and it is only there on some networks. The Docker networking documentation says containers on the default bridge network "receive a copy of" the host's /etc/resolv.conf, while containers on a custom network "use Docker's embedded DNS server", whose "address is 127.0.0.11". The same page says containers on the default bridge "cannot refer to each other by name" at all. On the default bridge there is no container name to resolve and nothing listening on 127.0.0.11.

The difference is visible in /etc/resolv.conf. On the test host, a container on the default bridge showed the host's upstream servers (nameserver 8.8.8.8), and a container on a user-defined network showed nameserver 127.0.0.11. Pointing the variable configuration at 127.0.0.11 from the default bridge produced:

send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
api could not be resolved (110: Operation timed out)

and a 502 for every request. Compose is not affected by default: docker compose up creates a network named <project-name>_default and attaches every service to it, and that is a user-defined network. The problem appears with plain docker run containers that were started without --network.

There is no IPv6 equivalent; Docker says the IPv4 address "works even in IPv6-only containers". If you template the configuration, the official nginx image can fill the address in: with NGINX_ENTRYPOINT_LOCAL_RESOLVERS set, its 15-local-resolvers.envsh entrypoint exports NGINX_LOCAL_RESOLVERS from the nameserver lines of /etc/resolv.conf, for use as resolver ${NGINX_LOCAL_RESOLVERS} valid=10s; in a template.

Compose-level alternatives, and why depends_on is not one#

When the nginx configuration cannot change (a vendor image, an old build), these are the options at the Compose layer:

The reload hook is the most dependable. docker compose exec nginx nginx -s reload makes the master re-read its configuration, which resolves every literal name again, and start new workers while old ones "continue to service old clients". There is one trap. The master "first checks the syntax validity" and if applying the new configuration fails "it rolls back changes and continues to work with old configuration". If the backend is down when you reload, the literal name does not resolve, the reload is rejected, and nginx carries on with the stale address. Reload after the backend is up: docker compose up -d api, then the reload.

Plain depends_on does not help, and it is the most common wrong answer. The Compose documentation says it controls "the order of service startup and shutdown", and that Compose "does not wait until a container is 'ready', only until it's running". Nothing about it acts after startup, so a backend recreated an hour later is invisible to it.

restart: true is the variant that does act after startup. The documentation says it restarts the dependent service if the dependency "is updated or restarted due to an explicit Compose operation, for example docker compose restart". Tested on Compose 2.35.1: docker compose restart api restarted nginx, and docker compose up -d after changing an environment variable on api restarted nginx. docker restart on the backend container did not, and neither did docker compose up -d --force-recreate api with no configuration change. It is also a full restart of the proxy container, not a reload. Treat it as a backstop, not the fix.

Failure modes after the fix#

Still 502 for ten minutes after a recreate. valid= is missing, so nginx honours Docker's TTL. Add valid=10s to the resolver line, including one inside the upstream block.

no live upstreams for a few seconds after the backend restarts. When the group holds more than one address (several server lines, or a name resolving to several containers), failed attempts during the restart mark each address unavailable for fail_timeout (default 10s, after max_fails, default 1). This is passive failure tracking working as designed. A group with a single server is never marked unavailable, so this does not happen with one server line resolving to one container.

Fixed on most paths, still 502 on one. Another proxy_pass or upstream still names the backend literally. Every literal occurrence is resolved at load and pinned, so one missed location keeps the fault for that path.

Variable form returns the backend's root page for every path. A URI was written after the variable (proxy_pass http://$b/;), which replaces the whole request URI. Remove it, or pass $request_uri explicitly.

For anything that does not match these, bisect the chain from the proxy host outwards as in the proxy debugging playbook, and confirm the general nginx upstream settings against nginx as a reverse proxy. Proxies that build their routes from Docker itself rather than from DNS, such as Traefik with its Docker provider, apply container changes as live dynamic configuration and so never pin an address at load.

Frequently asked questions#

Why does nginx return 502 after docker compose up recreates a container?#

Because nginx resolved the service name to an address once, at configuration load, and the recreated container may have a different address. nginx keeps connecting to the old one and logs connect() failed (111: Connection refused) or (113: Host is unreachable). Reloading nginx clears it once; resolve on nginx 1.27.3+ or a variable proxy_pass with a resolver stops it recurring.

Does nginx re-resolve DNS for upstream servers?#

Not for names written literally without the resolve parameter: those are resolved at startup and reload only. From 1.27.3 in open source builds, adding the resolve parameter to a server line in an upstream block with a zone makes nginx re-resolve on expiry of the cached answer, using the configured resolver.

What resolver address should nginx use inside Docker?#

127.0.0.11, Docker's embedded DNS server, on any user-defined network, including Compose project networks. The default bridge has no embedded DNS server, so it does not work there.

Does depends_on fix nginx 502 after a container restart?#

No. depends_on only orders startup. The restart: true option restarts nginx when the dependency is restarted or updated by a Compose command, but not after docker restart or a restart-policy restart, and it restarts the whole proxy instead of reloading it.

Is the resolve parameter available in open source nginx?#

Yes, since 1.27.3. Before that release it was available only as part of the commercial subscription (nginx Plus). The 1.28.x stable branch includes it; the 1.26.x branch does not, and rejects it with invalid parameter "resolve".