Timeout ladder checker
Enter the timeouts and retry counts for every hop in your chain and see which one gives up first, what the user sees, and where the ladder is inverted.
Check your timeout ladder
List the hops from the client inward. Each hop needs a per-attempt timeout and the number of attempts it makes (1 means no retry). The checker works out which hop gives up first, what the user sees, and where the ladder is inverted.
The rule the checker enforces#
Timeouts must get shorter as you move inward, and every hop must allow enough time for the retries of the hop inside it. The layer closest to the work should be the first to give up, because it is the only layer that knows what the work was and can say something useful about why it failed.
When the ladder is inverted, the outer hop fires first. The user gets a 504 from the load balancer, the application logs nothing at all because from its point of view the request is still in progress, and the investigation starts at the wrong end of the system. That specific shape, a gateway timeout with no corresponding upstream log line, is the signature of an inverted ladder and almost nothing else.
Defaults worth knowing before you set anything#
| Layer | Setting | Default |
|---|---|---|
| nginx | proxy_connect_timeout | 60s |
| nginx | proxy_read_timeout | 60s |
| nginx | proxy_send_timeout | 60s |
| nginx | keepalive_timeout (client) | 75s |
| HAProxy | timeout connect / client / server | none, and HAProxy warns at startup |
| Envoy | route timeout | 15s |
| Envoy | stream_idle_timeout | 5m |
| AWS ALB | connection idle timeout | 60s |
Two of these cause most of the surprises. Envoy's 15 second route timeout is far shorter than everything around it, so an Envoy hop dropped into a chain designed around 60 second timeouts starts failing long requests immediately. And HAProxy having no defaults means an unconfigured timeout server leaves connections open indefinitely, which looks fine until a slow dependency fills the connection table.
Retries multiply, they do not add#
A retry does not cost you one extra attempt, it multiplies the branch below it. Three hops each configured to try twice is eight units of work at the innermost layer for one client request. That is how a dependency that got slow becomes a dependency that is down: the retry traffic alone exceeds its capacity.
Retries are also only safe for idempotent requests. A proxy that retries a POST because the upstream closed the connection before responding may well have caused a duplicate order. nginx's proxy_next_upstream includes non_idempotent as an explicit opt in for exactly this reason.
What happens after the timeout fires#
The hop that timed out stops waiting. The hop inside it usually does not stop working. The database query keeps running, the worker stays busy, the connection stays held. Under sustained load that gap is where capacity disappears: every request is failing fast at the edge while the backend is still executing all of them. Cancellation has to be propagated explicitly, through request context in Go, cancellation tokens, or a statement timeout at the database, or your timeouts protect the client experience and nothing else.
The full treatment, including which status code each proxy returns for each timeout and how to read the resulting logs, is in timeout budgets across a proxy chain and 502 vs 503 vs 504.
Frequently asked questions#
Should the client timeout be longer or shorter than the server timeout?#
Longer. The client sits outermost, so it should be the last to give up. If the client gives up first, the server keeps working on a response nobody will read, and your error rate looks fine on the server side while users see failures.
Why do I get a 504 with nothing in the upstream logs?#
Because the hop that timed out is not the hop doing the work. The proxy stopped waiting and returned 504 while the upstream was still processing, so the upstream never logged a completed request. Compare the proxy timeout against the real upstream latency distribution rather than the median.
What is a sensible margin between hops?#
Enough to cover connection setup, TLS handshake and queueing at the inner hop, and comfortably more than the jitter you actually observe. Ten to twenty percent is a reasonable starting point, measured against the inner hop's worst case including its retries, not its median.
Does a keep-alive timeout belong in the ladder?#
It is a different axis but it interacts. If the upstream closes idle connections sooner than the proxy expects, the proxy will occasionally write a request onto a connection the upstream is closing and produce an intermittent 502. Keep the upstream idle timeout longer than the proxy's, as covered in keep-alive and upstream connection pooling.
Is a shorter timeout always safer?#
No. A timeout shorter than the real work turns successful slow requests into failures and, if retries are on, into extra load. Set timeouts from the observed high percentile latency of the operation plus headroom, not from a round number.
Primary sources#
Every normative claim on this page is checked against the specification or the vendor documentation listed here. Where behaviour is version dependent, the version is named in the text.