Envoy listeners, routes and clusters

Envoy has one data path: a listener binds a socket, a filter chain is selected for the connection, network filters run, the HTTP connection manager (HCM) parses HTTP, HTTP filters run in order ending with the router, the route table picks a route, the route names a cluster, and the cluster load balances across endpoints. Everything else (retries, timeouts, circuit breakers, outlier ejection) hangs off one of those objects, and each can be delivered statically in the bootstrap file or dynamically over xDS. Naming the stage a request died in is most of the debugging, and the access log response flag names it for you.

The object model in one table#

If you come from nginx: the HCM is roughly server {}, the route configuration is the set of location {} blocks, and the cluster is upstream {}. The difference that matters is that the HCM is a network filter, so TLS termination, TCP proxying and HTTP proxying are just different filter stacks on the same listener.

What xDS is, and when static config is the right answer#

xDS is the family of discovery APIs Envoy uses to fetch configuration at runtime over streaming gRPC, REST polling, or a watched file. Each API supplies one layer of the object model.

ADS exists because of an ordering hazard: on separate streams Envoy can learn about a route pointing at a cluster it has not received yet, or drop endpoints for a cluster still in use. ADS serialises updates so the safe sequence (CDS, EDS, LDS, RDS on the way up, reverse on the way down) is possible. Use ADS unless you have a reason not to.

Static configuration is not a toy. A single static bootstrap is correct when the upstream set is fixed or discovered by DNS, because STRICT_DNS clusters already give you runtime endpoint changes with no control plane. Reach for xDS when endpoints, routes or certificates change faster than you want to redeploy a file.

Route matching, and where it differs from nginx#

Route selection is two steps. First Envoy picks a virtual host by matching the request :authority against each virtual host's domains list: exact match wins, then the longest suffix wildcard (*.example.com), then prefix wildcard, then the catch-all *. Then it walks that virtual host's routes array in order and takes the first entry whose match succeeds.

prefix_rewrite replaces only the portion the prefix matched, so prefix: "/api/" with prefix_rewrite: "/" turns /api/users/7 into /users/7. regex_rewrite applies a RE2 pattern to the whole path with a substitution using \1 style references, for rewrites that are not a leading-segment swap. Either way Envoy records the pre-rewrite path in x-envoy-original-path, which is why the default log format prints %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%.

Cluster types, and STRICT_DNS versus LOGICAL_DNS#

The practical difference: STRICT_DNS holds a connection pool per resolved address, so a record with 200 A entries gives 200 pools per Envoy. LOGICAL_DNS holds one logical host, keeping memory and connection count flat but giving up per-endpoint load balancing and outlier ejection.

Circuit breakers, outlier detection and retries#

These three are different mechanisms and are commonly confused.

• Circuit breakers are cluster-wide concurrency limits checked before dispatch: max_connections, max_pending_requests and max_requests default to 1024, max_retries to 3. Tripping one gives 503 with flag UO.
• Outlier detection ejects individual endpoints on observed failures. Defaults: consecutive_5xx 5, interval 10s, base_ejection_time 30s, max_ejection_percent 10. Ejection time grows with repeat ejections.
• Retries are per route: retry_on (5xx, gateway-error, reset, connect-failure, retriable-4xx, refused-stream, retriable-status-codes, envoy-ratelimited), num_retries, per_try_timeout, and exponential backoff with a 25ms base interval.

max_retries defaulting to 3 is a cluster-wide concurrency budget, not per request. Under a partial upstream failure the whole cluster is limited to three retries in flight and the rest fail immediately with URX. That is deliberate (it stops a retry storm finishing off a struggling backend), but it surprises people who set num_retries: 3 on a route and see almost no retries happen.

The timeout hierarchy#

Streaming responses (server-sent events, long gRPC streams) need the route timeout set to 0s and stream_idle_timeout raised on that route specifically. Disabling it globally on the HCM removes your only protection against stuck streams.

Response flags: the fastest Envoy debugging artefact#

The default access log format includes %RESPONSE_FLAGS%. Each flag names the stage that ended the request.

Add %RESPONSE_CODE_DETAILS% to the log format. It turns UF into something like upstream_reset_before_response_started{connection_failure}, and distinguishes via_upstream (the upstream really did return that 503) from an Envoy-generated local reply. Flag plus code details settles most "is it Envoy or the backend" arguments in one line, the same question 502 vs 503 vs 504 covers at the HTTP level.

Worked example: a complete static bootstrap#

This is a runnable v3 bootstrap that terminates plain HTTP on 10000, strips the /api prefix, and proxies to a DNS-resolved cluster with retries, circuit breakers and outlier detection.

admin:
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:
  listeners:
  - name: listener_http
    address:
      socket_address: { address: 0.0.0.0, port_value: 10000 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          codec_type: AUTO
          use_remote_address: true
          request_timeout: 10s
          stream_idle_timeout: 300s
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
              path: /dev/stdout
              log_format:
                text_format_source:
                  inline_string: "%RESPONSE_CODE% %RESPONSE_FLAGS% %RESPONSE_CODE_DETAILS% %DURATION%ms %UPSTREAM_HOST% \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%\"\n"
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains: ["*"]
              routes:
              - match: { path_separated_prefix: "/api" }
                route:
                  cluster: service_backend
                  prefix_rewrite: "/"
                  timeout: 3s
                  retry_policy:
                    retry_on: "connect-failure,reset,5xx"
                    num_retries: 2
                    per_try_timeout: 900ms
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  clusters:
  - name: service_backend
    type: STRICT_DNS
    connect_timeout: 1s
    lb_policy: ROUND_ROBIN
    dns_lookup_family: V4_ONLY
    circuit_breakers:
      thresholds:
      - priority: DEFAULT
        max_connections: 512
        max_pending_requests: 256
        max_requests: 512
        max_retries: 16
    outlier_detection:
      consecutive_5xx: 5
      interval: 10s
      base_ejection_time: 30s
      max_ejection_percent: 20
    load_assignment:
      cluster_name: service_backend
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: backend.internal, port_value: 8080 }

Observable behaviour: GET /api/users/7 reaches the backend as GET /users/7, and GET /apiary returns 404 with NR because path_separated_prefix will not match a partial segment. If backend.internal refuses connections you get 503 with UF after 1s. If it accepts and hangs you get 503 with URX at roughly 1.8s (two attempts of 900ms), not 504 with UT, because the per-try timeout fires first. URX versus UT is how you tell "retries exhausted" from "total budget spent".

Failure modes#

• 404 with NR on a request you are sure is configured. The virtual host did not match. Envoy matches :authority including the port when the client sends one, so domains: ["example.com"] does not match Host: example.com:8443. List both, or use *.
• 503 with UH right after a deploy. The cluster exists but has no healthy members. Check the admin /clusters endpoint for health flags and ejection counters before blaming DNS. Outlier detection ejecting most of a two-host cluster is a classic, and max_ejection_percent is the guard. See health checks and upstream failover.
• 502 with UPE on gRPC traffic. The cluster is speaking HTTP/1.1 to an h2c-only backend. Set explicit HTTP/2 upstream options on the cluster; details in gRPC through a reverse proxy.
• Config accepted, traffic still wrong. The admin /config_dump endpoint shows the merged effective config, including anything xDS overwrote. Diff that, not the file you edited.
• DC with no status code. The client or a load balancer in front disconnected first. If it lands on a round number of seconds, something upstream of Envoy has a shorter idle timeout than Envoy does.

Why pick Envoy over nginx#

Three reasons hold up, and the rest is taste. Config changes without a reload: xDS updates listeners, routes, clusters, endpoints and certificates in a running process, where nginx forks a new worker generation and long-lived connections keep the old one alive. Observability is an output, not an add-on: per-cluster and per-route counters, histograms, response flags and tracing propagation ship in the binary. HTTP/2, HTTP/3 and gRPC are native on both sides, including h2c upstream, gRPC health checks and JSON transcoding, with no third-party modules.

The cost is verbose config, a control plane for anything dynamic, and a larger footprint per instance. The reverse proxy comparison sets that against HAProxy, Caddy and Traefik.

Frequently asked questions#

What is the difference between STRICT_DNS and LOGICAL_DNS in Envoy?#

STRICT_DNS makes every address returned by DNS a separate host with its own connection pool, health state and load balancing weight. LOGICAL_DNS keeps a single logical host and only uses the first resolved address for new connections, which keeps connection counts flat for very large or rapidly changing DNS records but gives up per-endpoint load balancing and ejection.

What does response flag UH mean in Envoy access logs?#

UH means "no healthy upstream": the route matched and the cluster exists, but every endpoint in it is unhealthy, ejected by outlier detection, or absent. Envoy returns 503. Check the admin /clusters endpoint for member counts, health flags and outlier_detection ejection counters.

What is Envoy's default route timeout?#

The route timeout field defaults to 15 seconds. It covers the whole request from the moment the complete request is received to the moment the response is fully processed, and it is not reset between retry attempts, so per_try_timeout times the number of attempts has to fit inside it.

Why is my Envoy route not matching?#

The two usual causes are virtual host selection (the :authority header, including any port, did not match the domains list) and route ordering (an earlier, broader prefix route matched first, because Envoy takes the first match in array order rather than the longest prefix). Confirm with /config_dump and look for NR in the access log.

How do I enable WebSockets in Envoy?#

WebSocket upgrades are not enabled by default. Add an upgrade_configs entry with upgrade_type: websocket on the HTTP connection manager, or per route, and set the route timeout to 0s so a long-lived connection is not cut at 15 seconds.

What is the difference between circuit breakers and outlier detection?#

Circuit breakers are cluster-wide concurrency limits applied before dispatch (too many in-flight requests, pending requests, connections or retries) and produce flag UO. Outlier detection removes individual misbehaving endpoints from the load balancing set based on observed errors, showing up as shrinking healthy member counts and eventually UH.