TLS & security

Mutual TLS through a proxy

Verify client certificates at nginx, HAProxy, Envoy, Traefik and Caddy, forward the identity safely, and avoid the XFCC spoofing hole and TLS 1.3 traps.

· 16 min read · How we verify this

Key points

  • If the proxy terminates TLS the client certificate never reaches the application, so the proxy must verify it and forward the identity in a header.
  • That header is now a spoofing target: every ingress must strip the inbound copy unconditionally, using proxy_set_header or Envoy's default SANITIZE mode, not a conditional rule.
  • TLS 1.3 removed renegotiation, so per-path client certificate requirements that worked on TLS 1.2 now need post-handshake auth, which most browsers do not implement.
  • 400 No required SSL certificate was sent means the handshake succeeded and nginx rejected at HTTP level; a silent handshake failure means no client certificate matched the advertised CA list.

Mutual TLS breaks in a specific way when a proxy is in the path: the client certificate is consumed by whichever hop completes the TLS handshake. If the proxy terminates, the application behind it sees an ordinary unauthenticated connection from the proxy, no matter how carefully the client presented its certificate. The proxy must therefore verify the certificate itself and forward the resulting identity to the backend, normally in an HTTP header. That header is now the entire basis of the application's authentication decision, which makes stripping any inbound copy of it a hard requirement rather than a hardening tip.

Only three arrangements are actually possible:

  1. Passthrough. The proxy does not decrypt, the backend performs the handshake and sees the real certificate. mTLS works unchanged, at the cost of everything described in TLS termination, passthrough and re-encryption: no L7 routing, no header injection, PROXY protocol for the client IP.
  2. Terminate and forward identity. The proxy verifies the chain and injects a header the backend trusts. This is the common design and the one with the spoofing hazard.
  3. Terminate and re-originate. The proxy verifies the client, then presents its own client certificate to the backend on a second TLS session. The backend authenticates the proxy, not the end client, and still needs the forwarded header to know who the end client was.

A fourth case is worth naming because it is diagnosed as a server fault: an intercepting proxy on the client side, which cannot present the client's certificate upstream and so kills the handshake before it reaches you. That is covered in TLS interception and corporate root CAs.

How each proxy requires and forwards a client certificate#

ProxyRequire a client certificateIdentity exposed asDefault header injectedInbound header sanitised by default
nginxssl_verify_client on; plus ssl_client_certificate ca.pem;$ssl_client_verify, $ssl_client_s_dn, $ssl_client_i_dn, $ssl_client_serial, $ssl_client_fingerprint, $ssl_client_escaped_certNone, you write proxy_set_header yourselfNo, you must overwrite every header explicitly
HAProxybind :443 ssl crt srv.pem ca-file ca.pem verify requiredssl_c_verify, ssl_c_s_dn, ssl_c_i_dn, ssl_c_sha1, ssl_c_serial, ssl_c_notafter, ssl_c_used, ssl_c_derNone, you write http-request set-header yourselfNo, but set-header replaces rather than appends
Envoyrequire_client_certificate: true in DownstreamTlsContext plus a validation_contextx-forwarded-client-cert keys By, Hash, Cert, Chain, Subject, URI, DNSx-forwarded-client-certYes, forward_client_cert_details defaults to SANITIZE
TraefikTLS option clientAuth.clientAuthType: RequireAndVerifyClientCert plus caFilesPassTLSClientCert middleware fields (subject, issuer, serial, notAfter, sans)X-Forwarded-Tls-Client-Cert, X-Forwarded-Tls-Client-Cert-InfoThe middleware overwrites the headers it sets
Caddytls { client_auth { mode require_and_verify ... } }placeholders such as {http.request.tls.client.subject}, {http.request.tls.client.issuer}, {http.request.tls.client.serial}, {http.request.tls.client.certificate_der_base64}None, you set headers in reverse_proxyNo, you must set the headers explicitly

Two structural points fall out of that table. Envoy is the only one of the five that sanitises the identity header by default, because it treats XFCC as a protocol element with defined semantics rather than as an arbitrary header. Everywhere else, an unset header is simply passed through from the client, which is the default that produces the vulnerability.

nginx: verification and the variables worth forwarding#

nginx
server {
    listen 443 ssl;
    server_name api.example.com;

    ssl_certificate     /etc/ssl/server.pem;
    ssl_certificate_key /etc/ssl/server.key;

    # CAs whose DNs are advertised to the client in CertificateRequest
    ssl_client_certificate /etc/ssl/client-ca.pem;
    ssl_verify_client on;
    ssl_verify_depth  2;                 # default is 1
    ssl_crl           /etc/ssl/client-ca.crl;
    ssl_ocsp          on;                # nginx 1.19.0 and later
    ssl_ocsp_responder http://ocsp.internal.example.com/;

    location / {
        proxy_pass http://app_backend;

        # Overwrite, never append. These run for every request.
        proxy_set_header X-Client-Verify      $ssl_client_verify;
        proxy_set_header X-Client-DN          $ssl_client_s_dn;
        proxy_set_header X-Client-Issuer-DN   $ssl_client_i_dn;
        proxy_set_header X-Client-Serial      $ssl_client_serial;
        proxy_set_header X-Client-Fingerprint $ssl_client_fingerprint;
        proxy_set_header X-Client-Cert        $ssl_client_escaped_cert;
        proxy_set_header X-Client-Expires     $ssl_client_v_end;
    }
}

Details that matter:

  • ssl_client_certificate versus ssl_trusted_certificate. Both supply trust anchors, but only the DNs from ssl_client_certificate are sent to the client in the CertificateRequest message. Browsers use that list to decide which certificates to offer. Put the issuing CA in ssl_client_certificate so clients can find their certificate; use ssl_trusted_certificate for anchors you want to trust without advertising, which keeps the CertificateRequest small when you trust many CAs.
  • ssl_verify_depth defaults to 1. nginx documents it only as "the verification depth in the client certificates chain", and how the number maps to chain length is easy to get wrong in either direction, so treat it empirically: a three-level PKI (root, issuing CA, client) is widely reported to need ssl_verify_depth 2, and a value that is too low produces client SSL certificate verify error: (22:certificate chain too long). Raise it to match the real chain and confirm with a test handshake rather than reasoning about the number.
  • Use $ssl_client_escaped_cert, not $ssl_client_cert. The latter emits multi-line PEM with tab continuations, which is a legal but hostile header value that many upstream parsers reject or truncate. $ssl_client_escaped_cert (nginx 1.13.5 and later) is a single-line percent-encoded PEM.
  • $ssl_client_s_dn is RFC 2253 format since nginx 1.11.6, with the RDN order reversed relative to the old slash-separated form. $ssl_client_s_dn_legacy preserves the old output. Application code that pattern-matches on /CN= will silently stop matching after an upgrade that predates that change.
  • $ssl_client_v_remain (nginx 1.11.7 and later) gives days until the client certificate expires. Logging it turns client certificate expiry from an outage into a dashboard.

nginx also drops request headers containing underscores by default (underscores_in_headers off), which accidentally protects X_Client_DN but not X-Client-DN. Do not rely on that as a control.

optional and optional_no_ca#

ssl_verify_client has four values, and the difference between the last two is routinely misunderstood.

ValueCertificateRequest sentChain validatedConnection continues without a certificate
offNon/aYes
onYesYesNo, request rejected with 400
optionalYesYes, if presentedYes, $ssl_client_verify is NONE
optional_no_caYesNoYes

optional_no_ca asks for a certificate, accepts whatever arrives, and does not validate it against any CA. It sounds like a footgun and mostly is, but it has legitimate uses: when validation is delegated to the application or an external authorisation service that has richer policy than a CA bundle, when identities are SPIFFE SVIDs validated by a sidecar, or when you want to record the presented certificate for audit without gating on it. The rule is that with optional_no_ca, $ssl_client_verify tells you nothing useful about trust, so the backend must validate $ssl_client_escaped_cert itself. If the backend just checks that the header is non-empty, you have built certificate authentication that accepts a self-signed certificate anyone can generate.

HAProxy#

haproxy
frontend api
    bind :443 ssl crt /etc/ssl/server.pem \
         ca-file /etc/ssl/client-ca.pem verify required \
         crl-file /etc/ssl/client-ca.crl \
         alpn h2,http/1.1
    mode http

    # ssl_c_verify is 0 on success; anything else is an OpenSSL verify code
    http-request deny deny_status 403 unless { ssl_c_used 1 } { ssl_c_verify 0 }

    http-request set-header X-Client-DN        %{+Q}[ssl_c_s_dn]
    http-request set-header X-Client-Issuer-DN %{+Q}[ssl_c_i_dn]
    http-request set-header X-Client-Serial    %[ssl_c_serial,hex]
    http-request set-header X-Client-SHA1      %[ssl_c_sha1,hex]
    http-request set-header X-Client-NotAfter  %[ssl_c_notafter]

    default_backend app

verify optional plus an explicit ssl_c_used test is the pattern when some routes need a certificate and others do not, because HAProxy cannot re-request a certificate mid-connection any more than nginx can. %{+Q} quotes the value, which matters for DNs containing commas and spaces. HAProxy's ACL and header syntax is covered further in HAProxy configuration for HTTP reverse proxying.

Envoy and the XFCC header#

yaml
transport_socket:
  name: envoy.transport_sockets.tls
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
    require_client_certificate: true
    common_tls_context:
      tls_certificates:
      - certificate_chain: { filename: /etc/ssl/server.pem }
        private_key:       { filename: /etc/ssl/server.key }
      validation_context:
        trusted_ca: { filename: /etc/ssl/client-ca.pem }
        crl:        { filename: /etc/ssl/client-ca.crl }
        match_typed_subject_alt_names:
        - san_type: URI
          matcher: { prefix: "spiffe://example.com/ns/prod/" }

and on the HTTP connection manager:

yaml
forward_client_cert_details: SANITIZE_SET
set_current_client_cert_details:
  subject: true
  uri: true
  dns: true
  cert: false
  chain: false

The resulting header looks like this, comma-separated if multiple hops contribute:

http
x-forwarded-client-cert: By=spiffe://example.com/ns/prod/sa/api;Hash=468ed33be74eee6556d90c0149c1309e9ba61d6425303443c0748a02dd8de688;Subject="CN=client-a,O=Example,C=GB";URI=spiffe://example.com/ns/prod/sa/client-a

The five forward_client_cert_details modes, and when each is correct:

ModeBehaviourUse when
SANITIZE (default)Removes any inbound XFCC and adds noneEnvoy is not doing client certificate auth
FORWARD_ONLYKeeps the inbound XFCC unchanged, adds nothingEnvoy is a middle hop in a mesh and trusts the previous hop
APPEND_FORWARDKeeps the inbound XFCC and appends this hop's detailsMulti-hop mesh where the full chain of identities is wanted
SANITIZE_SETDiscards any inbound XFCC and sets its ownEnvoy is the trust boundary at the edge; this is the safe default for ingress
ALWAYS_FORWARD_ONLYForwards inbound XFCC even on non-mTLS connectionsAlmost never; it forwards an unauthenticated header verbatim

ALWAYS_FORWARD_ONLY is the one to audit for. It exists for migration scenarios where an upstream component is known to be trustworthy, and it disables the only protection Envoy gives you by default. Hash, the SHA-256 digest of the presented certificate in hex, is always included when Envoy sets the header, and it is the most useful key for a backend to allowlist against, because unlike a DN it does not change when the CA reissues with a different subject formatting.

CRL, OCSP and expiry at the proxy#

Verifying a chain proves the certificate was issued. It does not prove it is still valid. Revocation checking is the part teams skip, and the part that matters on the day a laptop is stolen.

MechanismnginxHAProxyEnvoyPractical notes
CRL filessl_crlcrl-filecrl in validation_contextMust contain a CRL for every CA in the chain, not just the issuing one, or verification fails outright
OCSP for client certs`ssl_ocsp on \leaf with ssl_ocsp_responder`, nginx 1.19.0 and laterNot built inNot built inAdds a network dependency to the handshake path; leaf checks only the client certificate and skips intermediates
Reload behaviourCRL is read at config load, so a new CRL needs a reloadSameSame, unless delivered by SDSAn expired CRL causes verification to fail, so CRL expiry is itself an outage class
Expiry visibility$ssl_client_v_end, $ssl_client_v_remainssl_c_notafterCert/Chain keys in XFCC, parsed downstreamLog it and alert on the minimum across active clients

The failure nobody plans for is CRL expiry. A CRL has its own nextUpdate field, and OpenSSL treats an expired CRL as a verification failure, not as "no revocation information". If your CRL distribution job stops, every client certificate stops verifying at once, with client SSL certificate verify error: (12:CRL has expired). Alert on CRL freshness with the same seriousness as certificate expiry.

TLS 1.3 changes when client certificates can be requested#

Under TLS 1.2, a server that wanted a client certificate only for /admin could complete an anonymous handshake, and then, when the request arrived, trigger a renegotiation with a CertificateRequest. Apache httpd's per-directory SSLVerifyClient is built on exactly this.

TLS 1.3 removed renegotiation entirely (RFC 8446). Its replacement is post-handshake authentication (section 4.6.2), and it is opt-in from the client side: the client must have sent the post_handshake_auth extension in its ClientHello, otherwise the server may not request a certificate later. Most browsers do not implement post-handshake auth, and OpenSSL-based clients must call SSL_CTX_set_post_handshake_auth() explicitly.

The practical consequences:

  • A per-path client certificate requirement that worked on TLS 1.2 fails once the connection negotiates TLS 1.3. The visible symptom in Apache httpd is a 403 on the protected path while the rest of the site works, and a log line about renegotiation not being supported or the client not supporting post-handshake authentication.
  • nginx never supported renegotiation-triggered client auth, so nginx users hit this earlier and settled on the workaround that is now the general answer: request the certificate once, for the whole server block, with ssl_verify_client optional, then enforce per location.
nginx
server {
    listen 443 ssl;
    ssl_verify_client optional;
    ssl_client_certificate /etc/ssl/client-ca.pem;

    location /admin/ {
        if ($ssl_client_verify != SUCCESS) { return 403; }
        proxy_pass http://admin_backend;
    }
    location / {
        proxy_pass http://public_backend;
    }
}

The cost is that every client, including anonymous ones, receives a CertificateRequest and browsers may show a certificate selection prompt. That is the trade: one prompt for everyone, or a feature that no longer exists at TLS 1.3. Pinning the vhost to ssl_protocols TLSv1.2; to keep renegotiation working is a downgrade, and not a defensible one.

mTLS on the upstream leg#

The proxy can also be the client. This is how you stop a compromised pod on the same network from talking directly to your backend.

nginx
location / {
    proxy_pass https://app_upstream;

    proxy_ssl_certificate     /etc/ssl/proxy-client.pem;
    proxy_ssl_certificate_key /etc/ssl/proxy-client.key;
    proxy_ssl_password_file   /etc/ssl/proxy-key.pass;   # if the key is encrypted

    proxy_ssl_verify              on;
    proxy_ssl_trusted_certificate /etc/ssl/internal-ca.pem;
    proxy_ssl_verify_depth        2;
    proxy_ssl_server_name         on;      # default is off
    proxy_ssl_session_reuse       on;
}

The equivalents are server app1 10.0.0.1:443 ssl crt /etc/ssl/proxy-client.pem verify required ca-file /etc/ssl/internal-ca.pem in HAProxy, and tls_certificates inside an UpstreamTlsContext in Envoy.

Two things to watch. First, the proxy's own client certificate expires, and because it is infrastructure rather than an application artefact it tends not to be in anyone's renewal calendar; its expiry takes down every request at once. Second, proxy_ssl_server_name is off by default in nginx, so no SNI is sent upstream and a multi-tenant backend answers with its default certificate, which then fails the name check the moment you turn proxy_ssl_verify on.

Failure modes#

SymptomRoot causeFix
400 Bad Request with body No required SSL certificate was sent; log client sent no required SSL certificateHandshake completed, ssl_verify_client on, client sent no certificateClient must present a certificate; check it has one matching the advertised CA DN list
client SSL certificate verify error: (21:unable to verify the first certificate)Client did not send its intermediate, or the proxy lacks that intermediateHave the client send the full chain, or add the intermediate to ssl_client_certificate
client SSL certificate verify error: (22:certificate chain too long)ssl_verify_depth default of 1 is too shallow for the PKIRaise ssl_verify_depth to match the real chain length
client SSL certificate verify error: (12:CRL has expired) on all clients simultaneouslyThe CRL file passed its nextUpdateRefresh the CRL and reload; alert on CRL age
Browser shows a generic handshake failure, proxy log has nothing usefulClient had no certificate issued by any advertised CA, so it sent an empty Certificate message or abortedCheck openssl s_client -connect host:443 output for the Acceptable client certificate CA names list
HAProxy closes the connection at handshake with verify required and no log detailThe verify failure happens before any HTTP transaction exists to logTemporarily use verify optional plus ssl_c_verify logging to see the OpenSSL error code
Envoy rejects connections; ssl.fail_verify_no_cert incrementsrequire_client_certificate: true and clients are not sending oneConfirm the client is configured for mTLS; check ssl.fail_verify_error and ssl.fail_verify_san to distinguish causes
Backend authorises a request that presented no certificateInbound identity header not stripped, and the backend is reachable without the proxySet the header unconditionally at the proxy and block direct backend access
App stops recognising identities after an nginx upgrade$ssl_client_s_dn switched to RFC 2253 format in nginx 1.11.6, reversing RDN orderParse the DN properly, or use $ssl_client_s_dn_legacy as a stopgap
Per-path client cert requirement returns 403 for modern browsers onlyTLS 1.3 negotiated, renegotiation gone, client did not offer post-handshake authMove the requirement to the whole server block with optional plus a per-location check
Upstream handshake fails with a name mismatch after enabling proxy_ssl_verifyproxy_ssl_server_name off by default, so no SNI upstreamSet proxy_ssl_server_name on and an explicit proxy_ssl_name

Frequently asked questions#

Why does the application not see the client certificate behind a proxy?#

Because the proxy terminated the TLS session, so the certificate was presented to the proxy and consumed there. The connection from the proxy to the application is a separate one that carries no client certificate. The only ways for the application to see the real certificate are TLS passthrough, or having the proxy forward the certificate in a header for the application to parse.

What does "400 No required SSL certificate was sent" mean?#

It means nginx has ssl_verify_client on, the TLS handshake completed, and the client sent no certificate, so nginx rejected the request at the HTTP layer. It is not a handshake error. The usual cause is a client that holds no certificate issued by a CA in the DN list nginx advertised, so it declined to send anything.

How do I stop a client from spoofing the identity header?#

Overwrite it on every request at the verifying proxy rather than only setting it when a certificate is present, and make the backend unreachable except through that proxy. In Envoy, forward_client_cert_details: SANITIZE_SET does the overwrite for you; in nginx and HAProxy, an unconditional proxy_set_header or http-request set-header is required, because an unset header passes through from the client.

Is optional_no_ca ever safe to use?#

It is safe only when something downstream actually validates the certificate. It requests a certificate and accepts it without checking any CA, so on its own it authenticates nothing. Legitimate uses are delegating validation to an authorisation service with richer policy than a CA bundle, or capturing the presented certificate for audit while gating access on something else.

Can I require a client certificate only for one path?#

Not directly at TLS 1.3, because the certificate request happens during the handshake and TLS 1.3 removed renegotiation. The portable pattern is ssl_verify_client optional at the server level and a per-location check of $ssl_client_verify, accepting that all clients see a certificate request. Post-handshake authentication exists in TLS 1.3 but requires client opt-in that most browsers do not provide.

How does Envoy's x-forwarded-client-cert header work?#

Envoy encodes the verified client identity as semicolon-separated key-value pairs including By, Hash, Subject, URI and DNS, and appends its entry to any existing value depending on the forward_client_cert_details mode. Hash is the SHA-256 digest of the client certificate and is always present when Envoy sets the header. The default mode, SANITIZE, removes any inbound value.

Does the proxy check whether a client certificate is revoked?#

Only if you configure it. Chain verification alone proves issuance, not current validity. nginx supports CRL files via ssl_crl and OCSP checking of client certificates via ssl_ocsp in 1.19.0 and later; HAProxy and Envoy support CRL files. Note that an expired CRL causes verification to fail for every client, so CRL freshness needs its own monitoring.

Should the proxy also present a client certificate to the backend?#

Yes, if the backend network is not fully trusted. Upstream mTLS stops anything other than the proxy from reaching the backend, which is the control that makes a forwarded identity header safe to trust. Remember that the proxy's own client certificate has an expiry, and that its expiry fails every request at once rather than one tenant at a time.

Primary sources#

Every normative claim on this page is checked against the specification or the vendor documentation listed here. Where behaviour is version dependent, the version is named in the text.

  1. RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3
  2. RFC 5246: The Transport Layer Security (TLS) Protocol Version 1.2
  3. RFC 5280: Internet X.509 PKI Certificate and CRL Profile
  4. RFC 6960: X.509 Internet PKI Online Certificate Status Protocol
  5. nginx ngx_http_ssl_module
  6. nginx ngx_http_proxy_module
  7. HAProxy configuration manual
  8. Envoy HTTP connection manager x-forwarded-client-cert
  9. Envoy DownstreamTlsContext
  10. Traefik TLS options and PassTLSClientCert middleware

Found something wrong, or behaviour that differs on your version? Report it with the version number and a primary source. Anything substantive is fixed in the page and logged on the corrections page. See editorial standards for how pages are researched, sourced and reviewed.

More in tls and proxy security#