HAProxy health check failed, what "Layer4 connection problem", "Layer4 timeout" and "Layer7 wrong status" mean

The text after reason: in an HAProxy health check line is the check status. It says which layer of HAProxy's own test failed. Layer4 connection problem (L4CON) and Layer4 timeout (L4TOUT) mean the TCP connect failed. Layer6 means the TLS handshake failed. Layer7 wrong status, Layer7 invalid response and Layer7 timeout mean the server answered the test request, but with the wrong status, the wrong content, or too slowly.

Each line below was copied from HAProxy 3.2.26 logs on 5 November 2026. The setup is described further down. The last two columns name the most common cause and its fix. Each one is explained in its own section.

What a health check is, and what a check status is#

A health check is a small test request that HAProxy sends to each server on a timer, on its own, apart from real user traffic. If enough checks fail in a row, HAProxy marks the server DOWN and stops sending users to it, and if enough pass again, it marks it UP. You turn checks on with the word check on a server line.

The check status is the short code HAProxy gives the result of the last check. The management guide lists them in its CSV section: L4OK, L4TOUT, L4CON, L6OK, L6TOUT, L6RSP, L7OK, L7OKC, L7TOUT, L7RSP, L7STS, plus SOCKERR ("socket error"), INI and UNK. The stats page shows the code, the log spells it out in words, and the table above pairs the two.

The layer numbers come from the network model: layer 4 is the TCP connection, layer 6 is TLS and layer 7 is HTTP. So the code tells you how far the test got before it failed. A Layer 7 failure means TCP worked, and so did TLS if you use it.

What the check sends depends on its type. The manual's entry for check says that with no other method set, a server is healthy "when a connection can be established at the highest configured transport layer". That means TCP, or TLS when ssl or check-ssl is set. Only option httpchk adds an HTTP request on top, which is why a plain check can only fail at Layer4 or Layer6.

Where the line appears, and the line you may not see#

On HAProxy 3.2.26 with default settings, a server that fails its checks gives one line, at the moment it goes DOWN:

Server be_api/s1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.

The line many forum posts quote, Health check for server ... failed, did not appear in that test. It appeared only once option log-health-checks was added to the backend:

Health check for server be_flap_log/s1 failed, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms, status: 2/3 UP.
Health check for server be_flap_log/s1 failed, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms, status: 1/3 UP.
Health check for server be_flap_log/s1 failed, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms, status: 0/2 DOWN.
Server be_flap_log/s1 is DOWN. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.

The manual says that "by default, failed health check are logged if server is UP". In this test that was not what happened, so turn the option on if you want each failed check logged. It is worth it, because the status: 2/3 UP part counts down to DOWN, so you can see a server failing one check in three long before it falls out.

Note also that with the option on, the final is DOWN line loses its reason:, which now sits on the check lines above it.

How the cases were reproduced#

Everything below was observed on 5 November 2026 unless it is quoted from the manual. HAProxy 3.2.26 ran from the official haproxy:3.2-alpine image on Docker 28.1.1, on one user-defined network. Each case had its own backend with one server and no inter, rise or fall set, so the defaults applied. The servers were Python 3.13 test servers (one per behaviour: refuse, stall, slow, junk, redirect, 404, 503) and nginx 1.30 with a plain port 80 and a TLS port 443. The stats came from show stat on the runtime API.

Layer4 connection problem (L4CON)#

HAProxy tried to open a TCP connection and the server's machine said no. The info: part names the error, and the management guide gives two examples: "Connection refused" (tcp rst) or "No route to host" (icmp). In the test, a port with nothing on it gave Connection refused in 0 ms.

The usual causes:

• The app is not running, or it crashed.
• The server line has the wrong port, or the check uses a different port set with port.
• The app listens only on 127.0.0.1 on its own machine or container, so a connection from HAProxy's address is refused.

The fix is to make the app listen where the server line points. Test it from the HAProxy machine, not from your laptop, because only that machine's view matters; the section on what the check really sends shows how.

Layer4 timeout (L4TOUT)#

HAProxy sent a TCP connection request and got no answer at all, neither yes nor no. In the test, an address that nothing answered gave Layer4 timeout, check duration: 2001ms, and so did a container that was stopped, so that its address vanished.

The usual causes:

• A firewall or security group drops packets from the HAProxy host to that port. A rule that lets in your office IP but not HAProxy's has exactly this effect.
• The server's address is wrong or no longer exists, for example an old IP after a VM or container moved.

The 2001 ms is not a coincidence. Without timeout check, the manual says HAProxy "uses 'inter' for complete check timeout (connect + read)". inter defaults to 2000 ms, so the connect gave up at 2 seconds. The fix is on the network side: let the HAProxy host's IP reach the server's port, or correct the address.

Layer6 invalid response and Layer6 timeout: the TLS layer#

Layer 6 shows up only when the check uses TLS, which happens when the server line has ssl or when you add check-ssl. The manual says check-ssl "inserts an SSL transport layer below the checks".

L6RSP means the TLS handshake failed. Two quite different mistakes gave the very same line in the test:

• check-ssl on a port that speaks plain HTTP. The Python server logged a 400 for the garbled bytes, and HAProxy logged Layer6 invalid response, info: "SSL handshake failure".
• ssl verify required against a server whose certificate the ca-file cannot verify. Here it was a self-signed certificate checked against the system CA bundle.

Because the text is the same, try both in turn. First ask whether the port speaks TLS at all. Then check the CA file. With verify required, the manual says the certificate name is checked against sni or, without SNI, against verifyhost. Health checks do not use the server's sni: the manual says to set check-sni for that. The site's guide to upstream TLS handshake errors has a section on HAProxy's TLS defaults.

L6TOUT means the TCP connect worked but no TLS answer came back in time. In the test, a port that accepted the connection and then said nothing gave Layer6 timeout, check duration: 2002ms. Point the check at the real TLS port.

The check port trap on TLS servers#

This one looks like Layer 7, but it is a TLS problem. Take a TLS backend checked on its own port:

server s1 web:443 ssl verify required ca-file /lab/c.pem verifyhost web.internal check port 443

The same line without port 443 passed its check, so the TLS settings were right. With it, the check failed with Layer7 wrong status, code: 400, info: "Bad Request", and the manual's check entry explains why. Once addr or port is set, "it is assumed the server isn't checked on the service port, and configured encapsulation parameters are not reused". It adds that check-ssl must be set explicitly "to use SSL/TLS". So the check sent plain HTTP to nginx's TLS port, and nginx answered with the 400 page explained in plain HTTP sent to an HTTPS port. Adding check-ssl to the same line made the check pass with L7OK, 200.

The same 400 appears without any port when the server line has no ssl but points at port 443. Either way, the code is a Layer 7 one, but the fix is TLS.

Layer7 wrong status (L7STS)#

The server answered with valid HTTP, but the status code did not count as healthy. The manual's option httpchk entry sets the rule: "responses 2xx and 3xx are considered valid, while all other ones indicate a server failure". The code: and info: parts of the line are the server's own status and reason phrase, so read them first.

Most L7STS failures come from the default request. Per the manual, the method defaults to OPTIONS ("When not set, the 'OPTIONS' method is used"), the URI "defaults to '/'" and the version "defaults to 'HTTP/1.0'". No Host header is sent unless you add one. So a bare option httpchk sends this:

OPTIONS / HTTP/1.0

The Python test server logged exactly "OPTIONS / HTTP/1.0" for each check. Real users send GET / with a Host header, which is why the site can work in a browser while every check fails. The cases seen in testing:

• code: 501, "Unsupported method ('OPTIONS')". The app does not handle OPTIONS at all, as Python's built-in http.server does not.
• code: 404, "Not Found". Nothing lives at /, as with an API that serves only /api/... and /healthz.
• code: 400, "Bad Request". With http-check send ... ver HTTP/1.1 and no hdr Host, nginx rejected the request. The manual notes that "the Host field is mandatory in HTTP/1.1", and warns that "no Host header is automatically added in HTTP/1.1 or above requests". The same code also comes from the TLS trap in the section above.
• code: 503, "Service Unavailable". The app itself reports that it is not ready, so here the check is doing its job.

One fix covers the first three: tell the check what to send.

backend be_api
    option httpchk
    http-check send meth GET uri /healthz ver HTTP/1.1 hdr Host api.internal
    server s1 10.0.2.11:8080 check

That http-check send form arrived in HAProxy 2.2, and the HAProxy configuration guide covers it next to the older one-line form.

When a 301 fails the check, and when it does not#

A redirect alone does not fail a default check. A server that answered every request with 301 to https:// stayed UP, with L7OK, 301 in the stats. It failed only in a backend that also had http-check expect status 200:

Server be_redirect_expect/s1 is DOWN, reason: Layer7 wrong status, code: 301, info: "Moved Permanently", check duration: 1ms.

The expect rule replaced the "2xx and 3xx" default with one exact status. So if you see code: 301, look for an expect line. Then either check the URI the redirect points to, or accept the 301 with http-check expect status 301. Accepting the redirect can hide a broken app, so checking the real URI is the better fix.

Layer7 invalid response (L7RSP) and Socket error (SOCKERR)#

The management guide defines L7RSP as "layer 7 invalid response - protocol error". You might expect a server that sends junk to cause it, but in the test it did not. Four broken replies all gave Socket error (SOCKERR) instead:

• a reply of hello there with no HTTP status line;
• an SSH banner, as from an SSH port;
• a status line with letters for the code, HTTP/1.1 abc OK;
• a server that closed with no reply, or with a status line and no end of headers.

A reply whose body was shorter than its Content-Length still passed, since the status line and headers were enough.

L7RSP came from one place in the test: an http-check expect rule that did not match. With http-check expect string healthy against a page that said ok, the line read Layer7 invalid response, info: "HTTP content check did not match". That fits the manual's http-check expect entry, where error-status is the status set when an expect rule fails, and "By default 'L7RSP' is used".

So read them this way. L7RSP means your expect rule rejected a valid HTTP answer: fix the rule or the endpoint. SOCKERR on an HTTP check means the port does not speak HTTP the way the check expects, so make sure the port number really is the HTTP port.

Layer7 timeout (L7TOUT): timeout check against inter#

The server accepted the connection but did not finish its answer in time. In the test, a port that accepted and never replied gave Layer7 timeout, check duration: 2001ms, and so did a working app that took 3 seconds to answer.

The time limit is easy to miss. The manual's timeout check entry says that when it is not set, HAProxy "uses 'inter' for complete check timeout (connect + read)". So with no timeout check line, any check slower than 2 seconds fails. When it is set, the connect timeout becomes the smaller of timeout connect and inter, and timeout check is added on top as a read timeout.

Adding timeout check 5s to the same backend made the 3-second app pass, and the stats showed L7OK, 200 with a check duration of 3003 ms. The fix, then:

backend be_api
    option httpchk
    http-check send meth GET uri /healthz
    timeout check 5s
    server s1 10.0.2.11:8080 check

Before raising the limit, ask why a health endpoint takes seconds. A check that queries a database or calls other services is slow for reasons that have nothing to do with this one server. That design can take every server down at once, which health checks and upstream failover explains, with the check defaults of five proxies side by side.

The check is not your curl#

When a check fails and the app works for you, compare the two requests. They differ in four ways:

1. Source address. The manual says check traffic uses the backend's source address, so it comes from the HAProxy host. In the test, the Python server logged every check from the HAProxy container's IP. A firewall rule that lets your laptop in proves nothing about HAProxy.
2. Port. The check uses the server port, or the port setting if there is one.
3. Method, URI and version. OPTIONS / HTTP/1.0 by default, not GET /.
4. Host header. None by default. Your browser always sends one.

To see what the check sees, run the request from the HAProxy host itself. A curl command always adds a Host header, so send the bytes by hand:

printf 'OPTIONS / HTTP/1.0\r\n\r\n' | nc 10.0.2.11 8080

If that prints an error status, you have found your L7STS, and if it hangs, you have an L7TOUT. If nc cannot connect, you are back at Layer 4, and the firewall or bind address is the place to look.

How long until DOWN, and until UP again#

The manual gives the defaults: inter 2000 ms, fall 3 and rise 2, so a server goes DOWN after 3 failed checks in a row and UP after 2 passes. How long that takes depends on how the checks fail. These are the timings from the test:

The second row is the one to remember. In the test log, refused checks came 2 seconds apart and timed-out checks came 4 seconds apart: 2 seconds waiting, then the next 2-second gap. A server that stops answering therefore takes almost twice as long to leave the pool as one that refuses.

The last row is just as easy to miss. After a start (here, a container restart), a server with a refused port went DOWN within 0.8 seconds. One with a dead address went DOWN after a single 2-second timeout. It did not wait for three failures. The manual's init-state entry (default none) does not spell this out, so treat it as observed.

To detect failures faster, the manual offers fastinter, the interval used while a server is "transitionally" going down or coming up.

Worked case: one DOWN line traced to its cause#

Take an API on nginx behind HAProxy. It answers fine when you call it from a laptop, yet HAProxy marks both servers DOWN. The backend reads:

backend be_api
    option httpchk
    server s1 10.0.2.11:8443 ssl verify required ca-file /etc/haproxy/ca.pem check port 8443
    server s2 10.0.2.12:8443 ssl verify required ca-file /etc/haproxy/ca.pem check port 8443

Suppose the log shows Server be_api/s1 is DOWN, reason: Layer7 wrong status, code: 400, info: "Bad Request". Walk it through.

1. The code is Layer7, so TCP connected, and firewalls and bind addresses are not the problem.
2. The status is 400, not 404 or 501. So the server did not reject the method or the path, but the request itself.
3. Look at the server line: it has ssl and check port 8443. Per the manual's check entry, setting port means the encapsulation, TLS included, is not reused, so the check speaks plain HTTP to a TLS port.
4. nginx answers plain HTTP on a TLS port with a 400, as it did in the test. That fits the line.
5. The fix is to delete port 8443, since it matches the server's port anyway, or to add check-ssl. Once TLS works, the default OPTIONS / may still fail with 501 or 404, so add http-check send meth GET uri /healthz ver HTTP/1.1 hdr Host api.internal in the same change.
6. Had the line said Layer6 invalid response instead, the path would split at step 1. TLS itself would have failed, and the next suspect would be the ca-file.

Where this answer stops applying#

Everything above is HAProxy 3.2.26 in HTTP mode with option httpchk or a plain check, on Docker on Linux. Other versions may word the lines differently, so match on your own log. The test did not cover agent checks, option tcp-check sequences, the database checks (mysql-check, pgsql-check, redis-check), check-send-proxy, observe layer7, dynamic servers or HTTP/2 checks. The manual notes that dynamic servers do not inherit ssl for checks at all, even without port. A server that is DOWN gives users a 503 with <NOSRV>, which is covered in why HAProxy returns 503 No server is available. A passing check proves only that the check endpoint works, not the pages real users load, so keep watching real error rates in the access log too.

Frequently asked questions#

What does "Layer4 connection problem" mean in HAProxy?#

HAProxy's health check could not open a TCP connection to the server, and the server's machine actively refused it. In testing the info: part read Connection refused, which means nothing listens on that IP and port. Start the app, or fix the port or bind address.

What does "Layer4 timeout" mean in HAProxy?#

The health check sent a TCP connection request and got no answer before the timeout, 2 seconds by default. Usually a firewall drops packets from the HAProxy host, or the server's address no longer exists. Allow the HAProxy host's IP to the port, or fix the address.

Why does HAProxy say "Layer7 wrong status" when the site works?#

The check sends a different request from your browser. option httpchk sends OPTIONS / HTTP/1.0 with no Host header by default. Apps that reject OPTIONS, serve nothing at / or need a Host fail it with 501, 404 or 400. Set the request with http-check send meth GET uri /healthz.

Why does my HAProxy health check fail with code 400 Bad Request?#

Either the check uses HTTP/1.1 without a Host header, or it sends plain HTTP to a TLS port. Add hdr Host your.name to http-check send. On a TLS server with check port, add check-ssl, because setting port stops the check from using the server's ssl.

Does HAProxy treat a 301 redirect as a failed health check?#

Not by default. The manual counts every 2xx and 3xx answer as healthy. A 301 fails the check only when an http-check expect status 200 rule asks for one exact code. Point the check at the URI the redirect leads to instead.

What is the difference between timeout check and inter in HAProxy?#

inter is the time between checks, 2 seconds by default. If timeout check is not set, inter is also the whole time limit for each check. With timeout check set, it becomes the read timeout after the connection is made.

How long does HAProxy take to mark a server DOWN?#

With the defaults (inter 2s, fall 3), a server that refuses connections went DOWN 6.6 seconds after it stopped, in testing. A server that stopped answering took 11.1 seconds, because each timed-out check waits 2 seconds. A server that is already broken when HAProxy starts goes DOWN on the first check.