Why does nginx log connect() failed (13: Permission denied) while connecting to upstream?

nginx logs connect() to 127.0.0.1:3000 failed (13: Permission denied) while connecting to upstream on RHEL, Rocky, AlmaLinux, CentOS Stream and Fedora because SELinux runs nginx in the httpd_t domain, and in the stock policy httpd_t may not open an outgoing TCP connection to any port at all. The kernel refuses the connect() call before a packet leaves, nginx returns 502, and the backend never sees the request. The fix is a boolean: setsebool -P httpd_can_network_connect 1 allows every port, while setsebool -P httpd_can_network_relay 1 allows only ports labelled as web, cache or proxy ports. Relabelling the port with semanage port is not a fix on its own: it only moves the port into the set that httpd_can_network_relay opens.

Two answers circulate, and the reason both seem to work is that each is tested on a different machine. "Run setsebool -P httpd_can_network_connect 1" always works, because that boolean covers every port type. "Add the port to http_port_t with semanage port -a" works only on a server where httpd_can_network_relay (or the obscure httpd_graceful_shutdown) is already on, which is also the server where "8080 works and 3000 does not". On a fresh Rocky Linux 9.8 install, port 8080 failed exactly like port 3000, and so did port 9000, which is already http_port_t. The tables below come from that machine, with SELinux enforcing.

Which upstream ports work, under each boolean#

nginx was pointed at 30 local backend ports, one HTTP server on each, and every port was requested once per boolean setting. 200 means nginx reached the backend; 502 means the connect was denied with error 13.

Read the table from the port your app uses. If its type appears in the httpd_can_network_relay rows, turn that boolean on and you are done. If not, you either relabel the port into one of those types and turn relay on, or turn on httpd_can_network_connect. The common development ports are the awkward ones: Node and Grafana's 3000 is ntop_port_t, Flask's 5000 is commplex_main_port_t, Django and Gunicorn's 8000 is soundd_port_t, and 8081 is transproxy_port_t. None of them is in the relay set, which is the whole of the "8080 works, 3000 does not" mystery.

Two port types in the relay set are easy to miss. The ephemeral range (32768-60999 on these releases) is included, so an app on a high random port works with relay alone. memcache_port_t (11211) and squid_port_t (3128, 3401, 4827) are included too, and so are the FTP and gopher ports, because the boolean was written for httpd acting as a relay.

How the results were produced#

Everything here was observed on 24 October 2026 unless it is attributed to documentation. A Rocky Linux 9.8 GenericCloud image ran as a KVM guest (kernel 5.14.0-687.10.1.el9_8) with SELinux enforcing, selinux-policy-targeted-38.1.75-2.el9_8.1, policycoreutils-python-utils-3.6-5.el9, and nginx 1.20.1 from the Rocky AppStream package, started by its own systemd unit. TCP backends were one Python process listening on 127.0.0.1 on each port; Unix socket backends were the same server on a socket. nginx used one variable proxy_pass (with an IP literal, so no resolver is needed; the URI rules are in nginx proxy_pass and the trailing slash) so every port could be requested by URL:

location ~ ^/p/(\d+)$ { proxy_pass http://127.0.0.1:$1/; }

Booleans were switched with setsebool between runs, and the table above was repeated for each. The policy on AlmaLinux 9.8, Rocky Linux 10.2, AlmaLinux 10.2, CentOS Stream 10 and Fedora 44 was then inspected with seinfo and sesearch (not run with nginx): all four booleans named on this page default to off on every one of them, the labels of ports 3000, 5000, 8000, 8080, 8081, 8888, 9000 and 50080 are the same as in the table, and httpd_can_network_relay opens the same port types.

The denial does not depend on the address. Pointed at 10.0.2.2:3000, the guest's gateway, nginx logged the same 13: Permission denied. It does not depend on whether anything listens, either: port 3999 with no listener gave 13 under the stock policy and 111: Connection refused once httpd_can_network_connect was on. SELinux checks the port label before the connection is attempted, so a 13 tells you nothing about the backend's health.

What the denial looks like in each log#

The nginx error log line is logged at crit, while a refused connection is error. That level difference is a quick way to spot SELinux in a log full of 502s:

[crit] 16031#16031: *301 connect() to 127.0.0.1:3000 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /p/3000 HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "127.0.0.1:8099"

The audit log has the reason. This is the AVC record for the same request, from /var/log/audit/audit.log:

type=AVC msg=audit(1792836361.057:520): avc:  denied  { name_connect } for  pid=16031 comm="nginx" dest=3000 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:ntop_port_t:s0 tclass=tcp_socket permissive=0

Three fields matter. name_connect with tclass=tcp_socket is an outgoing TCP connection. dest=3000 is the port. tcontext=...ntop_port_t is the port's label, and that label is what decides which fix applies. Red Hat's troubleshooting guide recommends searching with ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts recent. If you run it from a script, a cron job or over a non-interactive SSH session, add --input-logs: in testing, ausearch without it printed <no matches> from a non-interactive shell while the log held hundreds of denials.

audit2why translates an AVC record into a cause. Fed the line above:

	Was caused by:
	The boolean httpd_can_network_connect was set incorrectly. 
	Description:
	Allow httpd to can network connect

	Allow access by executing:
	# setsebool -P httpd_can_network_connect 1

For port 8080 (http_cache_port_t), it listed both httpd_can_network_connect and httpd_can_network_relay. For port 9000 (http_port_t), it listed four: those two, httpd_graceful_shutdown and nis_enabled. For port 8888 (unreserved_port_t), it listed httpd_can_network_connect and nis_enabled. The number of booleans audit2why offers is itself a hint: one choice means the port type is outside the relay set.

With setroubleshoot-server installed, sealert -l "*" gives the same advice in longer form, ranks it (the catchall_boolean plugin suggested httpd_can_network_connect at 89.3 confidence for port 3001), and for an unreserved port such as 7860 its connect_ports plugin suggests changing the port type instead. Every sealert report also ends with an audit2allow -M my-nginx recipe. For a TCP port, do not use it: a boolean already exists for exactly this, and Red Hat's guide says audit2allow should not be the first option.

The boolean or the port label: which fix#

Why semanage port alone did nothing#

Red Hat's RHEL 9 guides show semanage port -a -t http_port_t -p tcp 3131 (and 9876 in the troubleshooting chapter) as the way to allow a non-standard port. For a server that listens on that port, the label is the whole fix. For a proxy connecting to it, the label is not enough. After semanage port -a -t http_port_t -p tcp 3000 and semanage port -a -t http_port_t -p tcp 8888, both ports still returned 502 with relay off. With relay on, both returned 200, and port 3001, still ntop_port_t, returned 502. The F5 article reaches the same structure from the other direction: its sesearch output shows that httpd_can_network_relay is what lets httpd_t connect to http_port_t.

The policy source explains it. In apache.te, the only rules that let httpd_t connect to http_port_t sit inside tunable_policy blocks for httpd_can_network_relay and httpd_graceful_shutdown, and every one of those tunables is declared false. httpd_graceful_shutdown is described as "Allow HTTPD to connect to port 80 for graceful shutdown"; it opens http_port_t only (8008, 8443 and 9000 returned 200 with it, 8080 did not). It is not meant as a proxy switch, so use relay instead.

"Port already defined": policycoreutils 3.6 reassigns it anyway#

The F5 article warns that if semanage port -a reports a port as "already defined", the port belongs to another type, and you should not reassign it, "because other services might be negatively affected". The port it adds as its example is 8082, which on current policy is already us_cli_port_t.

On Rocky 9.8 the warning has become a notice. semanage port -a -t http_port_t -p tcp 8080 printed Port tcp/8080 already defined, modifying instead, exited 0, and the port moved to http_port_t in the local policy. The same happened for the range 3000-3001. For a single port inside a policy range, such as 3000 alone, it printed nothing and succeeded. So the safety net the older advice relied on is gone: check semanage port -l | grep -w PORT before you add, and if the port has a type some other confined service uses, prefer the boolean. To see and undo local changes:

semanage port -l -C                         # only your local changes
semanage port -d -t http_port_t -p tcp 3000 # restores ntop_port_t

nis_enabled is not a fix#

audit2why offers nis_enabled for unreserved and http_port_t ports. With it on, nginx reached every unreserved_port_t and ephemeral port in the test, plus the http_port_t ones, but still not 3000, 5000 or 8080. The boolean is not an httpd switch: in the policy it applies to the nsswitch_domain attribute, a large set of domains that do name lookups, so turning it on widens many services at once to fix one proxy.

What httpd_can_network_connect really costs#

The F5 article says this boolean lets httpd_t processes "connect to all TCP socket types that have the port_type attribute". For a reverse proxy whose upstreams are all local, that is more than it needs: any code running as httpd_t can now reach databases, SSH and internal APIs on any host. Whether that matters depends on what else runs in httpd_t and on how far a request can steer proxy_pass; a variable upstream built from request data is the case where it matters most, as covered in SSRF and proxies. On a box where nginx proxies to arbitrary internal hosts on many ports, it is the honest setting, and it is what most guides end up recommending.

setsebool -P writes the setting into the policy store; plain setsebool lasts until reboot. The persistent form took 2.1 seconds in the test, and semanage boolean -l -C then listed httpd_can_network_relay (on , on). Red Hat notes that -P rebuilds the whole policy, so expect it to take longer on a busier system.

Unix sockets: the same 13, three different causes#

proxy_pass http://unix:/run/app/app.sock:; avoids ports, so no port boolean applies, and none of them helps. Error 13 on a socket comes from one of three checks, and the audit log tells you which:

The permissive test separates SELinux from file permissions in one step: setenforce 0, retry, setenforce 1. If the request still fails, the problem is ordinary Unix permissions on the socket or a directory above it, and ls -l of every path component will show it.

The label. A socket inherits its directory's type, and a new directory under /run is var_run_t, which httpd_t may not write to. The policy already labels /var/run/nginx.* and /var/run/php-fpm(/.*)? as httpd_var_run_t. For your own directory, note that semanage fcontext -a -t httpd_var_run_t "/run/app(/.*)?" failed with ValueError: File spec /run/app(/.*)? conflicts with equivalency rule '/run /var/run'; Try adding '/var/run/app(/.*)?' instead. The working form is:

semanage fcontext -a -t httpd_var_run_t "/var/run/app(/.*)?"
restorecon -Rv /run/app

The domain. After the label fix, the next denial was connectto, which checks the domain of the process that owns the listening socket. sesearch lists the domains httpd_t may connect to, and they are specific services (PostgreSQL, Redis, memcached, httpd script domains, init_t and others), not unconfined_t or unconfined_service_t, and on these releases no boolean adds them. A Python app started from a login shell is unconfined_t; started by its own systemd unit it is unconfined_service_t. Socket activation did not change it: with a .socket unit creating /run/app2/app.sock, the AVC still named unconfined_service_t, because systemd labels the socket with the context of the service it will start.

For that case audit2why reports Missing type enforcement (TE) allow rule: no boolean and no label will fix it. What is left is a local policy module or a switch to TCP. Red Hat's guide treats a local module as the last resort, and this is the case where you have reached it. The rule audit2allow produced is one line, and with it loaded both the socket-activated and the plain systemd backend returned 200:

module nginx_app_sock 1.0;
require {
    type httpd_t;
    type unconfined_service_t;
    class unix_stream_socket connectto;
}
allow httpd_t unconfined_service_t:unix_stream_socket connectto;

checkmodule -M -m -o nginx_app_sock.mod nginx_app_sock.te
semodule_package -o nginx_app_sock.pp -m nginx_app_sock.mod
semodule -i nginx_app_sock.pp

That rule lets nginx connect to the socket of any unconfined service, not just yours. If that is too broad, the alternative is TCP on loopback with relay and a port in the relay set, which needs no custom policy.

Plain permissions. With the socket at 0660 owned by appsvc:appsvc, nginx (user nginx) got the same 13 with no AVC at all, and the request still failed in permissive mode. Adding nginx to the group and restarting nginx moved it on to the SELinux connectto check above. A socket under a 0700 home directory failed the same way: the directory, not the socket, blocks the path. Neither is SELinux, and no boolean will change them.

Failure modes#

[crit] connect() to 127.0.0.1:PORT failed (13: Permission denied) while connecting to upstream, nginx on RHEL, Rocky, Alma or Fedora. SELinux name_connect denial. Find the port's type with semanage port -l | grep -w PORT and use the decision table above.

It worked on port 8080 and breaks on 3000 after a redeploy. httpd_can_network_relay is on for this host; 3000 is ntop_port_t, outside its set. Relabel 3000 or turn on httpd_can_network_connect.

semanage port -a succeeded, still 502 with 13. Relay is off, so the new label grants nothing. Turn on httpd_can_network_relay.

A fix that worked until reboot. setsebool without -P. Run it again with -P.

connect() to unix:... failed (13: Permission denied). Check audit.log for sock_file (label), connectto (backend domain), or nothing (file permissions), per the socket table.

connect() to unix:/tmp/... failed (2: No such file or directory) though the socket exists. The nginx systemd unit uses PrivateTmp=true and sees its own /tmp. The nginx documentation's example path unix:/tmp/backend.socket hits exactly this. Move the socket to /run.

connect() failed (111: Connection refused) after the fix. SELinux is no longer the problem; nothing listens on that address and port. This is the ordinary connect failure in 502 vs 503 vs 504, and if nginx runs in a container, read why nginx in Docker returns 502 for proxy_pass localhost instead.

open() ... proxy_temp ... failed (13: Permission denied) while reading upstream. A different 13: nginx can reach the backend but cannot write its buffer files, and the download is cut short rather than failing with 502. That case is in why nginx cuts off large downloads.

Where this answer stops applying#

The port matrix, log lines and Unix socket results are from Rocky Linux 9.8 with selinux-policy-targeted-38.1.75-2.el9_8.1 and nginx 1.20.1 from AppStream. The boolean defaults and port labels were confirmed in the policy files of AlmaLinux 9.8, Rocky 10.2, AlmaLinux 10.2, CentOS Stream 10 and Fedora 44, but nginx was not run on those. Older releases, Amazon Linux and Oracle Linux were not checked; the F5 article itself says their defaults "might differ". The die.net copy of httpd_selinux(8) is from an older policy and lists http_port_t as 80, 443, 488, 8008, 8009 and 8443, and 3128 under http_cache_port_t; current policy adds 81 and 9000 to http_port_t and gives 3128 to squid_port_t, so read the list from semanage port -l on your own host. Upstream TLS does not change any of this, since the check happens before the handshake. For the rest of a working proxy configuration see nginx as a reverse proxy, and for a 502 that is none of these, bisect it with the proxy debugging playbook.

Frequently asked questions#

Should I use setsebool -P httpd_can_network_connect 1 or semanage port?#

For a port outside the relay set, httpd_can_network_connect is the single command that works. semanage port -a -t http_port_t is narrower but only works together with setsebool -P httpd_can_network_relay 1; on its own it changed nothing in testing.

Why does nginx connect to port 8080 but not 3000?#

Because httpd_can_network_relay is on for that host. It opens http_cache_port_t, which includes 8080, but port 3000 is labelled ntop_port_t. On a fresh Rocky 9.8 install neither port works.

Which ports can nginx proxy to with SELinux enforcing and no changes?#

None, in the stock policy on EL9, EL10 and Fedora 44. Every boolean that lets httpd_t make an outgoing TCP connection is off by default, including for port 80 and the other http_port_t ports.

Is it safe to disable SELinux to fix this?#

It removes the error and every other protection SELinux gives the host. A single boolean fixes the proxy case. To confirm that SELinux is the cause, the F5 article suggests making only httpd_t permissive: with semanage permissive -a httpd_t, port 3000 returned 200 and the AVC was logged with permissive=1; semanage permissive -d httpd_t put it back to 502.

Does httpd_can_network_connect fix permission denied on a Unix socket?#

No. It covers TCP ports only. A socket needs the httpd_var_run_t file label, a backend domain nginx may connectto, and file permissions that let the nginx user write to the socket.

Why does semanage say "already defined, modifying instead"?#

The port already has a type, and policycoreutils 3.6 on Rocky 9.8 reassigns it instead of failing. Check what used it with semanage port -l | grep -w PORT first, and undo with semanage port -d.