no live upstreams while connecting to upstream: What nginx Is Telling You

The message is logged at the error level by ngx_http_upstream_connect() when the balancer returns "busy", that is, when it cannot offer a single peer (ngx_http_upstream.c). Because nginx was trying to connect at that moment, the log line reads:

[error] 1234#1234: *5678 no live upstreams while connecting to upstream, client: 198.51.100.7, server: example.com, request: "GET / HTTP/1.1", upstream: "http://app_servers/", host: "example.com"

The client receives 502 Bad Gateway: the failure type falls into the default branch of ngx_http_upstream_next(), which maps to NGX_HTTP_BAD_GATEWAY. Note the upstream: field: it shows the group name (app_servers), not an IP, because no server was chosen.

This line is almost never the root cause. It is the consequence of earlier failures: connect() failed (111: Connection refused), upstream timed out or similar errors marked the servers as unavailable, and for the next requests nginx had nobody left to try.

The round-robin balancer skips a server when any of these hold (ngx_http_upstream_round_robin.c):

The upstream module documentation adds an important exception: "If there is only a single server in a group, max_fails, fail_timeout and slow_start parameters are ignored, and such a server will never be considered unavailable" (ngx_http_upstream_module). With one server line the message therefore points at down or max_conns. Careful: a hostname in a server line that resolves to several addresses becomes several servers, and then the failure counting applies.

Which events count as a failure is set by proxy_next_upstream: errors, timeouts and invalid headers always count; http_500, http_502, http_503, http_504 and http_429 count only when listed (proxy_next_upstream).

1. Read the lines before it.

   grep -E 'upstream|no live' /var/log/nginx/error.log | tail -50

   The first errors per server name the real fault: refused, timed out, reset.
2. Test each backend directly from the nginx host:

   curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' http://10.0.0.11:8080/health
   curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' http://10.0.0.12:8080/health

3. Check the effective config for forgotten down flags and limits: nginx -T 2>/dev/null | grep -A10 'upstream app_servers'.
4. Fix the backends (start the service, free memory, fix the crash), then wait fail_timeout or, for an upstream without a shared-memory zone, reload nginx to clear the counters: nginx -t && systemctl reload nginx.
5. Tune failure detection if short blips take the whole pool out: raise max_fails or lower fail_timeout so a server returns sooner, and do not list http_5xx in proxy_next_upstream unless you mean it.
6. Add a fallback: server 10.0.0.20:8080 backup; receives traffic only when the primary servers are unavailable.

More on reading nginx logs: how to debug nginx errors.

A configuration in which one bad minute does not empty the whole group:

upstream app_servers {
    server 10.0.0.11:8080 max_fails=3 fail_timeout=15s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=15s;
    server 10.0.0.20:8080 backup;
    keepalive 16;
}

server {
    location / {
        proxy_pass http://app_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_next_upstream error timeout;
        proxy_next_upstream_tries 2;
    }
}

Three failures within 15 seconds are needed before a server is skipped, and the skip lasts 15 seconds, so a single timeout during a deploy no longer removes a server. The backup server only receives requests when both primaries are unavailable, which turns "no live upstreams" into a slower but working site. proxy_next_upstream_tries 2 caps how many servers one request may try, so a broken pool fails fast instead of walking every server. These numbers are a starting point: tune them against how long your backends actually take to recover. If the backends are containers whose addresses change, remember that nginx resolves a hostname in a server line when it loads the configuration (unless the line carries the resolve parameter), so a stale address can look exactly like a dead server until the next reload.

curl -s -o /dev/null -w '%{http_code}\n' https://example.com/
# 200
tail -f /var/log/nginx/error.log | grep --line-buffered 'no live upstreams'
# (nothing new for several minutes)

Give it longer than fail_timeout: servers marked failed come back only after that window. The HTTP status checker confirms what your public URL returns from outside your network.

• upstream prematurely closed connection — a backend hung up mid-request
• no healthy upstream — the Envoy equivalent
• 502 Bad Gateway fixes — all the ways nginx produces 502
• 502 Bad Gateway — the status code itself
• ECONNREFUSED — the refused connect behind most of these