upstream connect error or disconnect/reset before headers: Reading the Reset Reason

Envoy's router builds this body when an upstream request ends before the backend sent response headers. In the current source the string is assembled as upstream connect error or disconnect/reset before headers. , then retried and the latest if retries happened, then reset reason: and the reason, and optionally , transport failure reason: with the low-level detail (router.cc). A typical response:

HTTP/1.1 503 Service Unavailable
server: envoy

upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

The status is 503, except when the reset reason is protocol error, which gives 502. Unlike no healthy upstream, Envoy did choose a host and tried to talk to it. Older Envoy releases printed shorter reasons such as reset reason: connection failure; the meaning is the same. Operators can hide the transport detail with the runtime flag envoy.reloadable_features.hide_transport_failure_reason_in_response_body, so the tail may be missing.

The reason strings come from Http::Utility::resetReasonToString() (utility.cc); the access log flag is set from the same reason.

The transport failure reason narrows it further: an errno text such as Connection refused means nothing listens; a TLS error string means certificate, SNI or protocol mismatch between the proxy and the backend.

• App listens on localhost only. A server bound to 127.0.0.1 inside the pod is invisible to traffic arriving on the pod IP in some setups. Bind to 0.0.0.0.
• Wrong targetPort. The Service sends to a port the container does not open. kubectl get svc reviews -o jsonpath='{.spec.ports}' must match containerPort.
• mTLS mismatch. A PeerAuthentication in STRICT mode with a client that sends plain text, or a DestinationRule that originates TLS to a pod that already terminates TLS, ends in connection termination or a TLS transport reason.
• Protocol selection. Istio picks the protocol from the Service port name or appProtocol. A port named http in front of a gRPC or HTTP/2-only server produces protocol error.
• Keep-alive race. The app closes idle connections sooner than Envoy stops reusing them; intermittent connection termination on low traffic is the signature.

1. Reproduce from the proxy's network position. From the client sidecar:

   kubectl exec deploy/productpage -c istio-proxy -- curl -sv http://reviews:9080/ 2>&1 | tail -5

2. Check the listener inside the backend pod.

   kubectl exec deploy/reviews -c reviews -- ss -ltn
   # State  Recv-Q Send-Q Local Address:Port
   # LISTEN 0      128          0.0.0.0:9080

   A port bound to 127.0.0.1, or a different port, explains connection failure.
3. Check TLS policy. istioctl x describe pod REVIEWS_POD shows the effective mTLS mode and DestinationRule; for TLS to an external backend test the handshake: openssl s_client -connect 203.0.113.10:443 -servername api.example.com -brief </dev/null.
4. Check port naming. Name the Service port grpc or http2, or set appProtocol, when the backend speaks HTTP/2.
5. Fix keep-alive. Make the backend's idle timeout longer than the proxy's, or limit how long Envoy keeps idle upstream connections (common_http_protocol_options.idle_timeout on the cluster).
6. Watch the backend's own logs at the same timestamp: an OOM kill or restart lines up with connection termination.

for i in $(seq 1 50); do curl -s -o /dev/null -w '%{http_code}\n' https://example.com/api/; done | sort | uniq -c
#      50 200

Intermittent cases need volume: fifty requests without a single 503 is a better test than one. In the access log the UF, UC or UPE flags must stop appearing for the cluster. For the public address, the HTTP status checker shows the status code and body the proxy returns.

• no healthy upstream — no host to try at all
• upstream request timeout — connected, but no answer in time
• upstream prematurely closed connection — the nginx counterpart of connection termination
• ECONNREFUSED — the same refused connect seen from an application
• mTLS — mutual TLS between services