SSL Handshake Failed: Root Causes and Step-by-Step Diagnosis

SSL Handshake Failed: Root Causes and Step-by-Step Diagnosis

“SSL handshake failed” is an umbrella term for a dozen different problems that can occur during TLS connection setup. The client and server couldn't agree on protocol version, cipher suite, certificate, or SNI. This guide walks through every common cause, shows the diagnostic commands, and gives you a 15-minute algorithm that leads to the root cause.

What a TLS handshake is and where it can break

A TLS handshake is a multi-step message exchange before encrypted data flows. TLS 1.2 needs 2 RTT, TLS 1.3 — 1 RTT. Each step can fail:

1. ClientHello: client sends supported TLS versions, cipher suites, and SNI.
2. ServerHello: server picks version and cipher, sends its certificate.
3. Certificate verification: client checks chain, expiry, and hostname.
4. Key exchange: ECDHE or RSA session key agreement.
5. Finished: both sides confirm agreed parameters.

A handshake failure is a failure at one of these steps. To find where, run a verbose log:

openssl s_client -connect example.com:443 -servername example.com -tlsextdebug -status
# verbose curl
curl -vI https://example.com 2>&1 | grep -E "TLS|SSL|handshake"

Cause 1: incompatible protocol versions

Most common cause — client only supports TLS 1.0/1.1, server dropped them (correctly). Check which versions the server accepts:

for v in tls1 tls1_1 tls1_2 tls1_3; do
    echo -n "$v: "
    openssl s_client -connect example.com:443 -$v < /dev/null 2>&1 | grep "Cipher is"
done

If the client is old Java (pre-8u261), iOS < 11, Windows XP, Android < 4.4 — it doesn't speak TLS 1.2+. Fix: upgrade the client, or temporarily allow TLS 1.1 on the server until migration. See our TLS 1.3 vs 1.2 migration guide.

Cause 2: SNI mismatch

SNI (Server Name Indication) is a TLS extension that lets multiple SSL/TLS проверку sites live on one IP. The client must specify which hostname it wants in ClientHello. No or wrong SNI — server returns a default cert or rejects the handshake:

# Without SNI — often fails
openssl s_client -connect example.com:443

# With SNI — works
openssl s_client -connect example.com:443 -servername example.com

Seen with old curl, wget, Python 2 scripts, IoT devices. Upgrade the client or pass SNI explicitly.

Cause 3: incompatible cipher suites

Client and server need at least one cipher in common. If you hardened nginx to AEAD-only, older clients (RSA key exchange) will break. Check what the server offers:

nmap --script ssl-enum-ciphers -p 443 example.com

Safe and compatible Mozilla Intermediate profile:

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:\
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:\
ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305;
ssl_prefer_server_ciphers off;

More on cipher choice — weak cipher suites.

Cause 4: problem with the server certificate

The handshake can fail because of the cert itself:

• Expired or not-yet-valid — see NET::ERR_CERT_DATE_INVALID.
• Incomplete chain (missing intermediate) — see incomplete chain.
• Self-signed — see self-signed certificate.
• CN/SAN doesn't cover the requested hostname.
• Certificate revoked (CRL/OCSP).

Full chain check:

openssl s_client -connect example.com:443 -servername example.com -showcerts < /dev/null \
    | openssl verify -CApath /etc/ssl/certs

Cause 5: mutual TLS (client certificate)

If the server uses mTLS, it demands a client cert. Without one the handshake fails with CertificateRequired. In nginx:

ssl_client_certificate /etc/nginx/ca.crt;
ssl_verify_client on;  # or optional

Client must provide the cert:

curl --cert client.crt --key client.key https://api.example.com

Cause 6: MITM proxies, antivirus, corporate firewalls

The client side may have a TLS-intercepting proxy (Kaspersky, ESET, corporate Zscaler). It replaces the server cert with its own, which the client doesn't trust. Symptom: “self-signed certificate in chain” from curl on a healthy site. Fix — trust the proxy's CA, or disable cert verification only for tests (never in prod).

15-minute diagnostic algorithm

1. Run enterno.io SSL Checker — instant overview and issue list.
2. openssl s_client -connect host:443 -servername host — look at the first error.
3. Check protocol versions and cipher suites (above).
4. Check cert expiry and chain.
5. Try from a different client / network (rule out MITM proxy).
6. Compare nginx/Apache config to Mozilla SSL Config Generator.

Frequently asked questions

Handshake works with curl but fails in the browser — why?

Usually an OCSP stapling issue or an unsupported cipher. Browsers are stricter than curl. Run a test at SSL Labs — it simulates dozens of different clients.

How do I enable verbose TLS logs in nginx?

error_log /var/log/nginx/error.log debug; plus ssl_debug on; in recent versions. Look for “SSL_do_handshake() failed”.

Is TLS 1.3 always compatible with old clients?

No. Pre-2018 clients often lack TLS 1.3 support. Keep TLS 1.2 and 1.3 enabled in parallel.

What does “alert handshake failure” mean?

A generic OpenSSL message. You need a more detailed log — server-side (nginx debug) or client-side (openssl s_client -tlsextdebug -msg).

Conclusion

SSL handshake failures always have a concrete cause: version, cipher, cert, SNI, or mTLS. A methodical openssl check gets you to the root cause in 95% of cases within 15 minutes. The enterno.io SSL Checker accelerates step one — run it before hand-debugging to rule out obvious issues. Continuous monitoring via Monitors warns when a deploy regresses your config.

TLS 1.3 — RFC 8446. SNI — RFC 6066. Config generator — Mozilla SSL Config Generator.