How to Debug a TLS Handshake

1. Quick probe: openssl s_client -connect example.com:443 -servername example.com -showcerts
2. Check ALPN: openssl s_client -alpn h2,http/1.1 -connect example.com:443
3. Check TLS versions: openssl s_client -tls1_3 -connect example.com:443, -tls1_2, -tls1
4. Check a specific cipher: openssl s_client -cipher ECDHE-RSA-AES128-GCM-SHA256 -connect example.com:443
5. Network-level with Wireshark: capture tcp port 443, use "(pre)-Master-Secret log" to decrypt
6. Application-level: curl -v https://example.com 2>&1 | grep -E "SSL|TLS|ALPN"
7. Overall assessment: Enterno SSL checker — grade + full chain + issues

• Without -servername openssl skips SNI → server sends default cert, not yours
• Wireshark without TLS keys can't decrypt payload (metadata visible)
• curl CURLOPT_VERBOSE on PHP skips TLS handshake detail, only application
• Session tickets cache and replay a successful handshake — use -no_ticket for debug
• Corporate proxy with TLS inspection alters the handshake result — test outside proxy

To debug a TLS handshake, use tools like OpenSSL to capture and analyze the handshake process. Start with the command openssl s_client -connect example.com:443 -msg to view detailed messages exchanged during the handshake. If issues arise, check the server's certificate chain, verify supported protocols, and inspect firewall settings. For a deeper analysis, consider using Wireshark to capture network packets.

The TLS handshake is a crucial part of establishing a secure connection between a client and a server. It involves several steps, including:

1. Client Hello: The client sends a 'Client Hello' message to the server, which includes supported TLS versions, cipher suites, and a randomly generated number.
2. Server Hello: The server responds with a 'Server Hello' message, confirming the TLS version and cipher suite chosen from the client's offered list.
3. Server Certificate: The server sends its digital certificate to the client, which contains the server's public key and is issued by a trusted Certificate Authority (CA).
4. Key Exchange: The client and server perform a key exchange to establish a shared secret, using the server's public key.
5. Finished Messages: Both parties send 'Finished' messages to confirm that the handshake was successful.

Debugging this process requires a thorough understanding of each stage and the potential issues that can arise. Here are some common problems and how to troubleshoot them:

• Certificate Issues: Ensure the server’s certificate is valid, trusted, and not expired. Use the command openssl x509 -in server.crt -text -noout to inspect the certificate details.
• Protocol Mismatches: Verify that both the client and server support compatible TLS versions. Use the command openssl s_client -connect example.com:443 -tls1_2 to test TLS 1.2 specifically.
• Cipher Suite Compatibility: Check that the selected cipher suite is supported by both parties. Use openssl ciphers -v to list the available ciphers on the server.

Additionally, you can use Wireshark to capture and analyze the TLS handshake. Start Wireshark and apply a filter for TLS traffic:

tls

After capturing the packets, look for the 'Client Hello' and 'Server Hello' messages to diagnose where the handshake fails. Each message will provide insights into the supported versions and cipher suites, allowing you to identify mismatches.

For instance, if the 'Client Hello' offers TLS 1.2 but the 'Server Hello' responds with a TLS 1.0 session, it indicates a version mismatch. In such cases, ensure that the server is configured to support the desired protocols by checking server settings in configuration files like nginx.conf or httpd.conf.

In conclusion, debugging a TLS handshake involves a systematic approach: analyze the handshake messages, verify certificates, check protocol and cipher compatibility, and utilize network analysis tools. By methodically addressing each component, you can effectively troubleshoot and resolve TLS handshake issues.