2023-03-31
When setting up certificates on a website, you might get warnings like these:
Or doing API requests, you can get exceptions like these (example using Kotlin & JVM)
java.nio.channels.ClosedChannelException: null
at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1065)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)or
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:353)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:296)This points to a problem with certificates.
Certificates are cryptographic documents which can be used to attest that you are connecting to a server which has successfully claimed the ownership of the domain you are connecting to. This is useful to prevent a lot of “impersonation“ or “Man in the Middle“ (MitM for short) attacks, where a third-party, malicious server pretends to be a legitimate one you were trying to access. The usage of TLS Certificates is what distinguishes HTTPS from HTTP (which has no cryptographic assurances that the request or response was not tampered with).
For example, the current website were you are reading (or have retrieved) this page probably was protected by a certificate. To quickly check some information about a given certificate, web browsers such as Firefox and Chrome typically have a mechanism associated with the padlock right next to the URL bar, but the exact mechanism varies. Once you get there, you may see something like this:
The most important fields for troubleshooting are typically:
Common Name (CN, right in the general info) and Certificate Subject Alternative Names (part of the “Extensions“ of the certificate, can be found in the detailed view): contains the DNS names for which the certificate is valid. In the pictured example, the certificate is valid for abc.atlassian.net, but not for abc.atlassian.com (the end does not match) or abc.cde.atlassian.net (the * works as a wildcard on 1 level only, not multiple) or abc.atlassian.net.com (the end portion really has to match and cannot have extraneous characters)
Validity Period, aka “Not Before“ and “Not After“ fields: tell when the certificate is valid. Outside that time range, the certificate is deemed invalid
Issued By / Certificate Hierarchy: to validate a given certificate, it has to be connected or “vouched“ by a Trusted Root Certificate. Trusted Certificates are a small set of certificates which are widely distributed on browsers and OSes, and serve as the basic set of certificates that are trusted. These certificates can be used to generate sub-certificates that are trusted by extension, and some of them can in turn produce other sub-sub-certificates, and so on, until they create a Leaf certificate, which represents the last level of certification. The path from a root certificate to a leaf certificate is called a certificate chain. For instance, in the example above, there are 3 certificates in the certificate chain:
USERTrust RSA Certification Authority - a root certificate
GEANT OV RSA CA 4 - an intermediate certificate
omega-names-00.tecnico.ulisboa.pt - the leaf certificate
Root certificates are certificates where the “Issued By” field points to themselves, and are many times called “self-signed certificates” (as they sign/vouch themselves).
This is a list of useful tools when dealing with TLS problems, ordered by highest return on investment for troubleshooting purposes.
Open the URL in the browser, and read what the browser is saying to you wrt the “safety“/”privacy” of accessing that website. For example, this message immediately indicates the problems on Common Name:
These websites, when given an URL, detect a large number of potential problems with TLS and website configurations, but they may require further background knowledge to filter some false positives. Other websites exist, but I’ve used these in the past and they worked.
OpenSSL is a widely used cryptographic library which can be directly used via the command openssl. To use it to establish the beginning of a TLS version with a given website, you can use:
Where <PORT> usually is 443, as that is the default port for HTTPS (and <URL> does not include https://). The output is something like this:
OpenSSL output
➜ ~ </dev/null openssl s_client -connect expired.badssl.com:443
CONNECTED(00000005)
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.badssl.com
verify error:num=10:certificate has expired
notAfter=Apr 12 23:59:59 2015 GMT
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.badssl.com
notAfter=Apr 12 23:59:59 2015 GMT
verify return:1
---
Certificate chain
0 s:OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.badssl.com
i:C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
1 s:C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
i:C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
2 s:C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
i:C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFSzCCBDOgAwIBAgIQSueVSfqavj8QDxekeOFpCTANBgkqhkiG9w0BAQsFADCB
kDELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
A1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxNjA0BgNV
BAMTLUNPTU9ETyBSU0EgRG9tYWluIFZhbGlkYXRpb24gU2VjdXJlIFNlcnZlciBD
QTAeFw0xNTA0MDkwMDAwMDBaFw0xNTA0MTIyMzU5NTlaMFkxITAfBgNVBAsTGERv
bWFpbiBDb250cm9sIFZhbGlkYXRlZDEdMBsGA1UECxMUUG9zaXRpdmVTU0wgV2ls
ZGNhcmQxFTATBgNVBAMUDCouYmFkc3NsLmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAMIE7PiM7gTCs9hQ1XBYzJMY61yoaEmwIrX5lZ6xKyx2PmzA
S2BMTOqytMAPgLaw+XLJhgL5XEFdEyt/ccRLvOmULlA3pmccYYz2QULFRtMWhyef
dOsKnRFSJiFzbIRMeVXk0WvoBj1IFVKtsyjbqv9u/2CVSndrOfEk0TG23U3AxPxT
uW1CrbV8/q71FdIzSOciccfCFHpsKOo3St/qbLVytH5aohbcabFXRNsKEqveww9H
dFxBIuGa+RuT5q0iBikusbpJHAwnnqP7i/dAcgCskgjZjFeEU4EFy+b+a1SYQCeF
xxC7c3DvaRhBB0VVfPlkPz0sw6l865MaTIbRyoUCAwEAAaOCAdUwggHRMB8GA1Ud
IwQYMBaAFJCvajqUWgvYkOoSVnPfQ7Q6KNrnMB0GA1UdDgQWBBSd7sF7gQs6R2lx
GH0RN5O8pRs/+zAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUE
FjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwTwYDVR0gBEgwRjA6BgsrBgEEAbIxAQIC
BzArMCkGCCsGAQUFBwIBFh1odHRwczovL3NlY3VyZS5jb21vZG8uY29tL0NQUzAI
BgZngQwBAgEwVAYDVR0fBE0wSzBJoEegRYZDaHR0cDovL2NybC5jb21vZG9jYS5j
b20vQ09NT0RPUlNBRG9tYWluVmFsaWRhdGlvblNlY3VyZVNlcnZlckNBLmNybDCB
hQYIKwYBBQUHAQEEeTB3ME8GCCsGAQUFBzAChkNodHRwOi8vY3J0LmNvbW9kb2Nh
LmNvbS9DT01PRE9SU0FEb21haW5WYWxpZGF0aW9uU2VjdXJlU2VydmVyQ0EuY3J0
MCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5jb21vZG9jYS5jb20wIwYDVR0RBBww
GoIMKi5iYWRzc2wuY29tggpiYWRzc2wuY29tMA0GCSqGSIb3DQEBCwUAA4IBAQBq
evHa/wMHcnjFZqFPRkMOXxQhjHUa6zbgH6QQFezaMyV8O7UKxwE4PSf9WNnM6i1p
OXy+l+8L1gtY54x/v7NMHfO3kICmNnwUW+wHLQI+G1tjWxWrAPofOxkt3+IjEBEH
fnJ/4r+3ABuYLyw/zoWaJ4wQIghBK4o+gk783SHGVnRwpDTysUCeK1iiWQ8dSO/r
ET7BSp68ZVVtxqPv1dSWzfGuJ/ekVxQ8lEEFeouhN0fX9X3c+s5vMaKwjOrMEpsi
8TRwz311SotoKQwe6Zaoz7ASH1wq7mcvf71z81oBIgxw+s1F73hczg36TuHvzmWf
RwxPuzZEaFZcVlmtqoq8
-----END CERTIFICATE-----
subject=OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.badssl.com
issuer=C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
---
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5003 bytes and written 446 bytes
Verification error: certificate has expired
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: C93AF5BA9DDC82FE5B275386EC18FAB736EDEC7071E3033D23C03E8B9389B5F5
Session-ID-ctx:
Master-Key: 7A11F3F97098E63CE02401342EA985ED66CED02350E56E2698063244940D89046508CB16A5ED2861CA1DBDA066B38053
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 95 79 9e a5 30 de f4 26-4f a9 f0 af d4 68 19 93 .y..0..&O....h..
0010 - d2 a3 a7 19 5f bc a9 35-d9 ff 5d 57 89 a8 e9 c4 ...._..5..]W....
0020 - 2c 06 48 cd a6 8d 76 64-e2 84 59 86 38 97 e3 77 ,.H...vd..Y.8..w
0030 - 14 6e 19 e0 9b 88 d5 78-89 97 a0 37 2c ae bd ee .n.....x...7,...
0040 - ac f3 54 9a cf b1 00 6e-ed e7 01 a4 1c e5 ae 10 ..T....n........
0050 - 6a c8 7c 5a 96 fd 40 ab-1b f9 ac 3e 89 0e b7 9c j.|Z..@....>....
0060 - a0 f6 3f ff 14 30 12 e0-e1 bc de 47 6b 6a f9 f5 ..?..0.....Gkj..
0070 - 22 27 a6 85 00 af 68 b4-17 81 00 bf 37 b0 05 71 "'....h.....7..q
0080 - af 47 46 97 fc 6c 8a ef-c6 f2 00 74 60 9a 6d 58 .GF..l.....t`.mX
0090 - db b2 b2 20 dd 11 65 34-42 e2 1a 88 fc db 13 70 ... ..e4B......p
00a0 - 2b cf 26 37 0c d0 5a 6f-00 23 1d 38 4b 6b 3c 58 +.&7..Zo.#.8Kk<X
00b0 - f4 56 6e 04 b8 b5 3c ae-ed cb 3e e7 ff bf b4 3e .Vn...<...>....>
Start Time: 1679498365
Timeout : 7200 (sec)
Verify return code: 10 (certificate has expired)
Extended master secret: no
---
DONE
➜ ~Relevant output parts, in order (and separated by —):
Verify return code: 10 (certificate has expired))Another useful OpenSSL command is to check if a given certificate is valid by itself, if you have already downloaded it (either via browser or by copying the certificates shown by the openssl s_client output into a file). The command is openssl verify cert.pem, and you can also use the options -CAfile root.pem to add a certificate as temporarily trusted and -untrusted intermediate.pem to supply a certificate that has a pending validation. The -CAfile option can also be used in the openssl s_client -connect command.
Nmap is a general-purpose tool for network exploration, such as portscanning. Over the years it grew to get more functionality, and one that might be relevant is to list all the ciphers that the remote server accepts for TLS. To do that, run the command
The output will be something like:
Nmap output
➜ ~ nmap --script ssl-enum-ciphers -p 443 badssl.com
Starting Nmap 7.93 ( https://nmap.org ) at 2023-03-22 16:50 WET
Nmap scan report for badssl.com (104.154.89.105)
Host is up (0.12s latency).
rDNS record for 104.154.89.105: 105.89.154.104.bc.googleusercontent.com
PORT STATE SERVICE
443/tcp open https
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 2048) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A
| TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256r1) - C
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 2048) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 2048) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 2048) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| warnings:
| 64-bit block cipher 3DES vulnerable to SWEET32 attack
| TLSv1.1:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 2048) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A
| TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256r1) - C
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 2048) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 2048) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 2048) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| warnings:
| 64-bit block cipher 3DES vulnerable to SWEET32 attack
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A
| TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 2048) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 2048) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 2048) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (dh 2048) - A
| TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A
| TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp256r1) - C
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C
| TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA (dh 2048) - A
| TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 2048) - A
| TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA (dh 2048) - A
| TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: server
| warnings:
| 64-bit block cipher 3DES vulnerable to SWEET32 attack
|_ least strength: C
Nmap done: 1 IP address (1 host up) scanned in 26.81 seconds
➜ ~This can be useful to check if the TLS versions on the local client side and the remote server endpoint are compatible - if there are no ciphers in common, then the TLS connection will never be established correctly.
One of the problems with diagnosing certificate problems is that any wrong field can invalidate the certificate, and multiple failures can happen. The first step should be either trying to open the website with a browser, and/or submit it to TLS checker websites. To simulate these errors, you can use badssl.com and test the above-mentioned tools to check how they would report these problems.
This one is the easiest one to diagnose with generic tools - double check the Canonical Name, as well as the Alternative Names of the certificate.
This happens when the “Not Before“ or “Not After“ dates are not being respected - either because we already passed the “Not After“ date (most common scenario), or we are not yet at the “Not Before“ date (rare, can happen). To check if this is an issue, you can use openssl, and in particular openssl x509 -noout -dates - this command will parse any certificate and show the notBefore and notAfter fields. Example:
➜ ~ </dev/null openssl s_client -connect expired.badssl.com:443 -tls1_2 | openssl x509 -noout -dates
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.badssl.com
verify error:num=10:certificate has expired
notAfter=Apr 12 23:59:59 2015 GMT
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.badssl.com
notAfter=Apr 12 23:59:59 2015 GMT
verify return:1
DONE
notBefore=Apr 9 00:00:00 2015 GMT
notAfter=Apr 12 23:59:59 2015 GMTPart of the certificate chain might have problems. Typically the problem is a missing intermediate certificate, but it is possible to have an untrusted root certificate.
The servers should send the Certificate Chain to the clients, as per RFC 5246 (link: https://www.rfc-editor.org/rfc/rfc5246#section-7.4.2, search for “certificate_list“), and not just the leaf certificate, but sometimes people forget this and send only the leaf, which results in a broken certificate chain. To check if this is the issue, retrieve the certificate from the endpoint and check if it has all the necessary certificates (the full chain except the root certificate, which may be omitted). You can also use the openssl verify command, and supply the missing intermediate certificate as -untrusted (typically browser can retrieve the intermediate certificate from other sources if it is missing, so check there and look for an export button). To fix this specific issue, the solution is to concatenate the missing intermediate certificates after the leaf certificate.
The root certificate that was used to sign the leaf certificate might not be present in the set of trusted certificates of our systems. The set of globally trusted certificates is typically stable and doesn’t change often; this happens more when someone creates a self-signed certificate for local testing purposes (by manually trusting the certificate in their own machine), and then tries to use that certificate in the public Internet - it works as long as only people who manually added that certificate needs to access the endpoint. To detect this, look into the O (organisation) or OU (organisational unit) fields and check if they point to an established corporation in the Web Certificates area. In self-signed certificates for development either these fields are not filed, or the fields are either clearly pointing to who created it, or they are otherwise filled with nonsense. To check if this is the only problem, you can use the -CAfile flag in the openssl s_client or the openssl verify command.
The solution to any of the above-mentioned problems is either by creating new certificates, amending the existing one (eg: in the “Missing intermediate certificate“ case), or changing web server configurations.
There are situations where adding a certificate to the set of Trusted Certificates would “solve” the problem (as it is easy to do in Postman, so it “works in Postman“). However, trusting random client certificates is not a viable, nor secure, nor a long-term solution for these issues, so it is better to fix the problem at the source.