Mutual certificates over SSL/TLS is a standard, “classical” and very popular way of securing REST APIs – but what about challenges in setting up this security model and how to address them properly?
Introduction
Nowadays, there are many known and broadly used API security models in the IT industry to, on the one side, ensure proper authentication and authorization of consumers before allowing them to access an API / Service, on the next side, to provide encryption and signatures to guarantee confidentiality, message integrity and proof of the caller’s identity.
Over the last years, new standards and best practices have emerged (for example: OAuth/OpenID Connect) to address security concerns for RESTful APIs. “Classic“ security models are still being actively used in the industry, simply because, over the decades of being around, they have proved to be “good enough” for those scenarios that require simplicity, interoperability based on the standards, and acceptable security levels at the same time. Basic Authentication and Mutual SSL X.509 Certificates are probably the oldest and most widely known industry standard security models, while mutual X.509 Certificates model is the strongest of these two because:
Client X.509 certificate identity adds an additional level of asymmetrical cryptography to the standard SSL/TLS channel. Basic Authentication or API Keys (commonly used nowadays) rely on a knowledge of a shared “secret”, which the API client sends as its identity over the SSL/TLS channel.
It uses long security keys (today 2048 bits is the minimum industry standard key length).
It has a built-in mechanism to deny expired and revoked certificates.
It is best fitted for application-to-application integration scenarios, when applications (not end users) must present their identity in a strong form of X.509 certificates.
Even though the mutual certificates security model over SSL/TLS sounds straight-forward, and it has been around for quite some time, it also has few operational and configurational challenges, especially when used to protect APIs. This article provides a detailed description of the most common challenges, lists typical requirements for X.509 certificates used in a mutual authentication, and describes few useful troubleshooting steps and tools.
Mutual X.509 Certificates Authentication
Mutual authentication in general (without any mentioning of a specific type of authenticated identity) means that:
The API (service) must authenticate itself to the client application (service must present its identity to the client). This authentication gives the client the confidence, that it is communicating with the API it actually intends to communicate with, and not with some malicious man-in-the-middle service.
Client must authenticate itself to an API (client must present its identity to an API). This authentication gives the API the confidence, that the client is who it claims to be.
In case of a mutual certificates authentication over SSL/TLS, both client application and API present their identities in a form of X.509 certificates. As part of the SSL/TLS protocol, client and service initiate a special protocol handshake (they exchange special protocol messages) before the actual REST API messages are sent / received. During this handshake, client and service exchange and validate each other’s certificates. This SSL/TLS handshake with certificates exchange and validation is what actually constitutes a Mutual X.509 Certificates authentication. Note, that the authorization aspect is always outside of the scope of the SSL/TLS protocol. For example, just because a client provides a valid authenticated certificate to an API server - that doesn’t necessarily mean that the client is authorized to make a call to the API.
Somewhat simplified, the sequence of SSL events during the Mutual SSL Authentication is depicted below.
A Mutual X.509 Certificates authentication fails during SSL/TLS handshake when either:
There is an SSL server certificate validation failure - implementation and configuration of the SSL protocol on the client application side fails to validate the received Service (API) certificate (service certificate is often called SSL server certificate).
There is a client certificate validation failure - implementation and configuration of the SSL protocol on the Service (API) side fails to validate the received client certificate.
Let’s take a look at the most common reasons for failures 1. and 2. above.
SSL server certificate validation failures
Certificate has expired.
All certificates have expiration date. If the SSL server certificate is expired, then the client application will not accept the server certificate and the API call will fail.
Certificate is not trusted.
An SSL server certificate received by the client application must be trusted by the same client application. Typically, a trusted certificate is the one signed by the Certification Authority’s (CA) certificate and possibly by a chain of Intermediate Certification Authorities (ICA). These CA and ICA certificates can be configured with the local machine, or a client application itself, as Trusted Root and Trusted Intermediary Certification Authorities in the local Certificate Store. For example, Windows OS has two types of Certificate Stores, one for the applications running under a specific Windows User Account (called Current User Store), and another for all applications running on the local machine (called Local Computer Store). These stores are pre-populated with “known” public Trusted Authorities, but you can also add your own authorities to these stores (you can also remove existing Authorities). Note that in the case of a Windows OS, any given application will check the trust against both stores, and if the trust is available via either, the overall trust is considered to be established.
Also, on Windows OS the trust may be established via stores other than the CA and ICA, but having the signing Authority certificate placed in these stores guarantee the end trust. Below is the screenshot of the Windows Certificates Stores.
No access to Certificate Revocation List.
Some certificates may have a Certificate Revocation List (CRL Distribution Points) property. This property specifies a public address offered by the Certificate Authority that issued the SSL certificate. This address may be used by the client application to retrieve the list of certificates revoked by the CA (CA may revoke issued certificates even before they have officially expired).
CRL Distribution Points property of a certificate viewed in Windows, and URL of a public CA address to the CRL document.
Some SSL stacks (for example, Windows SSL default implementation) will attempt to retrieve the CRL list from this CA public address; should the client application not have physical network access to this address, the CRL list retrieval will fail and as a result, the entire validation of the SSL server certificate will fail. This makes for an interesting behavior in Windows. If a certificate has a CRL property - it must be used, otherwise validation fails. But if certificate does not have the CRL property, no CRL validation will be executed.
Certificate Common Name mismatch.
The Host name of the URL the client application uses to make an API call, does not match the SSL server certificate’s Common Name. This requirement is very often overlooked by API developers and API testers.
When a client application makes an API call, it uses a hostname address that can be a DNS name, a computer name or an IP address. For example:
https://www.nevatech.com:[port]/[PathAndQuery], or
https://myapiserver:[port]/[PathAndQuery], or
https://192.168.0.1:[port]/[PathAndQuery]
All three forms of the request URL above can be valid from the networking perspective. For example, a client application can create either form of a request URL and this request will physically reach the same server and the same API. But because these are HTTPS requests, the SSL stack implementation, and its configuration on the client side, will check that the hostname highlighted in the examples above, matches Common Name of the SSL server certificate received from the API service. Without a match, the client application assumes a man-in-the-middle attack, and API call fails. A Common Name can be easily seen from the properties of the received SSL certificate (see screenshot below).
In the examples above, only requests that use www.nevatech.com as a hostname will have a chance to succeed, because the SSL certificate is issued to www.nevatech.com (but not to myapiserver or 192.168.0.1). Note, that this is a client-side validation of the SSL server certificate, which means that the API/service has no control over which hostname a client application will use, and therefore has no means to control success or failure of this this particular validation criteria. A common name may contain a wildcard, for example *.nevatech.com, in which case any client request ending with nevatech.com as a hostname will match the certificate Common Name.
The four most common validation criteria above are not necessarily all-inclusive (there may be others), but in the majority of real-life scenarios, they are sufficient to ensure that SSL server certificates are validated successfully by API client applications.
Very important to note, is that while the four conditions above are typically default and required, they can still be overwritten and effectively removed from the validation process. How (and if) this can be done, depends on the SSL stack implementation and its configuration with the client application. For example, the Windows SSL stack can be configured by a .NET client application to remove any or all of the requirements above. Programmatic or configured removal of these validations is not recommended in production environments, because it seriously elevates security risks. At the same time, removing some or all of them can be used in development, testing and debugging scenarios, to avoid having to deal with real commercial SSL certificates.
Client certificate validation failures
Validation criteria 1, 2 and 3 above for SSL server certificates are equally applicable to the client certificate validations, while Criteria 4 is not; i.e., a client certificate’s Common Name can be anything. For example, your client certificate may contain the Common Name My Client Certificate (which is not even a valid hostname, because it contains spaces).
At the same time, client certificates may have their own requirements:
Certificate Authentication Purpose or Enhanced Key Usage property.
This property specifies the purpose of the certificate’s usage and is assigned to a certificate by the Certificate Authority when issued.
The screenshot above shows the properties of two certificates issued by CA. The left certificate has only one purpose, Server Authentication, while the right certificate has both Server and Client Authentication enabled.
Very often, organizations acquire SSL certificates from CAs with the intent to use them as client certificates for their applications. If they do not explicitly request the CA to issue a certificate for Client Authentication, it is very likely it will be issued by the CA without it. In this case, the certificate may not be validated by an API/service, simply because the service expects a client certificate with Authentication enabled (it is OK to have a certificate with multiple purposes as shown for the right certificate above). Whether this strict requirement is enforced by the API/service, depends on the SSL stack implementation and the API configuration. For example, the Windows SSL stack by default checks this purpose. If the API/service does not see this purpose enabled in the client certificate, it will fail the client certificate validation. As a result, the entire API call will fail.
Just as with the SSL server certificates validation, the client certificate validation requirements may be removed by configuring the SSL stack and API application appropriately. How this is done (if it can be done), always depends on a specific SSL stack used by the API application. For example, removing this requirement for Client Authentication Purpose on Windows machines can be a challenging task. Some time ago, I found such a tool developed by Steve Johnson, https://www.stevestechspot.com . It can be downloaded from these links below:
https://www.stevestechspot.com/downloads/httpconfig.zip
https://www.stevestechspot.com/downloads/httpconfig_source.zip
Conclusion
When configuring Mutual SSL security for your APIs, or when calling somebody else’s API using this security model, pay attention to the 4 typical requirements listed in this article. Always take into consideration, that the SSL stack configuration on the API side may be quite different from the SSL stack used by API’s client applications.