HTTPS in transparent deployements and how SNI can help
Introduction
Be sure to read through before reading through this article. As in that previous article, this article covers an environment where HTTPS traffic will flow through the Web Gateway, yet no SSL inspection is enabled.
As in the other article, if you expect the Web Gateway to do anything other than just allow HTTPS traffic (block, redirect, etc.), the SSL client context must be enabled. However, in transparent deployments, there are common issues that must be handled properly:
- Common name mismatch on the certificate presented to the client, due to IP address populating common name (and not hostname)
- Issues blocking or allowlisting sites by their hostname (i.e. unable to properly allowlist 'https://www.google.com'). By default you need to use the IP address of the destination and not the hostname.
Read below to find out about why these two above issues are present, and how to properly handle these on the Web Gateway!
HTTPS in explicit proxy deployments
Review for a moment the below screenshot, depicting a client SSL request that the Web Gateway will 'interact' with:
Above, the Web Gateway 'sees' a CONNECT request for the host 'community...com', coming from the client. While not initially obvious, this host name is ONLY available to the Web Gateway in a direct (explicit) proxy setup. In a direct/explicit proxy deployment, the browser is 'aware' of the proxy. As a result, it sends a request over that contains the entire destination host name.
As mentioned in the previous document, in the event where the Web Gateway is going to block the SSL request, we must provide a valid certificate back to the client. This ensures we can issue our block properly, inside of the SSL tunnel.
HTTPS in transparent deployments
However, in a transparent deployment (WCCP, transparent router, etc.) there is a problem - the client is not 'aware' of the upstream proxy. As such, the client will perform its own DNS lookup to resolve the requested host to an IP, and then will make a request for that particular IP. When this request arrives at the Web Gateway, all the Web Gateway 'knows' about the request is the destination IP. This is illustrated in the screenshot below:
In the screenshot above, the Web Gateway must still issue a certificate.
Even though the client made a request for 'contentsecurity...com'...
... all that the Web Gateway sees is a request for the IP address (194.77.103.20). As such, the Web Gateway will issue the following certificate:
The browser will warn users about the certificate error, since the 'Issued to:' field doesn't match what the user typed in the browser.
This can be seen in Wireshark as well:
(Wireshark info above: Client IP 10.10.89.32, destination 194.77.103.20)
Proper Certificate creation for HTTPS requests in transparent deployments
There are two mechanisms that the Web Gateway can use to determine the requested hostname. - SNI (server name indication) and the 'Fix Hostname' rule.
Server Name Indication
Server Name Indication (SNI) is an extension added to the TLS encryption protocol, allowing the clients to 'indicate' what host they are attempting to connect to. (Ultimately, this allows web servers to virtually host multiple domains, and thus present multiple certificates from the same IP). Read a bit more here - http://en.wikipedia.org/wiki/Server_Name_Indication
The 'server name' value can actually be seen in a Wireshark capture, as below:
(In the above capture, the client IP is 10.10.89.1, and the destination IP is 194.77.103.20)
If available, the Web Gateway will use the 'server name' value to determine the requested host, and issue a certificate the client will not complain about:
However, in many cases, this value is not available, or is incorrect. It is 100% up to the client browser and operating system to decide if it will send SNI. THE Web Gateway HAS NO ABILITY TO INFLUENCE THIS. For example, no Windows XP browser supports SNI. If SNI is sent, the Web Gateway will automatically use that value. No additional configuration is needed.
Here's an example of an SNI value that is provided, but actually invalid:
(in above screenshot, client is 10.10.65.15, and external server is 74.200.15.166)
In the event that the SNI value is not available or is invalid, we can try another method to obtain the destination host name - the "Fix Hostname" rule.
NOTE: SNI is read from the client before the request hits the rule engine. So if the SNI is present and valid, the Web Gateway will use it for the hostname before any rules in the policy are applied.
Fix Hostname - rule
The 'Fix Hostname'-rule method is performed by evaluating a property that forces the Web Gateway to connect to the destination server. Once there is a TCP connection to that destination host, theWeb Gateway can send a 'Client Hello', on behalf of the client, and then will receive the full certificate from the destination server. Then, Web Gateway can grab the common name from that server, and correct the hostname in the cert given to the actual client.
This is illustrated below:
The image above illustrates the process Web Gateway will use to attempt to gather that host name. This process can be seen in Wireshark as well:
(client IP 10.10.89.32, destination server 194.77.103.20, and Web Gateway IP is 10.10.89.117)
In the above Wireshark capture, we can see the "Client Hello" from the client being sent to the external server in packet 4337.Then, shortly after, we can see the Web Gateway obtain the certificate. Finally, the Web Gateway can issue the proper certificate back to the client (in packet 4354).
Below, you'll see two screenshots, showing how to create this rule:
Fix Hostname - rule: considerations
Wildcard certificates
If the certificate received from the destination server is a wildcard cert (i.e *.skyhigh.com), the Web Gateway cannot use the wildcarded common name as the hostname for further rule processing. (Note we check for this case in the 'Cert Verify and fix hostname' rule set phase)
Certificate alternative names
If the destination server returns a certificate from a different host than what the client requested, known as an alternative name, this alternative name will be given back to the Web Gateway. Since this name is different than what was requested by the browser, the browser will likely throw a certificate error for common name mismatch. See below:
Summary
The Web Gateway has the flexibility to handle a variety of deployment methods. Each deployment method, and Web Gateway functionality, have some considerations. Above, considerations for interacting with SSL traffic in a transparent deployment method are discussed. Note these considerations are applicable for nearly all transparent deployment methods.
Fix Hostname - rule: considerations
Wildcard certificates. If the certificate received from the destination server is a wildcard cert (i.e *.skyhigh.com), the Web Gateway cannot use the wildcarded common name as the hostname for further rule processing. (Note we check for this case in the 'Cert Verify and fix hostname' rule set phase) Certificate alternative namesIf the destination server returns a certificate from a different host
than what the client requested, known as an alternative name, this alternative name will be given back to the Web Gateway. Since this name is different than what was requested by the browser, the browser will likely throw a certificate error for common name mismatch.