Direct Proxy vs. Transparent Deployment

Last updated
Save as PDF

Secure Web Gateway On-Prem allows for a multitude of different deployment methods. Here's a comprehensive overview of the different options as well as the benefits and downsides of each of them.

Introduction

First, we need to define a direct proxy and a transparent deployment.

Direct Proxy — The browser/client is "proxy aware" and will actively send traffic to Secure Web Gateway. A proxy and its options in regards to authentication (as an example) are defined in RFC and commonly supported by browsers and most client apps (like windows media player or java). Secure Web Gateway.offers a direct proxy option as well as a Proxy HA option for redundancy.
Transparent Proxy: — The browser/client is not "proxy aware". The client will try and directly communicate with its destination. The client requests need to be intercepted/redirected on their way to the destination to reach Secure Web Gateway. As the client is not aware of Secure Web Gateway. or its filtering, all correspondance needs to "look like" it is still happening with the original destination server. Functionality that is normally available to proxies (like authentication) cannot be used in their normal fashion. Secure Web Gateway offers WCCP, Transparent Router and Transparent Bridge as transparent deployment options.

A transparent deployment is often chosen mainly because administrators do not want to have to make browser changes. Unfortunately this is a common misconception and in reality changes often do need to be made to browsers, see also the table in the next section. In addition, transparent deployments often add more complexity to the deployment and require more work on the administration side.

Direct Proxy vs. Transparent Deployment Functional Comparison

Function	Direct Proxy	Transparent Deployment
Traffic direction	Proxy settings need to be pushed to each client	No settings on the client needed
Traffic exceptions	Easily added to proxy settings	Has to be done on the network/intercepting device
Special applications that cannot deal with proxies	Will ignore proxy settings and go direct	Might fail when getting intercepted transparently
Authentication	Native proxy authentication supported by browser	Need authentication server for redirecting client to special authentication URL
Promptless authentication	Browser automatically trusts a proxy with promptless credentials	Browser must be configured to trust authentication server to send promptless credentials (browser configuration on the clients)
Authentication sessions	No sessions needed, as each TCP connection is authenticated	Sessions (time-based or using cookies) are needed while third-party cookies must be explicitly allowed on each client (browser configuration on the clients)
SSL scanner	Need to push root CA to each client	Need to push root CA to each client
SSL traffic	Destination host names are seen by the filters	Only destination IP addresses are seen by the filters
DNS	Client only needs to talk to Secure Web Gateway, which is responsible for DNS	Client and Secure Web Gateway both need to complete their own DNS resolution
Troubleshooting	Easy to rule out Secure Web Gateway by temporarily disabling proxy on the client	Network changes required to go direct/without proxy
Troubleshooting	More control over traffic flow (which proxy is being used)	Harder to determine which proxy is used by a particular client

Deploying a Direct Proxy

For a direct proxy deployment, you must to make the clients/browsers aware of the proxy (its IP address and port). Here are a few common ways to roll these settings out in your network.

Proxy Settings

One way to get the proxy settings out is to modify these settings on the browser manually.

With most browsers, you can specify the proxy IP address and port. Many browsers also let you specify separate ports for different protocols, such as FTP, HTTP, HTTPS, and others.

For a Secure Web Gateway proxy, you must select the Same proxy for all protocols option. This method is very static and does not provide many options for changes or exceptions.

Proxy.pac/WPAD.dat

This is one of the more common ways. Administrators hand out proxy settings for browsers in a file.

The proxy.pac file contains all your proxy settings..This file allows for lots of configuration options. It also allows you to create rules, for example, for bypassing particular websites. .

Secure Web Gateway can host this file on the file server. There are also lots of resources available online if you need help with creating your proxy.pac file. See, for.example, http://findproxyforurl.com.

GPO Push

Another very common tool for direct proxy environments is the Group Policy Object (GPO) editor.

To be able to use this tool, you must be using Active Directory (AD). It can either be used to push out a proxy pac file to each browser within your network or to configure and enforce manually created proxy settings on each browser within your network.

GPO can also help you with some of the changes needed for transparent deployments, see further below.

Authentication

Direct proxy deployments need very few or no considerations regarding authentication. Transparent deployments are a lot more complex when it comes to authentication.

Here are some considerations for setting up authentication.

Promptless Authentication

For authentication to be promptless in a transparent environment (using Windows credentials without prompting the user to enter them), you must add the Secure Web Gateway IP address or host name to the trusted sites list of your browser.

This must be done because Secure Web Gateway redirects you to its internal authentication server, which asks the browser for credentials.The browser requires explicit trust to pass the credentials on without user interaction.

Failure to trust the Secure Web Gateway IP address or host name will cause the browser to prompt the user to enter credentials for every session.

Authentication Sessions

In a transparent deployment, authentication must be "session based". This means you either have a session bound to your IP address for a certain amount of time, also known as time/IP based, or you need a cookie that is valid for the length of your browser session.

This can cause issues in environments where you either have shared workstations (user A's session is still valid when user B starts using the workstation) or where you have multiple users with the same IP address (citrix/terminal servers, NATed environments, and so on).

In these environments, you will have to rely on cookie authentication.

Allowing Third-Party Cookies

If you are planning to use cookie authentication on Secure Web Gateway, this will be your method. With many browsers, you will have to enable the security option that allows the browser to accept third-party cookies.

Without enabling this method, you would most likely see malformed and incomplete web pages.

SSL Traffic

Here is some guidance on how to handle web traffic on connections that are secured under an SSL protocol in a transparent deployment.

Trusting the Secure Web Gateway Root CA

If you are using the HTTPS Scanning rule set, sometimes also referred to by its old name as SSL Scanner rule set, on Secure Web Gateway, we recommend that you import the Secure Web Gateway root CA into the browser as a trusted certificate authority.

Otherwise, you will be prompted to trust the certificate each time you browse to an SSL-secured site. This applies to both direct and transparent deployments.

Transparent SSL

When a request for access to an HTTPS-secured site comes in to Secure Web Gateway in a transparent environment, it will present an IP address instead of a domain name. {swebg} Gateway will then use this IP address as the domain name.

This can cause problems when trying to allow or block a domain name or to search for it in an access log.

To get the proper domain name, you can add a rule that fixes this. The rule will essentially take the common name from the certificate that is used in the communication and provide it as a value for the URL.Host property, which is then filled with this name.

DNS

In a transparent environment, DNS lookups are performed on the client, as it is the client that uses the IP address of the requested web server to connect to it. When a request is redirected to Secure Web Gateway, a DNS lookup is performed there as well before the request is eventually forwarded to the web server.

This can lead to issues, for example, when you get different results from the DNS server the client uses and from the one Secure Web Gateway uses. It can get even worse if the client is able to resolve a particular host name through DNS, but Secure Web Gateway is not. We strongly recommended making sure that Secure Web Gateway and its clients use the same DNS server.

Troubleshooting

Here are some things to take into consideration when troubleshooting issues that may arise with each of the different deployment methods.

Request Line in TCP Dumps

When troubleshooting, you will often need to look at a TCP dump and filter for HTTP requests. When looking at a TCP dump or packet capture, you will notice differences in what, for example, a GET request looks like. In both of the examples below, filtering has been performed for htttp.request.

In a direct proxy deployment, you will see what is known as the absolute URL in the request line. It is much easier to follow, as shown in this screenshot:
In a transparent deployment, you will see the relative URL in the request line, which is more difficult to follow. So, the Host header will show the domain name, and the request line will only show the relative path to the domain, as shown in this screenshot:

SSL Connection Differences

In a transparent environment, HTTPS requests will come through to Secure Web Gateway with an IP address instead of a domain name. This can make things difficult when trying to filter a TCP dump for a request or even when looking at an access log. In a direct proxy deployment this is different, as there is a CONNECT request first, which contains the domain name.

Here is an example of an HTTPS request in a TCP dump from a direct proxy deployment, where the CONNECT request is clearly seen and easy to follow:
Here is an example of an HTTPS request in a TCP dump from a transparent deployment. It is less easy to follow, as there is no CONNECT request. What you see here is how the client uses the IP address of the requested website to connect to it:

Tracing a Client Request in a TCP Dump

When tracing a client request in TCP dump, you will see how the way the client connects differs depending on the type of deployment.

In a direct proxy deployment, you can easily see the client with its IP address connecting to Secure Web Gateway under its IP address over the proxy port. This is not difficult to troubleshoot, see this example:
In a transparent deployment, tracing a client request in a TCP dump can be a problem. The client uses the IP address of the requested web server to connect to it of the web server, rather than the Secure Web Gateway IP address. Secure Web Gateway eventually forwards the client request to the web server using the IP address of the latter. When responding back to the client, Secure Web Gateway spoofs this web server IP address. This can be confusing when trying to follow web traffic in a TCP dump, see this example:

Support Recommendations and Best Practices

Now that we have discussed the different deployment options and how to troubleshoot issues, let us talk about recommendations and best practices from a support perspective. As you can imagine, support has seen every deployment scenario imaginable over the years.

Before we start, lets be clear that all of the deployment methods mentioned here are fully supported and function as designed. The below should be taken purely as support's experience with the different deployment methods.

At its core, Secure Web Gateway is a proxy. For this reason, the hands-down most suitable deployment method is a combination of direct proxy with a proxy.pac file and a hardware load balancer in front of the instances of Secure Web Gateway in your confíguration. This deployment gives you enterprise-wide flexibility when it comes to change management (easy changes in the proxy.pac file once deployed), high availability and load balancing as well as maintenance and troubleshooting.

If you decide to go for a transparent deployment method, the best experience we have is using it with WCCP.

WCCP is a protocol that has many enterprise features already built in. So, we often see it used in addition to a direct proxy deployment. For example, to cover a guest wireless network or to have a fall back/catch all for devices not covered by the direct proxy settings.
If you decide to set up your transparent deployment in Transparent Router or Transparent Bridge mode, the most common issue we see in support is with exceptions.

Everyone needs exceptions at some point, due to restrictions of internal resources, applications that are not standard-compliant, and so on. In Transparent Router mode, you can "route" traffic around Secure Web Gateway on a logical level.

In Transparent Bridge mode, Secure Web Gateway is in the physical path of the traffic (inline). Then it is almost impossible to get around it. Also, with the Transparent Bridge mode, there is the risk of a complete network outage if no precautions are taken before Secure Web Gateway goes offline.