FTP over HTTP
FTP over HTTP is the name used to refer to a specific type of HTTP traffic between a web browser and an explicitly configured proxy. For the most part, it is like any other HTTP request.The distinguishing factor is that the requested resource resides on an FTP server rather than an HTTP server. Correspondingly, this means an FTP over HTTP request will contain a URL prefixed with " ftp:// " instead of " http:// " and the HTTP "Host" header value will include port 21(instead of no port number at all, in which case port 80 is assumed).
An example of an initial FTP over HTTP request from a client to the Secure Web Gateway:
GET ftp://speedtest.tele2.net/HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0)
Host: speedtest.tele2.net:21
Proxy-Connection: Keep-Alive
When the Secure Web Gateway receives this type of request, it recognizes that the requested resource resides on an FTP server (due to the presence of "ftp://" and port 21). The Secure Web Gateway then uses native FTP to retrieve the fi le from the remote FTP server, on behalf of the client. FTP response traffic is then "translated" by the Gateway into HTTP before passing it back to the client. Sometimes this requires the Gateway to generate an HTML page so the client can display the results in the web browser. This translation is done for data such as an FTP directory listing:
How Does Secure Web Gateway Handle It?
The Secure Web Gateway handles FTP over HTTP just like it would any other HTTP communication with the client computer. This means the traffic should be sent to the Secure Web Gateway's HTTP proxy port (default = 9090). For FTP over HTTP, there is no need to enable, configure or use the Secure Web Gateway's FTP proxy port (default = 2121), which is only used for native FTP protocol sent by a client to the Secure Web Gateway. When the Secure Web Gateway receives FTP over HTTP communication, it then does native FTP between itself and the FTP server, sending commands just like a native FTP client.
Why/When Would I Use It?
FTP over HTTP has advantages and disadvantages. It can be used to retrieve files from an FTP server using a web browser rather than setting up and configuring an FTP client such as Filezilla. As explained above, this also means you do not have to configure or enable the Secure Web Gateway's FTP proxy port. The major disadvantage of FTP over HTTP is that it does not allow users to upload files. This requires the use of native FTP, an FTP client program on the client computer and the Secure Web Gateway's FTP proxy port, if you choose to send the traffic through the Gateway. Another disadvantage of FTP over HTTP is that different web browsers have different behaviors, quirks and bugs. More discussion of these below.
What Does it Look Like?
Below are two images of the same packet capture with different filters applied. The capture was taken on the Secure Web Gateway while an FTP over HTTP session was passing through it. You will see the following devices communicating:
Client computer: 10.10.80.1
Secure Web Gateway : 10.10.80.55 (with HTTP Proxy Port 9090)
FTP Server: 10.10.80.200
The first image is of the connection between the client computer and the Secure Web Gateway. You can see that Wire shark interprets this as HTTP traffic, which it is:
The second image shows communication between the Secure Web Gateway and the FTP server. The Gateway recognized that the requested resource resides on an FTP server, thus uses FTP protocol to retrieve data from the FTP server. Here, active FTP was used, thus there are two TCP connections between the Gateway and the FTP server. The "control channel" is used for sending commands back and forth between the Gateway and FTP server (highlighted in pink).The "data" channel is used to send fi les from the FTP server to the Secure Web Gateway (highlighted in purple):
Configurable Options on the Secure Web Gateway
There are two items on the Secure Web Gateway that can be configured in relation to the usage of FTP over HTTP:
Default FTP Username and Password
If the Gateway receives an FTP over HTTP request that does not contain FTP user credentials, the Gateway will send its pre-configured anonymous user credentials to the FTP server when attempting to login to the FTP server. You can change this username and password pair if you like. The settings can be found here: Configuration > Appliances > [CHOOSE_APPLIANCE] > Proxies (HTTP(S), FTP, ICAP and IM)>> section "HTTP Proxy"
Web Gateway Log-In Page
What is it?
This feature adds a JavaScript to the Gateway's "401 - Authentication Required" block page. When received by a web browser, it offers the user the option of inputting a username and password. The JavaScript then re-sends the request via FTP over HTTP and includes the credentials.
How do I get it?
If you have done a fresh installation of Skyhigh Secure Web Gateway v7.2 or newer, then you should already have the Log-In Page. If you upgraded from a version prior to v7.2, you probably do not have it. If you do not have the log-in page, you can add it yourself. To do so, find the template for the "Authentication Required" block page and replace its code with the contents of the fi le "MWG_ftp_login_page.txt" attached at the bottom of this article:
- To find the template go to: Policy > Settings > Actions > Block, select "(Default)"
- In the right window-pane, click "Edit..." next to the "Collection" drop-down box.
- The "Template Editor" will appear. Find "Authentication Required" and expand the tree until you find the "html" file. Select it.
- In the right window-pane, the "HTML Editor", replace all the contents with the contents of the attached fi le: "MWG_ftp_login_page.txt"
Overview of Common Web Browser Issues
Currently, Firefox is the only web browser we have tested that does not require special attention from the user when doing FTP over HTTP. There are four major issues in other browsers, present in different combinations depending on the browser:
- Some browsers require you take extra steps to log into an FTP server when a username and password other than "anonymous" is required. There are three ways to work around this:
- - Prepend credentials to the URL hostname like this: ftp://Username:Password@ftpserver.com
- - Use the Gateway's Log-In Page to collect user credentials (see information above).
- - Some browsers will prompt you for credentials if anonymous FTP is not allowed.
- Some browsers mis-encode FTP usernames and passwords containing special characters, rendering them useless, causing log-in to fail. There is no work around for this issue. These browsers are not recommended for FTP over HTTP when credentials are required.
- Some web browsers cannot do proxy authentication for FTP over HTTP traffic.Instead of sending the user's domain credentials in response to the Gateway's "407- Proxy Authentication Required" message, the browser simply displays the HTML page that was sent with the 407 code. To work around this issue, you cannot enforce proxy authentication for FTP over HTTP traffic from these browsers. To bypass these browsers from proxy authentication, write a rule that recognizes the user-agent and protocol (see below for more detail).
- Some browsers do not respect system proxy settings when protocol "ftp://" is entered in their address bar. Instead of sending an FTP over HTTP request to the proxy, they send native FTP traffic directly to the FTP server. There is no workaround for this issue. These browsers are not recommended for FTP over HTTP traffic.
Known Issues with Specific Browsers
Internet Explorer
To log into non-anonymous FTP servers, the user must send their credentials via the Gateway's Log-In page (described above) or add their username and password to the URL in this format: ftp://USERNAME:PASSWORD@servername.com
Internet Explorer cannot log into FTP servers if the username or password contain special characters, especially the “@” symbol, when doing FTP over HTTP. These characters are mis-encoded when sent by the browser. Therefore, the credentials are invalid when received and used by the Secure Web Gateway and FTP server.
Firefox
No known issues doing FTP over HTTP.
Chrome
- Chrome cannot access FTP servers that require credentials, other than the anonymous user, when doing FTP over HTTP. It does not send credentials as part of the URL, even if you add them in the address bar, or by any other method. Therefore,non-anonymous FTP over HTTP access is not possible.
- Chrome cannot do proxy authentication for FTP over HTTP traffic. Instead, it displays the Block Page that is sent by Secure Web Gateway with the 407 HTTP status code. To send Chrome FTP over HTTP traffic through the Secure Web Gateway, it must be exempted from proxy authentication. You could do so with a rule like this:
Header.Get("User-Agent") matches *Chrome* AND URL.Protocol equals "ftp" -->Stop Rule Set
Opera
Opera cannot do proxy authentication for FTP over HTTP traffic. It prompts the user for credentials when it receives the 407 HTTP status code from the Secure Web Gateway, but once they are inputted, the browser displays the Block Page sent by Secure Web Gateway with the 407 HTTP status code rather than retrieving FTP data from the Gateway. To send Opera FTP over HTTP traffic through the Secure Web Gateway, it must be exempted from proxy authentication. You could do so with a rule like this:
Header.Get("User-Agent") matches *Opera* AND URL.Protocol equals "ftp" --> StopRule Set
Safari
Safari cannot do FTP over HTTP. It does not respect system proxy settings when the protocol in the address bar is set to "ftp://". Instead, it sends native FTP traffic directly to the FTP server, bypassing the configured proxy.