FTP Timeouts on Large Files

Here is a description of the problem from the FileZilla documentation at http://wiki.filezilla-project.org/Network_Configuration#Timeouts_on_large_files

A few possible workarounds are described below.

Timeouts on large files

If you can transfer small files without any issues, but transfers of larger files end with a timeout, a broken router and/or firewall exists between the client and the server and is causing a problem.

As mentioned above, FTP uses two TCP connections: a control connection to submit commands and receive replies, and a data connection for actual file transfers. It is the nature of FTP that during a transfer the control connection stays completely idle.

The TCP specifications do not set a limit on the amount of time a connection can stay idle. Unless explicitly closed, a connection is assumed to remain alive indefinitely. However, many routers and firewalls automatically close idle connections after a certain period of time. Worse, they often don’t notify the user, but just silently drop the connection. For FTP, this means that during a long transfer the control connection can get dropped because it is detected as idle, but neither client nor server are notified. So when all data has been transferred, the server assumes the control connection is alive and it sends the transfer confirmation reply. Likewise, the client thinks the control connection is alive and it waits for the reply from the server. But since the control connection got dropped without notification, the reply never arrives and eventually the connection will timeout.

In an attempt to solve this problem, the TCP specifications include a way to send keep-alive packets on otherwise idle TCP connections, to tell all involved parties that the connection is still alive and needed. However, the TCP specifications also make it very clear that these keep-alive packets should not be sent more often than once every two hours. Therefore, with added tolerance for network latency, connections can stay idle for up to 2 hours and 4 minutes.

However, many routers and firewalls drop connections that have been idle for less than 2 hours and 4 minutes. This violates the TCP specifications (RFC 5382 makes this especially clear). In other words, all routers and firewalls that are dropping idle connections too early cannot be used for long FTP transfers. Unfortunately manufacturers of consumer-grade router and firewall vendors do not care about specifications … all they care about is getting your money (and only deliver barely working lowest quality junk).

To solve this problem, you need to uninstall affected firewalls and replace faulty routers with better-quality ones.

Some Possible Workarounds

1) Set the SoSndBuf and SoRcvBuf properties equal to 131072, which is the default used by FileZilla. If this improves the transfer rate, then it may be that the transfer takes a shorter amount of time and the timeout of whatever is interfering is not exceeded.

2) Set the SkipFinalReply property = true (or 1 if using the ActiveX). This will cause the FTP2 object to avoid trying to get the final control-channel reply. However, you should (A) close the connection and re-connect because if the final control-channel reply did arrive but remained unread, then the FTP client and server will be out-of-sync in terms of commands and replies. (In other words, the next FTP command sent on the control channel will get the previous reply.) (B) When using SkipFinalReply, write code to verify the size of the remote file after the transfer.