Did you know…? LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.
Many system administrators find themselves in need of an efficient way to transfer data from one server to another, whether that data is for backups, mirroring, or some other task. Fortunately, there are several tools available for this job: FTP, rsync , scp , and others. This article will cover how these compare and contrast with each other and when it might be best to use them for specific tasks.
FTP vs SSH/SCP Backups/Mirroring
There are many reasons why you would want to use SSH and SCP to copy files. Two of the most important ones are:
- Security. You can secure SSH or SCP with an encrypted connection, preventing snooping and man-in-the-middle attacks on your network.
- Convenience. One of the biggest reasons for using SSH is that it typically requires only a single command (such as ssh host_name mkdir -p /data/location).
For these two reasons, many people tend to utilize FTP only when security is not paramount—for example, in cases where they have remote servers that are not publicly accessible or when transferring data offsite to another company’s servers.
rsync vs Zypper/RPM There are two ways to use rsync for backups:
- Using tar
- Over SSH or SCP using ssh or scp. This is the method this article will cover.
The one major advantage of Zypper/RPM over rsync is that Zypper can work with delta RPMs, whereas rsync cannot process these directly. However, if you are utilizing hard links (which create 1 copy of data that means less space on your disk), then you do not need delta RPMs since each file only takes up as much space as it actually needs. For example, say you had 1GB of data in a file and made 10 copies of it using hard links. That would be 10GB altogether, whereas if you had used delta RPMs, it would be 1GB.
The one major advantage of rsync over Zypper is the ability to resume an interrupted download. Zypper can do this by using its implementation of aria2, which according to their documentation should allow restarting a partial transfer from where it left off. The only other thing I could find on the matter was this mailing list post, but that link shows that no other files were downloaded after resuming compared to before it was stopped, so perhaps the feature is not as useful as intended or as implemented as expected? In any case, for those unfamiliar with aria2, there are plenty of GUI clients available for both Windows and Linux, such as Aria2c, Aria2fe, and Aria2QL .
rsync vs FTP When transferring files over FTP, it is usually best to set up a cron job for automatic transfers so that the data being transferred is not sitting on an external server waiting for you to transfer it. This can be risky if your primary machine fails because then your backups will also be inaccessible until you replace the failed machine—not fun if the backup contains website databases or other important information that need to remain available. You could get around this by using SSH keys for authentication with FTP, but that would only secure the connection while it was communicating data, not when both ends are idle.
An advantage of rsync/SCP over FTP speed since SCP uses parallel uploads and rsync uses a smart way of comparing files. If you have a 20MB file, it will only upload the new/changed parts so the transfer is much quicker than FTP.
rsync vs Zypper/RPM Zypper’s main advantage over using rsync for backups is that Zypper can work with delta RPMs. This makes upgrades much faster because you download only the things that have changed since your last update from /var/cache/zypp/packages, which saves bandwidth and time. Some admins prefer to utilize tools such as svn upgrade or yum update instead to avoid this overhead, but I do not recommend doing this if possible due to security issues (such as a bug in a package being exploited) or if you have a lot of servers utilizing delta RPMs.
rsync vs Zypper/RPM rsync’s main advantage over Zypper is the ability to resume an interrupted transfer. This is mostly relevant for those using using hard links, as rsync has no way of being able to tell which parts have been updated from the old file and thus needs to re-download everything as if it was a brand new transfer. In most cases, this should not be a problem unless your connection goes down or rsync crashes halfway through the transfer (in which case you would probably want to fix whatever caused that anyways).
Conclusion:
Zypper can work with delta RPMs, whereas rsync cannot process these directly. Zypper’s main advantage over using rsync for backups is that Zypper can work with delta RPMs. This makes upgrades much faster because you download only the things that have changed since your last update from /var/cache/zypp/packages, which saves bandwidth and time. Some admins prefer to utilize tools such as svn upgrade or yum update instead to avoid this overhead, but I do not recommend doing this if possible due to security issues (such as a bug in a package being exploited) or if you have a lot of servers utilizing delta RPMs.
