Ignore existing files or update only newer files with rsync

Rsync is a useful command line utility for synchronising files and directories across two different file systems. I recently needed to use rsync to only copy over files that did not already exist at the other end, so this post documents how to do this.

Copying from local to remote

Note that all the examples shown in the post are for copying files from the local computer to a remote server/computer.

Default behavoir

The following command will recursively copy all files from the local filesystem from /var/www to the remote system at 10.1.1.1. Note the following:

1. Any files that do not exist on the remote system are copied over
2. Any that have been updated will be copied over, although note that rsync is extremely efficient in that only the changed parts of files are copied and if the file is exactly the same if it is not copied over at all
3. Any that have been deleted on the local system are deleted on the remote

rsync -raz --progress /var/www 10.1.1.1:/var

Ignore existing files

Use the –ignore-existing flag to prevent files from being copied over that already exist on the remote server. By adding this, we eliminate behaviors 2 and 3 in the list above and all that is done is this:

1. Any files that do not exist on the remote system are copied over

rsync --ignore-existing -raz --progress /var/www 10.1.1.1:/var

Update the remote only if a newer version is on the local filesystem

In the case I was talking about, we didn’t want to overwrite any files at the other end. However you may want to copy files over the have been updated more recently on the local filesystem which is done with the –update flag. The behavior is now like this:

1. Any files that do not exist on the remote system are copied over
2. Any files that exist on both local and remote that have a newer timestamp on the local server are copied over. (Conversely, any files that have an older timestamp are not copied over).

rsync --update -raz --progress /var/www 10.1.1.1:/var

Use –dry-run to see what will be copied

It can be a good idea to test what will be transferred before actually doing it to make sure you aren’t going to copy files that you don’t want to. The –dry-run flag helps you do this:

rsync --dry-run --update -raz --progress /var/www 10.1.1.1:/var

Combined with the –progress flag I have included in all the examples, –dry-run will output all the directories it looks at and then all the files that would be copied.