Using rsync to Transfer and Synchronize Local and Remote Systems

demo

Rsync is a powerful tool that facilitates the transfer and synchronization of data between both local and remote systems.


Basic rsync usage Let’s create two directories inside /tmp, called ”foo” and ”bar”, and generate a large amount of dummy files inside /tmp/foo

1. mkdir /tmp/foo /tmp/bar
   
2. for i in `seq 1 100`;do touch /tmp/foo/file$i;done

复制代码

Now, we’ve got 100 files in /tmp/foo; /tmp/bar should still have none.  We can use rsync to copy all of the files from /tmp/foo over to /tmp/bar:

1. rsync /tmp/foo/* /tmp/bar

复制代码

Using basic file globbing, we can grab all files and copy them to another directory.  What if there’s a directory inside /tmp/foo?  It won’t be transferred.  We will need to use the -r flag (–recursive) to traverse the directory, transferring every file inside:

1. rsync -r /tmp/foo/ /tmp/bar

复制代码

This is a very simple example and does not even begin to touch the real power of the rsync command.  There are flags to preserve permissions, ownerships, groups, symlinks, and so on. Because these flags are so often used, the -a (–archive) flag acts as an alias to incorporate them all, including -r.  
Clear out /tmp/bar, create a symlink to one file in /tmp/foo, and use rsync to recursively copy all files:

1. rm -rf /tmp/bar/*
   
2. ln -s /tmp/foo/file100 /tmp/foo/file101
   
3. rsync -r /tmp/foo/ /tmp/bar

复制代码

We see that rsync has omitted the symlink we created.  Clear out /tmp/bar again, and let’s try it again, this time using the -a flag:

1. rm -rf /tmp/bar/*
   
2. rsync -a /tmp/foo/ /tmp/bar

复制代码

Use chown to change the ownership of a file in /tmp/foo to another user, and copy over the files using -a to /tmp/bar.  Run ls -l and note that the ownership moved with the file.  Handy stuff!
NOTE: There is a difference between including a forward slash (/) at the end of the source path, and omitting it; the former will transfer all files INSIDE the specified directory, while the latter will transfer the directory itself with all files inside.

The -a Flag We mentioned earlier that the -a (–archive) flag is an alias for a collection of other flags, -rltpgoD.  Broken down, each flag does the following:
r – Recursive
l – Transfer any symlinks encountered
t – Preserve time stamps
p – Preserve permissions
g – Preserve groups
o – Preserve ownership
D – Preserve block and character devices
You may want to add the following to your command for easier to read file sizes:
h – Human-readable format of file sizes
Everyone Loves Feedback The -v (–verbose) flag will give you more output about the state of the transfer, including a summary at the end, which will look something like this:

1. $ rsync -av foo/ bar
   
2. building file list ... done
   
3. sent 1040 bytes  received 20 bytes  2120.00 bytes/sec
   
4. total size is 7  speedup is 0.01

复制代码

If you want more statistics, run rsync with the –stats flag.  This will give you a detailed list of the total number of files, files transferred, benchmarks, and even an averaged transfer speed.  On the other side, -q (–quiet) will suppress all output, which can be used for scripts when feedback is not required.
Remote Transfers Made Simple The true power of rsync is in its ability to perform not only local transfers but remote transfers as well.  If you’ve used scp before, the syntax for remote transfers is quite similar:

1. rsync [flags] [local path] [user]@[remote server]:[remote path]

复制代码

As an example, an rsync using this syntax would look like the following:

1. rsync -avh /tmp/foo/ root@host2:/tmp/bar

复制代码

Note the : (colon) between the remote server and the remote path; this is required.
More Options rsync comes with a large list of available options, far too many to go over in a single article.  The last flags we will cover are the –exclude, –exclude-from, –update and –delete flags
–exclude

  Exclude files based on a pattern.  Rsync doesn’t support regex yet, so only standard file matching and globbing work

–exclude-from

  Exclude files listed in a line-separated file

–update

Update files at the destination ONLY if the source copy has been modified more recently

–delete

Delete files on the destination ONLY if the source copy no longer exists.

Alternate SSH PortsIf you have changed the SSH port on your server you will need to tell rsync to use the new port number.
Example with normal SSH port:

1. rsync -azh /local/path/file user@host.com:/remote/path/file

复制代码

Example with alternate SSH port (22334):

1. rsync -azh /local/path/file -e 'ssh -p 22334' user@host.com:/remote/path/file

复制代码

Password-less Remote Transfers Remote-to-local or local-to-remote file transfers can be made easier with the use of SSH keys.  With SSH keys set up on both remote and local servers, synchronization can be scripted effortlessly and without human intervention (not having to enter a password every time).  In another article, we discuss how to setup SSH keys.

来自圈子: Demo俱乐部