Friday, 4 May 2012

Copy files from one server to another

In my work, I often need to copy files or whole directories between different servers. These are the ways I used often:

1. scp
scp -r directory remote:/path/to/directory

2. tar + ssh
tar -zcpf - directory | ssh remote tar -zxpf \
-C /path/to/directory
in Solaris, there is no -C -z flag, so I use:
tar -cpf - directory | ssh remote \
"(cd /path/to/directory; tar -xpf -)"

3. rsync over ssh
I have a central server, everyday there are some new files generated in one directory. I need to push this directory to remote servers everyday.
It's possible to schedule some cronjob to scp or tar + ssh this directory, But it will copy the whole directory everyday. To use rsync, we only need to copy the changes to this directory.
rync -e ssh -az --delete directory \
remote:/path/to/directory

4. nc + tar
A few years ago, I need to copy huge amount of data to a remote data center in another country, while exploring ways to copy the data, I found this utility nc.
But nc doesn't encrypt the data, I only tested it within the same data center.

on remote server:
cd /path/to/directory
nc -l -p 8888 | tar -zxvf -

on source server:
tar -czf - directory | nc remote 8888

on the remote server, I used -v flag, so I could monitor the copying progress.

No comments:

Post a Comment