Sync data using Veritas Volume Manager

I have been using rsync to do data migration, recently I need to do data migration again.

Since this time the data resides in Veritas volumes, I want to try the Veritas way.

My setup in the old server is like this:

# vxprint -g ftpdg
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0  PUTIL0
dg ftpdg        ftpdg        -        -        -        -        -       -

dm ftpdg01      tagmastore-usp0_0 -   461350656 -       -        -       -
dm ftpdg02      tagmastore-usp0_1 -   31439616 -        -        -       -

v  volb         gen          ENABLED  461314816 -       ACTIVE   -       -
pl volb-01      volb         ENABLED  461314816 -       ACTIVE   -       -
sd ftpdg01-01   volb-01      ENABLED  461314816 0       -        -       -

v  vols         gen          ENABLED  31438848 -        ACTIVE   -       -
pl vols-01      vols         ENABLED  31438848 -        ACTIVE   -       -
sd ftpdg02-01   vols-01      ENABLED  31438848 0        -        -       -


I allocated two LUNs from storage, they are identified as tagmastore-usp0_2 and tagmastore-usp0_3.



To enable the new disks
# vxdctl enable

My privlen for old disks was 2048, so set the same value for new disks
# vxdisksetup -i tagmastore-usp0_2 privlen=2048
# vxdisksetup -i tagmastore-usp0_3 privlen=2048

Add the new disks to disk group:
# vxdg -g ftpdg adddisk ftpdg03=tagmastore-usp0_2
# vxdg -g ftpdg adddisk ftpdg04=tagmastore-usp0_3

Create sub disk and plex
# vxmake -g ftpdg sd ftpdg03-01 ftpdg03,0,461314816
# vxmake -g ftpdg plex volb-02

# vxmake -g ftpdg sd ftpdg04-01 ftpdg04,0,31438848
# vxmake -g ftpdg plex vols-02

Attach the plex to volumes
# vxplex -g ftpdg att volb volb-02
# vxplex -g ftpdg att vols vols-02

After attaching the plex, both old and new disks should have identical data, we can detach the new disks and add it to our new server.

Detach the new plex
# vxplex -g ftpdg det volb-02
# vxplex -g ftpdg det vols-02
Disassociate the new plex
# vxplex -g ftpdg dis volb-02
# vxplex -g ftpdg dis vols-02

Create new volume using the detached plex
# vxmake -g ftpdg vol newb plex=volb-02
# vxmake -g ftpdg vol news plex=vols-02

Split the diskgroup
# vxdg split ftpdg ftpdg2 newb news

At this step, we will have a diskgroup ftpdg2, inside it there are two volumes: newb and news.
We are planning to use the same volume names on new server, so rename the volumes in ftpdg2
# vxedit -g ftpdg2 rename newb volb
# vxedit -g ftpdg2 rename news vols
We can deport ftpdg2 from our old server:
# vxdg deport ftpdg2

On new server we can import ftpdg2 as ftpdg
# vxdg import -n ftpdg ftpdg2

We also need to start the volumes
# vxvol -g ftpdg start volb
# vxvol -g ftpdg start vols

The device path for the volumes are:
/dev/vx/dsk/ftpdg/volb
/dev/vx/dsk/ftpdg/vols

we can mount the volumes and start using it!

Set up password-less SSH login using ssh-keygen and ssh-copy-id

Logging in remote systems without password is very useful in system administration. It can be used for executing command on remote systems, copying files to other systems using rsync/rdist.

For the past few years, I have been manually setting up the key trust between systems. Here are the steps I followed.

Suppose I need to login to localserver from remoteserver using account scripter without password.

1. generate public-private key pair on localserver

$ ssh-keygen -t dsa

    you can use either dsa or rsa.

2. copy scripter's public key to remoteserver

$ scp id_dsa.pub remoteserver:

3. on remoteserver, append the public key copied in step 2 to ~scripter/.ssh/authorized_keys

$ cat id_dsa.pub >> ~scripter/.ssh/authorized_keys

    if ~scripter/.ssh doesn't exist, manually create it

4. on localserver the directory and files are auto-generated, so the permissions are correct, on remoteserver, as we manually created the directories, we need to make the permissions are correct.

$ chmod 700 ~scripter/.ssh

$ chmod 600 ~scripter/.ssh/authorized_keys

After all these steps, we can login from localserver to remoteserver without entering password:

$ ssh remoteserver hostname

remoteserver

The steps are quite easy, but can it be easier?
Yes! recently I learned the script ssh-copy-id from a forum post, to set up key trust from localserver to remoteserver using ssh-keygen and ssh-copy-id
1. generate public-private key pair on localserver

$ ssh-keygen -t dsa

2. copy the public key from localserver to remoteserver, and set up the key trust

$ ssh-copy-id -i ~scripter/.ssh/id_dsa.pub \
scripter@remoteserver

After you entering the password, everything is set up for you, you can log in remotely without password

$ ssh remoteserver hostname

remoteserver

ssh-copy-id saves us all the troubles of creating files and setting up the proper permission :)

Copy files from one server to another

In my work, I often need to copy files or whole directories between different servers. These are the ways I used often:

1. scp

scp -r directory remote:/path/to/directory

2. tar + ssh

tar -zcpf - directory | ssh remote tar -zxpf \
-C /path/to/directory

in Solaris, there is no -C -z flag, so I use:

tar -cpf - directory | ssh remote \
"(cd /path/to/directory; tar -xpf -)"

3. rsync over ssh
I have a central server, everyday there are some new files generated in one directory. I need to push this directory to remote servers everyday.
It's possible to schedule some cronjob to scp or tar + ssh this directory, But it will copy the whole directory everyday. To use rsync, we only need to copy the changes to this directory.

rync -e ssh -az --delete directory \
remote:/path/to/directory

4. nc + tar
A few years ago, I need to copy huge amount of data to a remote data center in another country, while exploring ways to copy the data, I found this utility nc.
But nc doesn't encrypt the data, I only tested it within the same data center.

on remote server:

cd /path/to/directory

nc -l -p 8888 | tar -zxvf -

on source server:

tar -czf - directory | nc remote 8888

on the remote server, I used -v flag, so I could monitor the copying progress.