VVR Replication configuration

[lazaur]

HI Guys,

I am configuring the replication with our 2 node cluster using cluster. but when  adding the node is giving the error. i am not able to get proper logs also in engine.log and RVG.log. please help.

[root@AOSCEDA01 ~]# vradmin -g jceda_dg addsec jceda_rvg AOSCEDA01 AOSCEDA02 prlink=to_AOSCEDA01 srlink=to_AOSCEDA02
VxVM VVR vradmin ERROR V-5-52-417 RVG jceda_rvg already exists in disk group jceda_dg.
VxVM VVR vradmin ERROR V-5-52-802 Cannot start command execution on Secondary.
[root@AOSCEDA01 ~]#

[mikebounds]

Please post output of "vxprint -VPl" and vradmin -g jceda_dg repstatus jceda_rvg" from both nodes.

Mike

[lazaur]

please find the output

[root@AOSCEDA01 jceda]# vxprint -VPl
Disk group: jceda_dg

Rlink:    to_AOSCEDA01
info:     timeout=500 rid=0.1240
          latency_high_mark=10000 latency_low_mark=9950
          bandwidth_limit=none
state:    state=STALE
          synchronous=off latencyprot=off srlprot=autodcm
assoc:    rvg=jceda_rvg
          remote_host=AOSCEDA02 IP_addr=192.168.0.171 port=
          remote_dg=jceda_dg
          remote_dg_dgid=
          remote_rvg_version=unknown
          remote_rlink=to_AOSCEDA02
          remote_rlink_rid=0.0
          local_host=AOSCEDA01 IP_addr=192.168.0.170 port=4145
protocol: TCP/IP
flags:    write enabled detached inconsistent cant_sync disconnected

Rvg:      jceda_rvg
info:     rid=0.1168 version=3 rvg_version=45 last_tag=2
state:    state=ACTIVE kernel=ENABLED
assoc:    datavols=lvol01,lvol02
          srl=Lvol_SRL
          rlinks=to_AOSCEDA01
          exports=(none)
          vsets=(none)
att:      rlinks=
flags:    closed primary enabled attached
device:   minor=25003 bdev=199/25003 cdev=199/25003 path=/dev/vx/dsk/jceda_dg/jceda_rvg
perms:    user=root group=root mode=0600

[root@AOSCEDA01 jceda]# vradmin -g jceda_dg repstatus jceda_rvg
Replicated Data Set: jceda_rvg
Primary:
  Host name:                  AOSCEDA01
  RVG name:                   jceda_rvg
  DG name:                    jceda_dg
  RVG state:                  enabled for I/O
  Data volumes:               2
  VSets:                      0
  SRL name:                   Lvol_SRL
  SRL size:                   5.00 G
  Total secondaries:          1
Config Errors:
 AOSCEDA02:                  disk group missing

[root@AOSCEDA01 jceda]#

--------------------------------------------------------------------------------------------

[root@AOSCEDA02 ~]# vxprint -VPl
Disk group: jceda_dg

Rlink:    to_AOSCEDA02
info:     timeout=500 rid=0.1310
          latency_high_mark=10000 latency_low_mark=9950
          bandwidth_limit=none
state:    state=STALE
          synchronous=off latencyprot=off srlprot=autodcm
assoc:    rvg=jceda_rvg
          remote_host=AOSCEDA01 IP_addr=192.168.0.170 port=
          remote_dg=jceda_dg
          remote_dg_dgid=
          remote_rvg_version=unknown
          remote_rlink=to_AOSCEDA01
          remote_rlink_rid=0.0
          local_host=AOSCEDA02 IP_addr=192.168.0.171 port=4145
protocol: TCP/IP
flags:    write enabled detached inconsistent cant_sync disconnected

Rvg:      jceda_rvg
info:     rid=0.1306 version=0 rvg_version=45 last_tag=0
state:    state=ACTIVE kernel=ENABLED
assoc:    datavols=lvol01,lvol02
          srl=Lvol_SRL
          rlinks=to_AOSCEDA02
          exports=(none)
          vsets=(none)
att:      rlinks=
flags:    closed secondary enabled attached nologging
device:   minor=18003 bdev=199/18003 cdev=199/18003 path=/dev/vx/dsk/jceda_dg/jceda_rvg
perms:    user=root group=root mode=0600

[root@AOSCEDA02 ~]# vradmin -g jceda_dg repstatus jceda_rvg
VxVM VVR vradmin INFO V-5-52-1205 Primary is unreachable or RDS has configuration error. Displayed status information is from Secondary and can be out-of-date.
Replicated Data Set: jceda_rvg

Secondary:
  Host name:                  AOSCEDA02
  RVG name:                   jceda_rvg
  DG name:                    jceda_dg
  Data status:                inconsistent
  Replication status:         stopped (secondary detached)
  Current mode:               N/A
  Logging to:                 SRL (updates behind N/A)
  Timestamp Information:      N/A
  Last Update on Primary:     N/A
  Secondary up-to-date as of: N/A

[root@AOSCEDA02 ~]#

[mikebounds · Accepted solution]

I don't know why config is still messed up, so I would re-create:

vradmin -g jceda_dg delsec jceda_rvg 
vradmin -g jceda_dg delpri jceda_rvg (may need -f)

vradmin -g jceda_dg createpri jceda_rvg lvol01,lvol2 Lvol_SRL
vradmin -g jceda_dg addsec jceda_rvg AOSCEDA01 AOSCEDA02 prlink=to_AOSCEDA02 srlink=to_AOSCEDA01
vradmin -g jceda_dg -a startrep jceda_rvg

It may also be worth making sure AOSCEDA01 and AOSCEDA02 resolve on both hosts by adding to DNS or local hosts file on both hosts.

Mike

[lazaur]

HI Mike,

Aobove required to aply on both the node.

[mikebounds]

No just run on AOSCEDA01, but I would make sure host files on each host contain BOTH hosts) or they are in DNS

Mike

[lazaur]

Hi Mike,

No thise server are not in DNS. i add the servers entry in host.

but they can ping and have root password less logging.

getting below error now please guid me as i run only one server. please how we can add it in RDS.

[root@AOSCEDA01 ~]# vradmin -g jceda_dg createpri jceda_rvg lvol01,lvol02 Lvol_SRL
[root@AOSCEDA01 ~]# vradmin -g jceda_dg addsec jceda_rvg AOSCEDA01 AOSCEDA02 prlink=to_AOSCEDA02 srlink=to_AOSCEDA01
Message from Host AOSCEDA02:
VxVM vxvol ERROR V-5-1-10127 associating volume lvol02 with jceda_rvg:
        Volume or plex device is open or attached
[root@AOSCEDA01 ~]# vradmin -g jceda_dg -a startrep jceda_rvg
Message from Primary:
VxVM VVR vradmin ERROR V-5-52-432 RDS does not contain any Secondary host.
[root@AOSCEDA01 ~]#

[lazaur]

Hi Gaurav/Mike,

i am able to solved above error now stuck on below please help to resolve. its showing a disk group missing as i have the active passive setup so onside storage is mounted. is there anything required to excute on secondry node AOSCEDA02.

[root@AOSCEDA02 ~]# vxdisk list -o alldgs
DEVICE       TYPE            DISK         GROUP        STATUS
disk_1       auto:cdsdisk    -            (jceda_dg)   online
disk_2       auto:cdsdisk    -            (jceda_dg)   online
disk_3       auto:cdsdisk    -            (jceda_dg)   online
sda          auto:LVM        -            -            online invalid
[root@AOSCEDA02 ~]# vxprint -VPl
[root@AOSCEDA02 ~]#

[root@AOSCEDA01 ~]# tail -f /var/log/messages
Aug  2 14:46:59 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Stream error on remote
Aug  2 14:47:00 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Disk group or rlink not found on remote
Aug  2 14:47:21 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Stream error on remote
Aug  2 14:47:22 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Disk group or rlink not found on remote
Aug  2 14:47:43 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Stream error on remote
Aug  2 14:47:44 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Disk group or rlink not found on remote
Aug  2 14:48:05 AOSCEDA01 kernel: VxVM VVR vxio V-5-0-330 Unable to connect to rlink to_AOSCEDA02 on rvg jceda_rvg: Stream error on remote

-----------------------------------------------------------------------------------------------------------------------

[root@AOSCEDA01 ~]# vxprint -VPl
Disk group: jceda_dg

Rlink:    to_AOSCEDA02
info:     timeout=500 rid=0.1373
          latency_high_mark=10000 latency_low_mark=9950
          bandwidth_limit=none
state:    state=ACTIVE
          synchronous=off latencyprot=off srlprot=autodcm
assoc:    rvg=jceda_rvg
          remote_host=AOSCEDA02 IP_addr=192.168.0.171 port=4145
          remote_dg=jceda_dg
          remote_dg_dgid=1437914284.6.AOSCEDA02.DTP.AE
          remote_rvg_version=45
          remote_rlink=to_AOSCEDA01
          remote_rlink_rid=0.1450
          local_host=AOSCEDA01 IP_addr=192.168.0.170 port=4145
protocol: TCP/IP
flags:    write enabled attached consistent disconnected asynchronous

Rvg:      jceda_rvg
info:     rid=0.1321 version=3 rvg_version=45 last_tag=2
state:    state=ACTIVE kernel=ENABLED
assoc:    datavols=lvol01,lvol02
          srl=Lvol_SRL
          rlinks=to_AOSCEDA02
          exports=(none)
          vsets=(none)
att:      rlinks=to_AOSCEDA02
flags:    closed primary enabled attached
device:   minor=25003 bdev=199/25003 cdev=199/25003 path=/dev/vx/dsk/jceda_dg/jceda_rvg
perms:    user=root group=root mode=0600

[root@AOSCEDA01 ~]#

[root@AOSCEDA01 ~]# vxprint -l to_AOSCEDA02
Disk group: jceda_dg

Rlink:    to_AOSCEDA02
info:     timeout=500 rid=0.1373
          latency_high_mark=10000 latency_low_mark=9950
          bandwidth_limit=none
state:    state=ACTIVE
          synchronous=off latencyprot=off srlprot=autodcm
assoc:    rvg=jceda_rvg
          remote_host=AOSCEDA02 IP_addr=192.168.0.171 port=4145
          remote_dg=jceda_dg
          remote_dg_dgid=1437914284.6.AOSCEDA02.DTP.AE
          remote_rvg_version=45
          remote_rlink=to_AOSCEDA01
          remote_rlink_rid=0.1450
          local_host=AOSCEDA01 IP_addr=192.168.0.170 port=4145
protocol: TCP/IP
flags:    write enabled attached consistent disconnected asynchronous

[root@AOSCEDA01 ~]# vradmin -g jceda_dg repstatus jceda_rvg
Replicated Data Set: jceda_rvg
Primary:
  Host name:                  AOSCEDA01
  RVG name:                   jceda_rvg
  DG name:                    jceda_dg
  RVG state:                  enabled for I/O
  Data volumes:               2
  VSets:                      0
  SRL name:                   Lvol_SRL
  SRL size:                   5.00 G
  Total secondaries:          1
Config Errors:
 AOSCEDA02:                  disk group missing

[root@AOSCEDA01 ~]#

[lazaur]

Hi Mike/Gaurav,

Any suggetion please.

[mikebounds]

First issue was I guess you had filesystem mounted on 2nd node (volumes on secodary must be closed as in earlier comment - https://www-secure.symantec.com/connect/forums/vvr-replication-configuration#comment-11268211)

From output above, the diskgroup is deported so you should run:

vxdg import jceda_dg
vxrecover -sg jceda_dg

Mike

[lazaur]

hi MIke,

no FS mounted on other node check below logs.

You have new mail in /var/spool/mail/root
[root@AOSUAMQ01 ~]# vxdisk list -o alldgs
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:cdsdisk    jceda        jceda_dg     online
sda          auto:LVM        -            -            online invalid
sdb          auto:LVM        -            -            LVM
[root@AOSUAMQ01 ~]#

[root@AOSUAMQ02 ~]# vxdisk list -o alldgs
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:cdsdisk    -            (jceda_dg)   online
sda          auto:LVM        -            -            online invalid
sdb          auto:LVM        -            -            LVM
[root@AOSUAMQ02 ~]#

[mikebounds]

When you ran:

[root@AOSCEDA01 ~]# vradmin -g jceda_dg addsec jceda_rvg AOSCEDA01 AOSCEDA02 prlink=to_AOSCEDA02 srlink=to_AOSCEDA01
Message from Host AOSCEDA02:
VxVM vxvol ERROR V-5-1-10127 associating volume lvol02 with jceda_rvg:
        Volume or plex device is open or attached

This error usually indicates filesystem is mounted (which is why it is open).

But your issue now is that your diskgroup is deported on 02 host, so you need to import as in last post.

Mike

[lazaur]

Hi Mike,

i am also not able to mount the FS on secoundry in RW mode but i can mount in RO mode. please clear we can mount it on secondry mode as read only

root@AOSUAMQ02 ~]# mount -t vxfs /dev/vx/dsk/jceda_dg/lvol01 /jceda
UX:vxfs mount.vxfs: ERROR: V-3-21252: not super user

[root@AOSUAMQ02 ~]# mount -t vxfs -o ro /dev/vx/dsk/jceda_dg/lvol01 /jceda
[root@AOSUAMQ02 ~]#

[mikebounds]

You are able to mount second VVR node as RO, but you will not seen any updates written by primary as the writes happen underneath the filesystem.  So if you want to test data is being replicated then follow steps in earlier comment (https://www-secure.symantec.com/connect/forums/vvr-replication-configuration#comment-11268371).  

Just to be clear, only the system that is currently the VVR primary (see flags in vxprint -Vl) should have the filesystem mounted.

Mike

[lazaur]

HI Mike/Gaurav,

thanks for the help it working now actualy i configured the local cluster by mistek its done when reconfigured the cluster and  created remote cluster.

Regards

sarfraz