cancel
Showing results for 
Search instead for 
Did you mean: 

Import is not working on DR server however replication job is getting successful on Master server.

dmendhekar
Level 3

We are using AIR to replicate the images from Master to DR site. Replication job is getting completed sucessfully on Master server however images are not getting imported to DR server. Nbstlutil is showing complete status on master server but nbstlutil pendimplist showing nothing on DR site.

Master and DR NBU: 7.6.0.1
OS: Windows server 2008 R2 64bit
DD OS Version 5.4.5.0
DD Plugin version 3.0.1.0-451885.

 

C:\Program Files\Veritas\NetBackup\bin\admincmd>nbstlutil stlilist -lifecycle DD

Backup_AIR_Daily -U
Image msqlcluster_1432827026 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1432913426 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1432999822 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433158839 for Lifecycle DDBackup_AIR_Daily is COMPLETE

 

Replication Job Details:

6/5/2015 12:11:03 PM - requesting resource LCM_*Remote*Master*
6/5/2015 12:11:03 PM - Info nbreplicate(pid=7432) Suspend window close behavior is not supported for nbreplicate    
6/5/2015 12:11:03 PM - Info nbreplicate(pid=7432) window close behavior: Continue processing the current image     
6/5/2015 12:11:03 PM - granted resource LCM_*Remote*Master*
6/5/2015 12:11:03 PM - started process RUNCMD (7432)
6/5/2015 12:11:04 PM - requesting resource @aaab6
6/5/2015 12:11:04 PM - reserving resource @aaab6
6/5/2015 12:11:04 PM - reserved resource @aaab6
6/5/2015 12:11:04 PM - granted resource MediaID=@aaab6;DiskVolume=DDBackup_AIR;DiskPool=DDBackup_AIR;Path=DDBackup_AIR;StorageServer=192.168.7.124;MediaServer=ntnbuserver
6/5/2015 12:11:05 PM - Info bpdm(pid=8880) started            
6/5/2015 12:11:05 PM - started process bpdm (8880)
6/5/2015 12:11:08 PM - Info ntnbuserver(pid=8880) Using OpenStorage to replicate backup id ntsapdev1_1433477037, media id @aaab6, storage server 192.168.7.124, disk volume DDBackup_AIR
6/5/2015 12:11:09 PM - Info ntnbuserver(pid=8880) Replicating images to target storage server Unknown, disk volume DDBackup_AIR   
6/5/2015 12:20:11 PM - replicated backup id ntsapdev1_1433477037 successfully
6/5/2015 12:20:11 PM - end operation
the requested operation was successfully completed(0)

16 REPLIES 16

Marianne
Level 6
Partner    VIP    Accredited Certified
Please show us output of nbstl -L on both masters. We need to see the import SLP for the same name on the DR master.

revarooo
Level 6
Employee

So nbstlutil pendimplist shows nothing, makes me suspect the replication isn't working.

 

 

dmendhekar
Level 3

Hello Mariane,

Please find the attached nbstl output from Master and DR server.

dmendhekar
Level 3

Hello Revaroo,

Replication is geting successful.

C:\Program Files\Veritas\NetBackup\bin\admincmd>nbstlutil stlilist -lifecycle DD
Backup_AIR_Daily -U
Image msqlcluster_1432827026 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1432913426 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1432999822 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433158839 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433158840 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433158841 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433158842 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433162439 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433162440 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433162441 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433162446 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433166039 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433166040 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433166042 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433166056 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433169639 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433169640 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433169642 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433169646 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433172631 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433173254 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433216473 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433224863 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433224877 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433224912 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433224962 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433236618 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433236619 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433236620 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433236621 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433259031 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433311218 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433311219 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433311220 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433311221 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433314820 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapqas1_1433314821 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapsolman00_1433314822 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433314831 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433345432 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image msqlcluster_1433431831 for Lifecycle DDBackup_AIR_Daily is COMPLETE
Image ntsapdev1_1433477037 for Lifecycle DDBackup_AIR_Daily is IN_PROCESS
  Copy to *Remote*Master* of type REPLICATE_TO_REMOTE_MASTER is IN_PROCESS

Marianne
Level 6
Partner    VIP    Accredited Certified
All images not yet replicated : Image ntsapdev1_1433477037 for Lifecycle DDBackup_AIR_Daily is IN_PROCESS Copy to *Remote*Master* of type REPLICATE_TO_REMOTE_MASTER is IN_PROCESS

Pritesh_Pisal
Level 5
Partner Accredited

Is this a new setup? does it worked before this issue?

is your DD STU association is correct?

is your Destination Diskpool shows volume type Replication target ?

 

 

 

 

watsons
Level 6

Important question: has the import ever worked before in DR site?

If it never have, in your DR site, check the MSDP server has the following processes running:

   nbstserv, nbrmms, spoold & spad

If it had but not now, something must have broken. Make sure nbrmms is running and enable unified logs using this:

  vxlogcfg -a -p NB -o 395 -s DebugLevel=6  (logs will be in /usr/openv/logs/nbrmms/)

Look for error in there about loading event files (or can't find event files etc.). Refer this old technote for a sample of how the logs may look like: http://www.symantec.com/docs/TECH207523

dmendhekar
Level 3

Hello Watson,

We are using DataDomain for both source and target.

dmendhekar
Level 3

Please find the attached log. Import never worked in this environment.

 

Admin logs:

(Post edited by Marianne to move log into .txt)

watsons
Level 6

Can you pls edit your last post - to make it shorter? Logs attached instead of copy & paste , make it more readable-friendly.

Will take a look at the logs when I have some free time...

watsons
Level 6

Didn't find anything useful in logs.. maybe it's better to enable more:

vxlogcfg -a -p NB -o 395 -s DebugLevel=6  (logs will be in /usr/openv/logs/nbrmms/)

vxlogcfg -a -p NB -o 369 -s DebugLevel=6  (logs will be in /usr/openv/logs/nbstserv/)

Then just attach the raw logs in both dir above., do not copy and paste please..

dmendhekar
Level 3

Now i have upgraded the NBU to 7.6.0.4 on both source and destination server but no luck.

Marianne
Level 6
Partner    VIP    Accredited Certified

I saw the following in admin log:

1:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: checked for C:\Program Files\Veritas\NetBackup\db\config\ADJUST_DSUPATH_CAPACITY.Backup_DD
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: sts_get_lsu_prop_byname ... Backup_DD
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: stat_sts: sts_get_lsu_prop_byname error on LSU Backup_DD, err 2060018
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: tag NOT FOUND: mkdir
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: sending REPLY
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: info=sts_get_lsu_prop_byname error on LSU Backup_DD, err 2060018
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: status=ERROR
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: stsError=glp-2060018
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: stslog=STH_ESINFO: dpsProxy: [320:1678] ddp_open_dir() failed, Path Backup_DD//, Err: 5004-nfs lookup failed (nfs: No such file or directory)
11:59:07.659 [800.5752] <2> bpstsinfo/DPSPROXY DEBUG: stslog=STH_ESERROR: dpsProxy: C:\Program Files\Veritas\NetBackup\bin\\ost-plugins\libstspiDataDomain.dll:stspi_get_lsu_prop_byname_common cannot open storage-unit Backup_DD presumed not to exist

 

This points to a config issue or either comms issue between the 2 sites.

We don't know anything about config steps, and since this has never worked, my reccommendation is to have a look at each component of inititial setup as if this is a new environment. 

Compare steps outlined in this post: 

Step by step guide for AIR on DataDomain 

watsons
Level 6

I looked back into your original post and found this:

6/5/2015 12:11:09 PM - Info ntnbuserver(pid=8880) Replicating images to target storage server Unknown, disk volume DDBackup_AIR   

target storage server is "unknown", so the replication could have written to a dummy storage server, and the event file would not have been created on the correct target. Please reconfigure your AIR target storage server.

dmendhekar
Level 3

I checked the configuration and it seems ok. I have attahed the new set of logs for better understanding.

Marianne
Level 6
Partner    VIP    Accredited Certified

Before we open any logs - can you tell us if you have been able to troubleshoot and solve this error?

stspi_get_lsu_prop_byname_common cannot open storage-unit Backup_DD presumed not to exist 

This error points to a comms or config issue and needs to be resolved by carefully comparing step-by-step instructions in post referenced above.