Forum Discussion

Jan_Rustico's avatar
Jan_Rustico
Level 4
10 years ago

SLP Duplication issue

Currenlty we have upgraded to NBU 7.6.0.4 ruinning unnder windows server 2008 r2 standard. There are lots of duplication issues.

--

Exch server 2013 SP1

22/12/2014 2:28:04 PM - requesting resource LCM_ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 2:28:04 PM - granted resource LCM_ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 2:28:04 PM - started process RUNCMD (6556)
22/12/2014 2:28:04 PM - ended process 0 (6556)
22/12/2014 2:28:05 PM - begin Duplicate
22/12/2014 2:28:06 PM - requesting resource ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 2:28:06 PM - requesting resource @aaaal
22/12/2014 2:28:06 PM - reserving resource @aaaal
22/12/2014 2:28:06 PM - reserved resource @aaaal
22/12/2014 2:28:06 PM - granted resource 0052L5
22/12/2014 2:28:06 PM - granted resource IBM.ULT3580-HH5.001
22/12/2014 2:28:06 PM - granted resource ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 2:28:06 PM - granted resource MediaID=@aaaal;DiskVolume=StorageUnit2;DiskPool=DD160_POOL2;Path=StorageUnit2;StorageServer=ksc-2backup;MediaServer=ksc-mediasrv.alkhodari.local
22/12/2014 2:29:13 PM - Error bpduplicate(pid=6556) db_IMAGE() failed: database system error (220)       
22/12/2014 2:29:13 PM - Error bpduplicate(pid=6556) Status = no images were successfully processed.      
22/12/2014 2:29:13 PM - Error bpduplicate(pid=6556) Duplicate of backupid ksc-exch-dag1_1418641854 failed, database system error (220).    
22/12/2014 2:29:13 PM - Error bpduplicate(pid=6556) Status = no images were successfully processed.      
22/12/2014 2:29:13 PM - end Duplicate; elapsed time: 0:01:08
no images were successfully processed(191)

--

Catalog

22/12/2014 1:58:04 PM - begin Duplicate
22/12/2014 1:58:04 PM - requesting resource LCM_ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 1:58:04 PM - granted resource LCM_ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 1:58:04 PM - started process RUNCMD (1572)
22/12/2014 1:58:04 PM - ended process 0 (1572)
22/12/2014 1:58:04 PM - requesting resource ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 1:58:04 PM - requesting resource @aaaab
22/12/2014 1:58:04 PM - reserving resource @aaaab
22/12/2014 1:58:05 PM - reserved resource @aaaab
22/12/2014 1:58:05 PM - granted resource W160L5
22/12/2014 1:58:05 PM - granted resource IBM.ULT3580-HH5.000
22/12/2014 1:58:05 PM - granted resource ksc-mediasrv-hcart2-robot-tld-0
22/12/2014 1:58:05 PM - granted resource MediaID=@aaaab;DiskVolume=StorageUnit1;DiskPool=DD160_POOL;Path=StorageUnit1;StorageServer=ksc-backup;MediaServer=ksc-mediasrv.alkhodari.local
22/12/2014 1:58:07 PM - Info bpduplicate(pid=1572) window close behavior: Suspend         
22/12/2014 1:58:07 PM - Info bptm(pid=4188) start            
22/12/2014 1:58:07 PM - started process bptm (4188)
22/12/2014 1:58:07 PM - Info bptm(pid=4188) start backup           
22/12/2014 1:58:08 PM - Info bpdm(pid=1508) started            
22/12/2014 1:58:08 PM - started process bpdm (1508)
22/12/2014 1:58:08 PM - Info bpdm(pid=1508) reading backup image          
22/12/2014 1:58:08 PM - Info bpdm(pid=1508) using 30 data buffers         
22/12/2014 1:58:08 PM - Info bpdm(pid=1508) requesting nbjm for media         
22/12/2014 1:58:08 PM - Info bptm(pid=4188) Waiting for mount of media id W160L5 (copy 2) on server ksc-mediasrv.alkhodari.local. 
22/12/2014 1:58:08 PM - started process bptm (4188)
22/12/2014 1:58:08 PM - mounting W160L5
22/12/2014 1:58:08 PM - Info bptm(pid=4188) INF - Waiting for mount of media id W160L5 on server ksc-mediasrv.alkhodari.local for writing.
22/12/2014 1:58:10 PM - begin reading
22/12/2014 1:58:49 PM - Info bptm(pid=4188) media id W160L5 mounted on drive index 0, drivepath {3,0,3,0}, drivename IBM.ULT3580-HH5.000, copy 2
22/12/2014 1:58:49 PM - Info bptm(pid=4188) INF - Waiting for positioning of media id W160L5 on server ksc-mediasrv.alkhodari.local for writing.
22/12/2014 2:00:53 PM - Info bptm(pid=4188) waited for full buffer 1401 times, delayed 2280 times    
22/12/2014 2:00:54 PM - end reading; read time: 0:02:44
22/12/2014 2:00:54 PM - Info bptm(pid=4188) setting receive network buffer to 1049600 bytes      
22/12/2014 2:00:54 PM - Error bpdm(pid=1508) ERR - Error occurred reading TIR information,               expected this frag 501260 bytes, read this frag 0
22/12/2014 2:00:54 PM - Error bptm(pid=4188) socket operation failed - 10054 (at bptm.c.27398)      
22/12/2014 2:01:00 PM - Error bpduplicate(pid=1572) host ksc-mediasrv.alkhodari.local backup id ksc-mediasrv.alkhodari.local_1419239036 read failed, media read error (85).  
22/12/2014 2:01:00 PM - Info bptm(pid=4188) EXITING with status 23 <----------        
22/12/2014 2:01:00 PM - Error bpduplicate(pid=1572) host ksc-mediasrv.alkhodari.local backupid ksc-mediasrv.alkhodari.local_1419239036 write failed, socket read failed (23).   
22/12/2014 2:01:00 PM - Error bpduplicate(pid=1572) Duplicate of backupid ksc-mediasrv.alkhodari.local_1419239036 failed, socket read failed (23).    
22/12/2014 2:01:00 PM - Error bpduplicate(pid=1572) Status = no images were successfully processed.      
22/12/2014 2:01:01 PM - end Duplicate; elapsed time: 0:02:57
no images were successfully processed(191)

--

 

This only happens to Exchange and Catalog.

 

Also when I ran the 'nbstlutil stlilist -image_incomplete -l ' , results below:

--

:\Program Files\Veritas\NetBackup\bin\admincmd>nbstlutil stlilist -image_incomplete -l
7.6.0 I ksc-exch-dag1_1418641854 Exch_Weekly 2
7.6.0 C ksc-mediasrv-hcart2-robot-tld-0 1 1
7.6.0 I ksc-mediasrv.alkhodari.local_1418707389 Catalog 2
7.6.0 C ksc-mediasrv-hcart2-robot-tld-0 1 1
7.6.0 I ksc-mediasrv.alkhodari.local_1419239036 Catalog 2
7.6.0 C ksc-mediasrv-hcart2-robot-tld-0 1 1

I've tried to reran the duplicate process using 'nbstlutil active -backupid ' command, results below:

--

C:\Program Files\Veritas\NetBackup\bin\admincmd>nbstlutil active -backupid  ksc-exch-dag1_1418641854      

No images or lifecycles matching criteria found.                                                          

 

7.6
Backing Up
Backup and Recovery
Best Practice
Configuring
NetBackup
Windows Server (2003-2008)
  • RamNagalla's avatar
    RamNagalla
    Moderator
    10 years ago

    are you taking the GRT backups for exchage backups... if yes you can try "disable the cataloging of Exchange backups using Granular Recovery Technology"

    http://www.symantec.com/business/support/index?page=content&id=HOWTO85775

    To disable the cataloging of Exchange backups using Granular Recovery Technology

    1. On the master server, open the NetBackup Administration Console.

    2. In the left pane, expand Host Properties.

    3. Click Master Servers.

    4. In the right pane, right-click the master server click Properties.

    5. Click General Server.

    6. Uncheck Enable message-level cataloging when duplicating Exchange images that use Granular Recovery Technology.

    7. Click OK.

     

  • Andrew_Madsen's avatar
    Andrew_Madsen
    Level 6
    10 years ago

    Have you tried a restore of the image to validate the image database is not corrupted?

  • Jan_Rustico's avatar
    Jan_Rustico
    Level 4
    10 years ago

    Symantec NBU team told me that there is a TIR issue. It seems that there is a compatibility issue between our Symantec NBU 7.6.0.4 and EMC DataDomain with DD Boost. The current EMC DD Boost version is 2.6.1.0-396586. Tfhey also asked us to upgrade the current DD 160 OS from 5.1 to 5.4 or higher. Any thoughts about this? 

    Below is the email from Symantec NBU Technical Support after they obtain logs from us:

    --

    From: Enterprise Technical Support [mailto:enterprise_technical_support@symantec.com]

    Sent: Monday, December 29, 2014 11:58 AM

    To: XXXXXXXXXXXXXXXXXX

    Subject: RE: Case # 07XXXXX - Issues in catalog backup and duplication to tape has been created [ref:00D30jP [ ref:_00D30jPy._50050YOcHW:ref ]

     

    HelloXXXXX,

     

     

    DDBoost 2.6 that you are running needs to have at least DD OS 5.2 See page 32.

     

    I suggest to upgrade DD OS to 5.4.2.1.

     

    On page 32 of the guide below:

     

    December 08, 2014 - docu51631 - Data Domain Boost Version Compatibility Guide

     

    _https://support.emc.com/docu51631_Data_Domain_Boost_Version_Compatibility_Guide.pdf?language=en_US<_https://urldefense.proofpoint.com/v1/url?u=https://support.emc.com/docu51631_Data_Domain_Boost_Version_Compatibility_Guide.pdf?language%3Den_US&k=wdHsQuqY0Mqq1fNjZGIYnA%3D%3D%0A&r=ng%2BBygRLWNB7cERkdGe%2FpDlSYxYC5CgujnoMBlaQz2H0%2BVLWjq6tJzeDM7%2FnRAf8%0A&m=EdVPyvRj%2FsYjiE5hY%2B%2BI99DSaTVjR1kHxrKnq30q47s%3D%0A&s=6506e15bae0a3826677130e79e9937ec8614581f6f2c2bac4c51e47cc9e0f1bc>

     

     

    1.Run the some small backups to same DD and see if it works 2.Run verfify on the problematic backusp ID for which we got the error 3.Run the same problematic duplication again by enabling bpdm ,admin on media server and bpdbm and admin on master we need to get the logs along with ddfs.log from data Domian storage server it seems from DD we are not able to read the TIR fragment it could be issue from DD end

     

    Note:-send me above logs with same pid and verbose =5, specialy send ddfs log  once you run backup and it got fail

     

    Mean while open case with Data domain as we are not able to read TIR infromation from the disk which is in Data domain and we do not have access to data domain.

     

    In case of query feel free to contact

     

    Regards

    XXXXXXXX

     

  • DV1UBY's avatar
    DV1UBY
    Level 3
    10 years ago

    We've performed EMC DataDomain DD160 OS upgrade to 5.4.4.4.2. In the media server the old olt was removed and the new OST version is 3.0.0.1. Still there is a catalog duplication issue.