Highlighted

Duplication Error

   
6/24/2012 9:37:21 AM - granted resource 000032
6/24/2012 9:37:21 AM - granted resource HP.ULTRIUM3-SCSI.001
6/24/2012 9:37:21 AM - granted resource bs-itu01-002-hcart3-robot-tld-0
6/24/2012 9:41:02 AM - Error bpduplicate(pid=12680) db_IMAGE() failed: cannot connect on socket (25)      
6/24/2012 9:44:43 AM - Error bpduplicate(pid=12680) db_IMAGE() failed: cannot connect on socket (25)      
6/24/2012 9:48:24 AM - Error bpduplicate(pid=12680) db_IMAGE() failed: cannot connect on socket (25)      
6/24/2012 9:52:03 AM - Error bpduplicate(pid=12680) db_IMAGE() failed: cannot connect on socket (25)      
6/24/2012 9:55:50 AM - Error bpduplicate(pid=12680) db_IMAGE() failed: cannot connect on socket (25)      
6/24/2012 9:59:29 AM - Error bpduplicate(pid=12680) db_IMAGE() failed: cannot connect on socket (25)    

 

Duplication is stuck with the following error in job Details.

11 Replies
Highlighted

bpcd

 

C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -host bs-itu01-002 -verbose
1 1 1
172.16.0.59:65107 -> 172.16.0.62:1556
172.16.0.59:65108 -> 172.16.0.62:1556
PEER_NAME = bs-itu01-001
HOST_NAME = bs-itu01-002.kacst.ksa
CLIENT_NAME = bs-itu01-002.kacst.ksa
VERSION = 0x07500000
PLATFORM = win_x64
PATCH_VERSION = 7.5.0.1
SERVER_PATCH_VERSION = 7.5.0.1
MASTER_SERVER = bs-itu01-001.kacst.ksa
EMM_SERVER = bs-itu01-001.kacst.ksa
NB_MACHINE_TYPE = MEDIA_SERVER

----

C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -host bs-itu01-001 -verbose
1 1 1
172.16.0.62:52590 -> 172.16.0.59:1556
172.16.0.62:52591 -> 172.16.0.59:1556
PEER_NAME = bs-itu01-002
HOST_NAME = bs-itu01-001.kacst.ksa
CLIENT_NAME = bs-itu01-001.kacst.ksa
VERSION = 0x07500000
PLATFORM = win_x64
PATCH_VERSION = 7.5.0.1
SERVER_PATCH_VERSION = 7.5.0.1
MASTER_SERVER = bs-itu01-001.kacst.ksa
EMM_SERVER = bs-itu01-001.kacst.ksa
NB_MACHINE_TYPE = MASTER_SERVER

 

 

Highlighted

Check this out, I think it's

Highlighted

bpcd connection successful

but my bpcd connection is successful as i already attached the logs above.

Highlighted

Please tell us more about

Please tell us more about source and destination STU.
The extract from Job Details shows destination STU, not source.
Is source and destination attached to same media server or different ones?

How is duplication done? Manual? DSSU? Vault? SLP?
Is this a new duplication job that has never worked?

Please ensure that admin log is enabled on Master. This is normally step 1 in troubleshooting duplications.

Highlighted

DSU

6/25/2012 8:57:33 AM - begin Duplicate
6/25/2012 8:57:33 AM - end Duplicate; elapsed time: 00:00:00
found no images or media matching the selection criteria(190)

--------------------------------

6/25/2012 9:07:37 AM - Info nbjm(pid=7000) starting backup job (jobid=3869) for client bs-itu01-001.kacst.ksa, policy __DSSU_POLICY_DiskStagingIncremental, schedule DiskStagingIncremental  
6/25/2012 9:07:37 AM - Info nbjm(pid=7000) requesting NO_STORAGE_UNIT resources from RB for backup job (jobid=3869, request id:{F1FD24E4-7DB6-4F46-978D-C1B48F4BE5F1})  
6/25/2012 9:07:37 AM - requesting resource bs-itu01-001.kacst.ksa.NBU_CLIENT.MAXJOBS.bs-itu01-001.kacst.ksa
6/25/2012 9:07:37 AM - requesting resource bs-itu01-001.kacst.ksa.NBU_POLICY.MAXJOBS.__DSSU_POLICY_DiskStagingIncremental
6/25/2012 9:07:37 AM - granted resource bs-itu01-001.kacst.ksa.NBU_CLIENT.MAXJOBS.bs-itu01-001.kacst.ksa
6/25/2012 9:07:37 AM - granted resource bs-itu01-001.kacst.ksa.NBU_POLICY.MAXJOBS.__DSSU_POLICY_DiskStagingIncremental
6/25/2012 9:07:37 AM - estimated 0 Kbytes needed
6/25/2012 9:07:37 AM - begin Parent Job
6/25/2012 9:07:37 AM - begin Disk Staging, Start Notify Script
6/25/2012 9:07:37 AM - Info RUNCMD(pid=8960) started            
6/25/2012 9:07:37 AM - Info RUNCMD(pid=8960) exiting with status: 0         
Status 0
6/25/2012 9:07:37 AM - end Disk Staging, Start Notify Script; elapsed time: 00:00:00
6/25/2012 9:07:37 AM - begin Disk Staging, Execute Script
6/25/2012 9:07:37 AM - started process bpbrm (11504)
6/25/2012 9:07:43 AM - end writing
Status 191
6/25/2012 9:07:43 AM - end Disk Staging, Execute Script; elapsed time: 00:00:06
6/25/2012 9:07:43 AM - begin Disk Staging, Stop On Error
Status 0
6/25/2012 9:07:43 AM - end Disk Staging, Stop On Error; elapsed time: 00:00:00
6/25/2012 9:07:43 AM - begin Disk Staging, End Notify Script
6/25/2012 9:07:43 AM - Info RUNCMD(pid=12940) started            
6/25/2012 9:07:43 AM - Info RUNCMD(pid=12940) exiting with status: 0         
Status 0
6/25/2012 9:07:43 AM - end Disk Staging, End Notify Script; elapsed time: 00:00:00
Status 191
6/25/2012 9:07:43 AM - end Parent Job; elapsed time: 00:00:06
no images were successfully processed(191)

Since the data center reboot after power failure i am facing this issue. any solution for this.??

 

Highlighted

These errors can be seen

These errors can be seen throughout Master server's admin log:

<16> free_allocated_resources: Free resource allocations failed

This would say to me that the entire environment could do with clean restart.
At least all NBU processes on master and ALL media servers if you are unable to request reboot.

I see DAG duplication error at 23:58:

23:58:38.418 [9476.12740] <2> db_end: no DONE from db_getreply(): database system error
23:58:38.418 [9476.12740] <16> build_catalog_for_virtual_browse_images: db_IMAGE() failed: database system error (220)
23:58:38.418 [9476.12740] <16> bpduplicate: Status = no images were successfully processed.

23:58:38.418 [9476.12740] <2> db_fork_lock_dbm: Sending Q_IMAGE_FORK_LOCK_DBM
23:58:38.418 [9476.12740] <2> ConnectionCache::connectAndCache: Acquiring new connection for host bs-itu01-001.kacst.ksa, query type 110
23:58:38.434 [9476.12740] <2> logconnections: BPDBM CONNECT FROM 172.16.0.59.50028 TO 172.16.0.59.13721 fd = 880
23:58:38.434 [9476.12740] <8> vnet_check_vxss_client_magic_with_info: [vnet_vxss_helper.c:871] Ignoring VxSS authentication 2 0x2
23:58:38.465 [9476.12740] <2> bpduplicate: Duplicating policy Exch_2010_DAG schedule GRT_FULL backup id KACST-DAG001_1340219513 copy 1 created on 06/20/2012 22:11:53 on source path F:\ExchangeGRT
23:58:38.465 [9476.12740] <16> bpduplicate: Duplicate of backupid KACST-DAG001_1340219513 failed, database system error (220).

I have found this TN where Master was on 7.1 and client on 7.0 : http://www.symantec.com/docs/TECH143796

The solution was to upgrade client to 7.1.
If your entire environment (incl Exchange nodes) is already on 7.5, best to open a support call with Symantec. Above TN will tell which logs will be required by support.

Similar discussion about a year ago that was unfortunately never resolved: http://www.symantec.com/connect/forums/duplication-fails-database-system-error-220

 

**** PS ****

Your last post seems to be totally unrelated to the opening post where Status 25 errors were seen.

The master's admin log does not contain any information about problem mentioned in your opening post.
Are you still seeing status 25's?
If so, we need admin log that covers the date and time when this error is seen.
If not, please tell us how this was resolved.

 

Highlighted

Status Code 190 & 191

Now the status code have changed in duplication job Status code 25 is resolved now as it was some network issue as whole datacenter was rebooted so things take time to become smooth, now the error we are getting is 190 & 191.

 

 

Highlighted

About the errors seen at

About the errors seen at 09:07, see Admin log:

09:07:38.967 [9452.12540] <4> add_to_worklist: Skipping backup id 212.26.44.119_1340110817, it does not contain data.
09:07:38.967 [9452.12540] <4> add_to_worklist: Skipping backup id 212.26.44.119_1340197216, it does not contain data.
09:07:38.982 [9452.12540] <4> add_to_worklist: Skipping backup id 212.26.44.119_1340456424, it does not contain data.
09:07:38.982 [9452.12540] <4> add_to_worklist: Skipping backup id 212.26.44.119_1340542819, it does not contain data.
09:07:38.982 [9452.12540] <2> db_end: Need to collect reply
09:07:38.998 [9452.12540] <4> bpduplicate: found no images or media matching the selection criteria
09:07:38.998 [9452.12540] <2> bpduplicate: EXIT status = 190

Timestamps for backup images for client 212.26.44.119 are:

/root >bpdbm -ctime 1340110817
1340110817 = Tue Jun 19 15:00:17 2012
/root >bpdbm -ctime 1340197216
1340197216 = Wed Jun 20 15:00:16 2012
/root >bpdbm -ctime 1340456424
1340456424 = Sat Jun 23 15:00:24 2012
/root >bpdbm -ctime 1340542819
1340542819 = Sun Jun 24 15:00:19 2012
 

You may want to check what happened with these client backups. Seems backup was started but nothing backed up?

 

 

Highlighted

Backup Time

 

We Change the duplication time previously backup and duplication was going on same time so we are getting this error, now duplication time changes once the backup completed then duplication started. So far now we are not getting this error.

 

Highlighted

We Change the duplication

We Change the duplication time previously backup and duplication was going on same time 

We are back to my questions from a month ago:

How is duplication done? Manual? DSSU? Vault? SLP?

Highlighted

Duplication

Duplication Done with DSSU