Offline catalog backup fails with error 124

[ramkr2020]

Hi All,

Offline catalog backup is continously failing since 31st Jan in our netbackup environment, below is error which we are receiveing at the end of catalog backup. Please advice. 

/6/2014 12:17:53 AM - begin Catalog Backup

2/6/2014 1:14:02 AM - end Catalog Backup; elapsed time: 00:56:09

NB database backup failed, a path was not found or is inaccessible(124)

2/6/2014 1:14:02 AM - Error bpbackupdb(pid=9532) Offline catalog backup to path \\server\Apps\veritascatalog FAILED      

[ramkr2020]

Below is the Error: on the hotling catalog backup logs is

2/11/2014 5:03:45 AM - begin writing

2/11/2014 5:03:45 AM - positioned L442L4; position time: 00:00:02

2/11/2014 5:41:35 AM - Error bpbrm(pid=20996) from client apsydbu01: ERR - failure reading file: C:\Program Files\VERITAS\NetBackup\db\images\apsydnd01\1213000000\NT-Servers_1213181070_FULL.f (WIN32 1: Incorrect function. )

2/11/2014 5:48:13 AM - end writing; write time: 00:44:28

the requested operation was partially successful(1)

The job was successfully completed, but some files may have been

busy or unaccessible. See the problems report or the client's logs for more details.

Offline catalog backup activity logs:

2/11/2014 1:42:03 AM - begin Catalog Backup

2/11/2014 2:37:59 AM - end Catalog Backup; elapsed time: 00:55:56

NB database backup failed, a path was not found or is inaccessible(124)

2/11/2014 2:37:59 AM - Error bpbackupdb(pid=20584) Offline catalog backup to path \\apsydfs01\Apps\veritascatalog FAILED      

Attached the bpbar and bpdm log of master/media server. 

[Marianne]

Seems problem starts with this image file:

1:42:07.579 AM: [22388.35072] <4> backup_create: ......
2:27:23.166 AM: [22388.35072] <2> tar_base::V_vTarMsgW: WRN - error reading file: C:\Program Files\VERITAS\NetBackup\db\images\apsydnd01\1213000000\NT-Servers_1213181070_FULL.f (WIN32 1: Incorrect function. )
2:27:23.166 AM: [22388.35072] <2> tar_base::V_vTarMsgW: WRN - error reading byte: 305152
2:27:23.166 AM: [22388.35072] <2> tar_base::V_vTarMsgW: WRN - wanted buffer size: 32768
2:27:23.322 AM: [22388.35072] <16> dtcp_write: TCP - failure: send socket (1832) (TCP 10054: Connection reset by peer)

Seems this is an image file dating back to 2008:
UNIX time 1213181070 is 06/11/2008 10:44am

At the same time, it also seems as if the network connection got broken:

dtcp_write: TCP - failure: send socket (1832) (TCP 10054: Connection reset by peer)
dtcp_write: TCP - failure: attempted to send 172 bytes

Suggestions from my side:

1. Check database consistency with 'bpdbm -consistency 2 move' (See http://www.symantec.com/docs/HOWTO43651 )

2. As a test, find a USB hard drive, connect it to the master, configure it as Disk storage unit, change catalog backup to this local drive. If this works, you need to find out what is wrong with the network share (not NBU issue...)

3. UPGRADE your environment!