cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Veritas.com
Support

Catalog backup stuck [NBU 7.7.3]

dpx_99
Level 4

‎03-02-2017 01:06 AM - edited ‎03-02-2017 01:09 AM

Hi Experts,

catalog [diff] backup is stuck while child jobs are completed but paraent job couldn't completed. Is there any solution available for this issue?

Netbackup is installed on windows 2008 standard R2 server.

Thanks,

dpx

0 Kudos
Reply
6 REPLIES 6

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

‎03-02-2017 01:18 AM

You had the same problem 3 weeks ago, marked solution without replying to attempts to help:
https://vox.veritas.com/t5/NetBackup/Catalog-backup-session-hung/m-p/825966

Please copy the text in parent job and all of the child jobs and save in a .txt file.

Please upload the txt file here.

 


Handy NetBackup Links
0 Kudos
Reply

dpx_99
Level 4
In response to Marianne

‎03-02-2017 01:56 AM

Hi Marianne,

Yes, you are right - after cancelling the session ,it started to work again. but now i see that differential backup creating problem and stuck.

Below are details from session windows.

03/02/2017 09:53:36 - Info nbjm (pid=9876) starting backup job (jobid=2032347) for client master.nbu, policy catalog, schedule Differential-Inc
03/02/2017 09:53:36 - Info nbjm (pid=9876) requesting CATALOG_BACKUP_RESOURCE resources from RB for backup job (jobid=2032347, request id:{B0F500B3-D782-4E15-9064-D76F2D0C4750})
03/02/2017 09:53:36 - requesting resource master.nbu .NBU_CATALOG.MAXJOBS
03/02/2017 09:53:36 - granted resource  master.nbu NBU_CATALOG.MAXJOBS
03/02/2017 09:54:45 - estimated 0 kbytes needed
03/02/2017 09:54:45 - begin Parent Job
03/02/2017 09:54:45 - begin Catalog Backup: Start Notify Script
03/02/2017 09:54:46 - Info RUNCMD (pid=1712) started
03/02/2017 09:54:46 - Info RUNCMD (pid=1712) exiting with status: 0
Operation Status: 0
03/02/2017 09:54:46 - end Catalog Backup: Start Notify Script; elapsed time 0:00:01
03/02/2017 09:54:46 - begin Catalog Backup: Database Manager Query                     ##stuck here......

0 Kudos
Reply

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
In response to dpx_99

‎03-02-2017 06:00 AM

You say child jobs are completed .... ?

Please post the text in Details tab of the completed child jobs.


Handy NetBackup Links
0 Kudos
Reply

dpx_99
Level 4
In response to Marianne

‎03-02-2017 11:50 PM

Yes, child jobs status is completed.

Please find logs from all sessions. i noticed some error on 2nd child session.


############################## First Child session #########################################

03/02/2017 14:58:41 - Info bpdbm (pid=14096) staging relational database files for catalog backup
03/02/2017 14:58:41 - Info bpdbm (pid=14096) staging NBAZDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 14:58:42 - Info bpdbm (pid=14096) done staging NBAZDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 14:58:42 - Info bpdbm (pid=14096) staging NBDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 14:59:17 - Info bpdbm (pid=14096) done staging NBDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 15:01:16 - Info bpdbm (pid=14096) validating NBAZDB backup in D:\Veritas\NetBackupDB\staging
03/02/2017 15:01:18 - Info bpdbm (pid=14096) done validating NBAZDB backup in D:\Veritas\NetBackupDB\staging
03/02/2017 15:01:18 - Info bpdbm (pid=14096) validating NBDB backup in D:\Veritas\NetBackupDB\staging
03/02/2017 15:05:36 - Info bpdbm (pid=14096) done validating NBDB backup in D:\Veritas\NetBackupDB\staging
the requested operation was successfully completed  (0)

########################### 2nd child session ###############################################


03/02/2017 15:06:23 - Info nbjm (pid=9876) starting backup job (jobid=2032819) for client master.nbu, policy catalog, schedule Differential-Inc
03/02/2017 15:06:23 - Info nbjm (pid=9876) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=2032819, request id:{9F419C09-9E16-4B1A-BDA1-C628E771B4EC})
03/02/2017 15:06:23 - requesting resource Media.nbu-disk
03/02/2017 15:06:23 - requesting resource master.nbu.NBU_CLIENT.MAXJOBS.master.nbu
03/02/2017 15:06:23 - requesting resource master.nbu.NBU_POLICY.MAXJOBS.catalog
03/02/2017 15:06:23 - granted resource  master.nbu.NBU_CLIENT.MAXJOBS.master.nbu
03/02/2017 15:06:23 - granted resource  master.nbu.NBU_POLICY.MAXJOBS.catalog
03/02/2017 15:06:23 - granted resource  MediaID=@aaaaI;DiskVolume=E:\;DiskPool=Media.nbu-disk;Path=E:\;StorageServer=Media.nbu;MediaServer=Media.nbu
03/02/2017 15:06:23 - granted resource  Media.nbu-disk
03/02/2017 15:06:24 - estimated 0 kbytes needed
03/02/2017 15:06:24 - Info nbjm (pid=9876) started backup (backupid=master.nbu_1488463583) job for client master.nbu, policy catalog, schedule Differential-Inc on storage unit Media.nbu-disk
03/02/2017 15:06:26 - started process bpbrm (pid=14580)
03/02/2017 15:06:28 - Info bpbrm (pid=14580) master.nbu is the host to backup data from
03/02/2017 15:06:28 - Info bpbrm (pid=14580) reading file list for client
03/02/2017 15:06:30 - connecting
03/02/2017 15:06:32 - Info bpbrm (pid=14580) starting bpbkar32 on client
03/02/2017 15:06:32 - connected; connect time: 0:00:00
03/02/2017 15:06:34 - Info bpbkar32 (pid=12704) Backup started
03/02/2017 15:06:34 - Info bpbkar32 (pid=12704) change time comparison:<disabled>
03/02/2017 15:06:34 - Info bpbkar32 (pid=12704) archive bit processing:<enabled>
03/02/2017 15:06:35 - Info bptm (pid=11064) start
03/02/2017 15:06:35 - Info bptm (pid=11064) using 262144 data buffer size
03/02/2017 15:06:35 - Info bptm (pid=11064) setting receive network buffer to 1049600 bytes
03/02/2017 15:06:35 - Info bptm (pid=11064) using 30 data buffers
03/02/2017 15:06:37 - Info bptm (pid=11064) start backup
03/02/2017 15:06:39 - Info bptm (pid=11064) backup child process is pid 15248.15296
03/02/2017 15:06:39 - Info bptm (pid=15248) start
03/02/2017 15:06:39 - begin writing
03/02/2017 15:08:41 - Info bptm (pid=11064) waited for full buffer 445 times, delayed 7829 times
03/02/2017 15:08:56 - Info bptm (pid=11064) EXITING with status 0 <----------
03/02/2017 15:10:43 - Error bpbrm (pid=14580) could not write KEEPALIVE to COMM_SOCK
03/02/2017 15:10:44 - Info bpbrm (pid=14580) validating image for client master.nbu
03/02/2017 15:11:51 - Error bpbrm (pid=14580) could not send server status message
03/02/2017 15:11:54 - Info bpbkar32 (pid=12704) done. status: 0: the requested operation was successfully completed
03/02/2017 15:11:54 - end writing; write time: 0:05:15
the requested operation was successfully completed  (0)

###################################### 3rd child session ##########################################

03/02/2017 14:58:41 - Info bpdbm (pid=14096) staging relational database files for catalog backup
03/02/2017 14:58:41 - Info bpdbm (pid=14096) staging NBAZDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 14:58:42 - Info bpdbm (pid=14096) done staging NBAZDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 14:58:42 - Info bpdbm (pid=14096) staging NBDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 14:59:17 - Info bpdbm (pid=14096) done staging NBDB backup to D:\Veritas\NetBackupDB\staging
03/02/2017 15:01:16 - Info bpdbm (pid=14096) validating NBAZDB backup in D:\Veritas\NetBackupDB\staging
03/02/2017 15:01:18 - Info bpdbm (pid=14096) done validating NBAZDB backup in D:\Veritas\NetBackupDB\staging
03/02/2017 15:01:18 - Info bpdbm (pid=14096) validating NBDB backup in D:\Veritas\NetBackupDB\staging
03/02/2017 15:05:36 - Info bpdbm (pid=14096) done validating NBDB backup in D:\Veritas\NetBackupDB\staging
the requested operation was successfully completed  (0)

###################################### Parent session #################################################

03/02/2017 14:57:29 - Info nbjm (pid=9876) starting backup job (jobid=2032804) for client master.nbu, policy catalog, schedule Differential-Inc
03/02/2017 14:57:29 - Info nbjm (pid=9876) requesting CATALOG_BACKUP_RESOURCE resources from RB for backup job (jobid=2032804, request id:{3CA57FBB-9CEB-4637-8F6E-2D29751B315C})
03/02/2017 14:57:29 - requesting resource master.nbu.NBU_CATALOG.MAXJOBS
03/02/2017 14:57:29 - granted resource  master.nbu.NBU_CATALOG.MAXJOBS
03/02/2017 14:58:40 - estimated 0 kbytes needed
03/02/2017 14:58:40 - begin Parent Job
03/02/2017 14:58:40 - begin Catalog Backup: Start Notify Script
03/02/2017 14:58:40 - Info RUNCMD (pid=12616) started
03/02/2017 14:58:41 - Info RUNCMD (pid=12616) exiting with status: 0
Operation Status: 0
03/02/2017 14:58:41 - end Catalog Backup: Start Notify Script; elapsed time 0:00:01
03/02/2017 14:58:41 - begin Catalog Backup: Database Manager Query    ################# it is stuck here

 

 

0 Kudos
Reply

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
In response to dpx_99

‎03-03-2017 01:51 AM

You have posted the exact same information for First Child session and 3rd child session.

Please have another look. 

To track comms issues between processes, you need verbose logging (level 3 should be fine) of the following:
bpdbm, bpbrm, bpbkar

One more thing - have you had a look at this section in NetBackup Backup Planning and Performance Tuning Guide  ?
OS-related tuning factors for Windows

Lack of tuning at OS-level can lead to all sorts of weird issues...


Handy NetBackup Links
0 Kudos
Reply

mph999
Level 6
Employee Accredited
In response to Marianne

‎03-03-2017 01:53 AM

Just to add to Mariaanes suggestions, for comms issues I would consider turning up vx log numbers 156 and 137.

These do not create seperate log files, the lines go into the logs of the processes.   If the processis a unified, or 'vx' process, you need -i <oid> to view the extra lines, not the standard -o <oid>

Reply