cancel
Showing results for 
Search instead for 
Did you mean: 

Restore Job incompletes with status 5

Shelly12
Level 3

Hi,

Facing an issue while doing restore. on client Solaris 10 NBU 6.5,media and master are also on NBU 6.5 with Solaris 10.

getting message:

 Info nbjm (pid=2617) NBU status: 2075, EMM status: Fibre Transport resources are not available
May 9, 2013 1:35:52 AM - Info nbjm (pid=2617) NBU status: 2075, EMM status: Fibre Transport resources are not available
May 9, 2013 1:35:56 AM - connecting
May 9, 2013 1:35:57 AM - connected; connect time: 0:00:00
May 9, 2013 1:35:57 AM - started process bptm (pid=8529)
May 9, 2013 1:36:02 AM - begin reading
May 9, 2013 10:43:28 AM - begin reading
May 9, 2013 10:43:30 AM - end reading; read time: 0:00:02
May 9, 2013 11:01:41 AM - restored from image xxxxxxx.mt.xxx.com_1365892228; restore time: 9:25:49
May 9, 2013 11:01:41 AM - Error bpbrm (pid=8527) client restore EXIT STATUS 5: the restore failed to recover the requested files
May 9, 2013 11:01:41 AM - Warning bprd (pid=20309) Restore must be resumed prior to first image expiration on June 14, 2013 06:30:28 PM EDT

Job incompletes with status 5.All images are on disk.

Please suggest.

15 REPLIES 15

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

 On the master server, open Host Properties -> Master Server -> Properties

Change the 'Fiber Transport' parameter to "Never" and restart NBU

Shelly12
Level 3

I have attached screenshot of master host properties.I didnt seem to find the option that you mentioned...

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Try to change it from cmd on the master server:

 

/usr/openv/netbackup/bin/admincmd/nbftconfig -setconfig -ftpref never

Nicolai
Moderator
Moderator
Partner    VIP   

From the Netbackup GUI run the "Reports" run the "All Log Entries".

 

Shelly12
Level 3

hi,

 

@Marianne-Please find the image..it says that SAN Client Option isnt licensed after running the command

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

NBU status: 2075, EMM status: Fibre Transport resources are not available

Do you perhaps know why NBU is looking for FT resources?

Did you perhaps run NBU at some point in time with demo keys?

One last thought:

Are you perhaps trying to restore via a different media server? Maybe a FORCE_RESTORE.... entry in bp.conf on the master server?

Different error codes, but similar Fibre Transport error with disk storage units: 
http://www.symantec.com/docs/TECH55584

 

Shelly12
Level 3

Do you perhaps know why NBU is looking for FT resources?--No

Also the backups images are on the same media server from where we are trying to do restore

it keeps on mounting and then eventually fails.....

 Its a disk based restore...

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

hi ,

provide the restore log form the location /usr/openv/netbackup/logs/user_ops/<userid>/logs/

it will give some hints for restore failurs

and also you need to check bpbrm log with PID bpbrm (pid=8527) 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

it keeps on mounting and then eventually fails..... 

Really? With disk based backup?
Where do you see mount attempt?
Is the disk storage not already mounted?

The only thing left is to look at logs - especially bptm.
bpbrm may also be helpful.
Please post logs as File attachments.

I cannot even suggest that you log a Support call, as NBU 6.5 ran out of support in Oct 2012...

 

Shelly12
Level 3

@marianne--In activity monitor i can see it says mounting under operation..

Disk storage is mounted since we are able to take backups on it successfully.

@nagalla-i have pasted the logs for bpbrm as well as for bptm.

 

Please check.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Where were logs collected? 

There is no record of image ......1365892228 in the bptm log.Or any restore attempt for that matter.

We need the bptm and bpbrm logs from the media server that attempted the restore that will contain the pid's seen in Activity Monitor:

May 9, 2013 1:35:57 AM - started process bptm (pid=8529)
May 9, 2013 11:01:41 AM - Error bpbrm (pid=8527)

bprd on master server may also be helpful. It will tell us which media server the restore is directed to.

Shelly12
Level 3

We initiated another restore job and it failed after writing few files iwth description

May 10, 2013 2:38:20 AM - begin Restore
May 10, 2013 2:41:31 AM - restoring from image rst6802b.mt.att.com_1365892228
May 10, 2013 2:41:31 AM - requesting resource TRANSPORT
May 10, 2013 2:41:31 AM - Info nbjm (pid=2617) NBU status: 2075, EMM status: Fibre Transport resources are not available
May 10, 2013 2:41:31 AM - Info nbjm (pid=2617) NBU status: 2075, EMM status: Fibre Transport resources are not available
May 10, 2013 2:41:35 AM - connecting
May 10, 2013 2:41:36 AM - connected; connect time: 0:00:00
May 10, 2013 2:41:37 AM - started process bptm (pid=7104)
May 10, 2013 2:41:41 AM - begin reading
May 10, 2013 9:02:43 AM - begin reading
May 10, 2013 9:02:45 AM - end reading; read time: 0:00:02
May 10, 2013 9:21:09 AM - restored from image xxxx.xx.com_1365892228; restore time: 6:39:38
May 10, 2013 9:21:09 AM - Warning bprd (pid=5276) Restore must be resumed prior to first image expiration on June 14, 2013 06:30:28 PM EDT
May 10, 2013 9:21:09 AM - Error bpbrm (pid=7103) client restore EXIT STATUS 5: the restore failed to recover the requested files
May 10, 2013 9:21:09 AM - end Restore; elapsed time 6:42:49
the restore failed to recover the requested files  (5)

 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

We need bptm and bpbrm logs from media server that attempted the restore.

If bprd log exists om the master, please post that as well.

Shelly12
Level 3

I have posted this since the full bptmlog was too large to be posted.Please check if this helps

 

more log.050913 | grep rst6802b.mt.att.com
12:54:56.552 [14512] <2> bptm: INITIATING (VERBOSE = 5): -r -c rst6802b.mt.att.com -cl rst6802b -bt 1365892228 -b rst6802b.mt.att.com_1365892228 -st 0 -mud 180 -dpath /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st -hostname rst6802b.mt.att.com -L /usr/openv/netbackup/logs/user_ops/sh401j/logs/jbp-15592368118265193392000000012-CRaaDE.log -ru root -rclnt rsprd416-ebr.mt.att.com -rclnthostname rsprd416-ebr.mt.att.com -firstblk 7702089 -restoreid 265019.001 -v -lcmsg en_US -lctime en_US -no_callback -connect_options 0x01010100 -jobid 265019 -masterversion 650000 -flport 0 -flipc /tmp/vnet-19188368118490695831000000002-hzaGEL -S rsprd416-ebr.mt.att.com
12:54:59.312 [14512] <4> report_client: VBRC 2 14512 0 rst6802b.mt.att.com_1365892228 -1 rst6802b 0 *NULL* 0 1 1
12:55:00.343 [14512] <2> log_frag_info: frag 1 file 0 of /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st/rst6802b.mt.att.com_1365892228_C1_F1 has blocks 0 to 143746495 for rst6802b.mt.att.com_1365892228
12:55:00.344 [14512] <2> read_backup: id = /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st/rst6802b.mt.att.com_1365892228_C1_F1
12:55:00.349 [14512] <2> read_backup: path /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st/rst6802b.mt.att.com_1365892228_C1_F1 (71873248 Kbytes) being considered for restore
12:55:00.831 [14512] <2> construct_sts_isid: master_server rsprd416-ebr.mt.att.com, client rst6802b.mt.att.com, backup_time 1365892228, copy_number 1, stream_number 0, fragment_number 1, resume_number 0
12:55:00.831 [14512] <2> bp_sts_open_image: STSBasicDisk:rsprd180g.mt.att.com /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st rst6802b.mt.att.com_1365892228_C1_F1 2 0 1 sth 0 2 8 LSU 0x7 0x1 0x2 saveas 0x4
12:55:00.894 [14512] <2> io_open_disk: file rst6802b.mt.att.com_1365892228_C1_F1 successfully opened
12:55:03.021 [14512] <4> read_backup: begin reading backup id rst6802b.mt.att.com_1365892228 (restore), copy 1, fragment 1, from media id  on drive  (index -1)
12:55:03.022 [14512] <2> read_backup: id = /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st/rst6802b.mt.att.com_1365892228_C1_F1

 

bpbrm log

 

 more log.050913 | grep rst6802b.mt.att.com
12:54:52.191 [14511] <2> logparams: -restore -mt 0 -k 0 -c rst6802b.mt.att.com -hostname rst6802b.mt.att.com -rclnt rsprd416-ebr.mt.att.com -rclnthostname rsprd416-ebr.mt.att.com -browse_clnt rst6802b.mt.att.com -ru root -firstblk 7702089 -cl rst6802b -bt 1365892228 -st 0 -secure 1 -b rst6802b.mt.att.com_1365892228 -cn 1 -rst 1368118268 -to 0 -mud 180 -ct 0 -S rsprd416-ebr.mt.att.com -masterversion 650000 -backup_copy 0 -latest_image -flport 0 -flipc /tmp/vnet-19188368118490695831000000002-hzaGEL -p /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st -rg root -cv -L /usr/openv/netbackup/logs/user_ops/sh401j/logs/jbp-15592368118265193392000000012-CRaaDE.log -clnt_lc_messages en_US -clnt_lc_time en_US -clnt_lc_ctype en_US -clnt_lc_collate en_US -clnt_lc_numeric en_US -connect_options 0x01030202 -jobid 265019 -restoreid 265019.001 -job_total 1
12:54:52.197 [14511] <2> bpbrm start_bpcd_stat: bpbrm.c.19676: bpcd_client: rst6802b.mt.att.com
12:54:52.197 [14511] <2> bpbrm start_bpcd_stat: bpbrm.c.19679: bpcd_client_hostname: rst6802b.mt.att.com
12:54:52.204 [14511] <2> hosts_equal: Comparing hosts <rst6802b.mt.att.com> and <rst6802b.mt.att.com>
12:54:52.885 [14511] <2> verify_client: bpbrm.c.33390: *client_hostname: rst6802b.mt.att.com
12:54:52.897 [14511] <2> hosts_equal: Comparing hosts <rst6802b.mt.att.com> and <rsprd416-ebr.mt.att.com>
12:54:52.897 [14511] <2> hosts_equal: host1 rst6802b.mt.att.com rst6802b.mt.att.com addr 135.21.126.27 (0x87157e1b)
12:54:52.898 [14511] <2> hosts_equal: name1=rst6802b.mt.att.com  name2=rsprd416-ebr.mt.att.com
12:54:52.898 [14511] <2> hosts_equal: hostbuf=rst6802b.mt.att.com  hostentry->h_name=rsprd416-ebr.mt.att.com
12:54:52.898 [14511] <2> hosts_equal: there were NO aliases for rst6802b.mt.att.com
12:54:52.898 [14511] <2> hosts_equal: rst6802b.mt.att.com=0x87157e1b rsprd416-ebr.mt.att.com=0x87198d3b
12:54:52.898 [14511] <2> hosts_equal: Comparing hosts <rst6802b.mt.att.com> and <rsprd180g.mt.att.com>
12:54:52.899 [14511] <2> hosts_equal: host1 rst6802b.mt.att.com rst6802b.mt.att.com addr 135.21.126.27 (0x87157e1b)
12:54:52.899 [14511] <2> hosts_equal: name1=rst6802b.mt.att.com  name2=rsprd180g.mt.att.com
12:54:52.899 [14511] <2> hosts_equal: hostbuf=rst6802b.mt.att.com  hostentry->h_name=rsprd180g
12:54:52.899 [14511] <2> hosts_equal: compare rst6802b.mt.att.com with alias rsprd180g.mt.att.com
12:54:52.899 [14511] <2> hosts_equal: compare rst6802b.mt.att.com with alias loghost
12:54:52.899 [14511] <2> hosts_equal: there were NO aliases for rst6802b.mt.att.com
12:54:52.899 [14511] <2> hosts_equal: rst6802b.mt.att.com=0x87157e1b rsprd180g.mt.att.com=0x8710d162
12:54:52.900 [14511] <2> bpbrm start_bpcd_stat: bpbrm.c.19676: bpcd_client: rst6802b.mt.att.com
12:54:52.900 [14511] <2> bpbrm start_bpcd_stat: bpbrm.c.19679: bpcd_client_hostname: rst6802b.mt.att.com
12:54:52.900 [14511] <2> hosts_equal: Comparing hosts <rst6802b.mt.att.com> and <rst6802b.mt.att.com>
12:54:53.303 [14511] <2> bpbrm main: restore command = /usr/openv/netbackup/bin/tar tar -x -v -Y -p -P -I 1368118268 -U 0 -Q -J clnt_lc_messages=en_US -J clnt_lc_time=en_US -J clnt_lc_ctype=en_US -J clnt_lc_collate=en_US -J clnt_lc_numeric=en_US -J restoreid=265019.001 -J job_total=1 -J client=rst6802b.mt.att.com -J requesting_client=rsprd416-ebr.mt.att.com -J browse_client=rst6802b.mt.att.com -f - -J backup_time=1365892228 -L /usr/openv/netbackup/logs/user_ops/sh401j/logs/jbp-15592368118265193392000000012-CRaaDE.log
12:54:55.902 [14511] <2> bpbrm main: read tar start message from rst6802b.mt.att.com
12:54:55.903 [14511] <2> bpbrm spawn_child: /usr/openv/netbackup/bin/bptm bptm -r -c rst6802b.mt.att.com -cl rst6802b -bt 1365892228 -b rst6802b.mt.att.com_1365892228 -st 0 -mud 180 -dpath /opt/app/ebr/rsprd180g/nfs_stu004/nbu_dsu_st -hostname rst6802b.mt.att.com -L /usr/openv/netbackup/logs/user_ops/sh401j/logs/jbp-15592368118265193392000000012-CRaaDE.log -ru root -rclnt rsprd416-ebr.mt.att.com -rclnthostname rsprd416-ebr.mt.att.com -firstblk 7702089 -restoreid 265019.001 -v -lcmsg en_US -lctime en_US -no_callback -connect_options 0x01010100 -jobid 265019 -masterversion 650000 -flport 0 -flipc /tmp/vnet-19188368118490695831000000002-hzaGEL -S rsprd416-ebr.mt.att.com
12:55:04.574 [14511] <2> bpbrm main: from client rst6802b.mt.att.com: CKP - 265019.001 1368118503 0 0 2 1 0 0 39 /siebel78/jdk-1_5_0_22-solaris-sparc.sh
 

 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Please!  we need logs without the greps for the client name!

In bptm we need all entries for pid  [14512] 

In bpbrm we need all entries for pid [14511]

Please post logs as File attachments.