Restore files not seen on catalog from tape

[coffee_drinker]

The backup job failed at 95% and therefore the files are not showing on the catalog. I know there are a lot of files and I nned to view and restore some of those files from the tape.

How do I view/list files saved onto specific tape backed up from server A?

If files are on that tape, how do I restore those files back to server A or any other?

Thanks for the help,

Mark

[Ray_Esperanzate]

Unfortunately you may be out of luck since there is no record of the backup in the catalog.  Also, since it fails at 95% chances are the tar archive on the tape may be incomplete. 

However, because it is a tar archive, you may be able to do the following:

http://www.symantec.com/business/support/index?page=content&id=TECH6501

Obviously its not a supported method, but if it's your last resort, it won't hurt to give it a try.

[coffee_drinker]

I just ran Tape Contents and seems like there are blocks of data from that job that failed; how do I see what's inside them?

[mph999]

Take a step back, what did the job fail with ? Look in the failed job details in activity monitor. bpmedialist -mcontents -m will load and read the tape, I think this is effectively what you did in the GUI report - this shows the backup headers on the tape. The backup id shows the client and ctime - you could look in (on master): /usr/openv/netbackup/db/images// Here you wlll see some files, if you are at a version of NBU below 7.5 there will be two files that have the ctime as part of the filename _ _.f We are only concerned with the .f file cat_convert -dump .f Will show the contents of the backup, but if you have TIR this output is not quite as it looks, if TIR is not used when what is listed in the output is what was backed up. Another way ... From http://www.symantec.com/docs/TECH145113 bplist -R 99 -C -l -k -X -s -e -I "/" You probably don't need the -k (not sure, try and see) If the backups are not multiplexed, and you are using linux/ unix media server it might be possible to read the tape using the netbackup tar command in /usr/openv/netbackup/bin/tar martin

[Ray_Esperanzate]

I think you will have to go the tar route that is detailed in the previous link i posted.  Unless someone else chimes in with a better way. 

[mph999]

I think we really need some moe details before offering 'tar' as a solution. The reason for the failure may be fixable, once we know what it is - then again, it may not be. Need the os version also, I don't think you'll get the tar method to work on windows for a start ... You never know, re-running the restore might work, or using a different media server of tape drive ... who knows ... M

[Nicolai]

Can't you just "resume" the orginal job and let it finish ?

In order for Netbackup to resume a failed job the "take check point every X minutes" need to be check in the policy.

http://www.symantec.com/docs/HOWTO34475

Database backup can't be resumed - only file system backups.

[Ray_Esperanzate]

mph999, i was going directly to the tar method because from what the original poster said, the initial backup failed at 95%, therefore no catalog entry, and no way (that i know of) to initiate a restore from NBU.

Also, i doubt he can resume the original job, according to the screenshot above, the backup was taken on June 9th.

[coffee_drinker]

Ray, you are obsolutely right. I cannot resume the job and the user requested files from that month and I'm hoping these files are on that tape.

From the screen shot above, you can tell I have few files from that server that failed to complete and yes, I could not resume that job. How do I open these files to see if the requested files are there and how to restore them to disk.

More info:

NetBackup 6.5.6 on Windows 2008 R2

[mph999]

The actual error you see would really be useful so we can see where we are with this issue. NBU uses tar to read the tape, so if this fils due to an unreadable tape, then it will fail if run manually. If the tape is multiplexed, then it is not possibe to even attempt to read the tape using the command line tar - so as you see, perhaps Ray is correct, but we need to understand exaactly what the problem is. Martin

[coffee_drinker]

8/8/2013 10:42:52 AM - requesting resource pa-bu-02-hcart2-robot-tld-0
8/8/2013 10:42:52 AM - requesting resource pa-bu-02.NBU_CLIENT.MAXJOBS.pa-dfs-02
8/8/2013 10:42:52 AM - requesting resource pa-bu-02.NBU_POLICY.MAXJOBS.dfs
8/8/2013 10:42:52 AM - granted resource pa-bu-02.NBU_CLIENT.MAXJOBS.pa-dfs-02
8/8/2013 10:42:52 AM - granted resource pa-bu-02.NBU_POLICY.MAXJOBS.dfs
8/8/2013 10:42:52 AM - granted resource 000484
8/8/2013 10:42:52 AM - granted resource QUANTUM.ULTRIUM5.000
8/8/2013 10:42:52 AM - granted resource pa-bu-02-hcart2-robot-tld-0
8/8/2013 10:42:52 AM - estimated 19786972 kbytes needed
8/8/2013 10:42:52 AM - started process bpbrm (1524)
8/8/2013 10:42:59 AM - connecting
8/8/2013 10:42:59 AM - mounting 000484
8/8/2013 10:43:01 AM - connected; connect time: 00:00:02
8/8/2013 10:44:20 AM - mounted; mount time: 00:01:21
8/8/2013 10:44:26 AM - positioning 000484 to file 3
8/8/2013 10:44:41 AM - positioned 000484; position time: 00:00:15
8/8/2013 10:44:41 AM - begin writing
8/8/2013 12:11:48 PM - Error bpbrm(pid=4408) socket read failed, An existing connection was forcibly closed by the remote host.  (10054)
8/8/2013 12:11:48 PM - Error bptm(pid=5568) socket operation failed - 10054 (at child.c.1294)     
8/8/2013 12:11:48 PM - Error bpbrm(pid=4408) could not send server status message      
8/8/2013 12:11:48 PM - Error bptm(pid=5568) unable to perform read from client socket, connection may have been broken
8/8/2013 12:13:28 PM - end writing; write time: 01:28:47
file read failed(13)

This specific job fails pretty often and above you will find error from August 8th (daily differential job)

[mph999]

Hmm, hopefully that is a network issue ... and not the case where things time out due to a tape read issue ...

Think we'll need the following logs pref at verbose 5 (and general 2 if windows)

Media server

bptm

bpbrm

Client 

tar

Martin

[coffee_drinker]

I have other jobs running on weekend that run for over 24 hours and are fine; for example: exchange, san or oracle. So I guess it is not tape problem.

It fails every other day when backing DFS server (windows server 2008 r2 std).

[coffee_drinker]

will taking check points work for exchange and DFS?

What about "Shadow Copy Components" and DFS?

[coffee_drinker]

Is it possible to disable "DFS Replication" service using netbackup policy before that policy starts? Manually I stopped DFS service and the backup job on netbackup ran much faster and seems like it is easier to browse/restore files if needed using BAR

thanks,

M

[Bmitche]

The way we handle our DFS replicated servers is..... we stop the dfs replication service via the "bpstart_notify.bat" and "bpend_notify.bat" files. Net stop dfsr & Net start dfsr. We started backing them up via the "Shadow Copy Components" but we did not want to backup the data being replicated "to" this server.

Using "Shadow Copy Components" you cannot exclude folders from the backup.

Most of our DFS replication servers are to and from servers....since the from data was already being backed up at the source location it did not need to be backed up again on the target.

When the backup starts it runs bpstart_notify.bat and stops the dfs replication services. We backup the drive letter that contains the data we want and use exclude to not backup the folders we don't want.

We have the dfs replication service set to restart the service after 30 minutes on the service recovery tab.....as long as the backup has already started it does not matter if the service starts while the backup is already running. Hope this all makes since....if not.....ask for more clarification.

[coffee_drinker]

Your solution seems like what I need; could you please explain in more details on how to use/implement "bpstart_notify.bat" and "bpend_notify.bat" files. Net stop dfsr & Net start dfsr?

Thank you for all your help,

Mark

[Bmitche]

On the server that is running DFS replication...agent install path\Veritas\NetBackup\bin. Edit or create "bpstart_notify.bat". Within this batch file put "NET STOP DFSR". Edit or create "bpend_notify.bat". Within this batch file put "NET START DFSR"

bpstart_notify runs when the backup of the client starts(right before the backup starts).

bpend_notify runs when the backup completes.

You can add whatever other stuff you want to the batch files....send emails...whatever.

We have been doing it like this for over a year and it's working well.