Copy of image removed from media server while RESTORE is running

[jbgeis]

We are running Netbackup 7.5.0.6 on a SuSE 2.6.16 Linux server. Backing up a couple of hundred servers (Linux, Solaris10, AIX, etc.)

One Netbackup Master server. One Netbackup Media server.

We have run into a situation where our database people run a backup of a production database. The backup (~800 Gb's) is stored on the media server in the basic disk storage unit (/dssu01). In the middle of the night, a restore is started that will load that backup into one of the development database servers.

Because the /dssu01 volume is quite full at this point, a duplication then copies the database backup from the temporary storage unit /dssu01 to tape.

The duplication completes successfully, THEN NetBackup deletes the "copy 1" image from the /dssu01 volume on the media server. Unfortunately, the RESTORE is still running! The RESTORE then abends with a "(83) media open error" because the image file it was using for the restore has disappeared!

This has happened a number of times. The RESTORE can be run again the next day and will complete successfully, because it will then mount the tape containing "copy 2" of the image. However, as this is an eight-hour job, the clients are more than a little annoyed at the delay. This only appears to happen to backups that are stored on the Media server. Those backups stored on the Master server are not interrupted, as it appears that the Master server is smart enough to realized that the image on it's basic disk storage unit is in use and doesn't delete it until it is finished. Alas, the Media server doesn't seem to communicate the same information to the Master server, so the image file gets removed by the Staging processes.

Has anyone else come across this? Either I need a setting that will prevent NetBackup from removing an image that is being used, or I need some command that will put a 'hold' on the image and prevent it from being deleted. I've hunted through the websites, but so far, have had no luck with it.

Thanks.

- jbgeis

[revarooo · Accepted solution]

The only thing I can think of is to add more disk space to this dssu and ensure the water marks are set appropriately so that when it duplicates the images it doesn't then run a cleanup on relatively new backups.

[Marianne]

I agree - add more space. Seems the sizing for the DSU was done incorrectly or else data has outgrown the backup disk. There should be enough space on disk to hold at least 2 full backups. Space management of DSU works strictly with High and Low water marks. The NBU admin selects the HWM at which NBU should start deleting duplicated images. Oldest images are deleted first. So, the only way to prevent this situation is to ensure HWM is not reached while restore is in progress.

[mph999]

7.6 offers a solution.  SLP with Windows, so you could backup, restore, then duplicate then expire, controlling the secondary SLP operation (the duplication) with an SLP window.

[Marianne]

Please be aware that in order to use SLPs you need Advanced or Dedupe disk for which a capacity type license is needed.

[jbgeis]

Thank you both for your advice.

We have been pushing >97% on the available space in the dssu volumes each night, so we are definitely at the point where more space is required. Additional disk trays had been ordered, and they showed up in the last week.

We have now added more space to the servers, creating a new dssu volume on both the master server and the media server. Hopefully, this will resolve the issue.

I will look into the SLP solutions, but as we are still running 7.5.0.6 it won't help at this time.

Thanks again.