Hi, I have a setup that is using the Shared Storage Option and started having problems with the B2T that is asking me to insert overwritable media when the jobs on both media servers have been configured with the option "Append to media, overwrite if no appendable media is available".
The setup is as follows:
2x Backup Servers running Backup Exec 2010 R3 on Windows 2008 R2 (BK1 - Primary SSO and BK2 - Secondary SSO)
1x HP MSL2024
The media servers are configured to backup 6 servers each with each server having its own deciated job. The backups are done as B2D and then B2T.
The backup servers themselves are also backed up through a seperate backup job and they are backed directly to tape.
The issue I am having is a bit random but normally the first job (Backup server to tape) runs succesfull, but then the second or third job is queued and prompts to insert ovewritable media.
All the jobs are confirgured as "Append to media, overwrite if no appendable media is available" so why is the second job asking to insert ovewritable media.
Is appendable media available at the time the job launches? I am assuming that there is not. so maybe you need to adjust your OPP and Append times?
Be aware that, If no appendable media is available, then BE switches the job to looking for overwritable media. Once a job starts looking for overwritable media, it will never switch back to look for appendable media.
Assuming that your tape library only has one drive, then I would avoid using a shotgun start of multiple jobs. Stagger the start time of your jobs by 5 minutes so that when a job runs, there will be appendable media available.
Yes there is appendable media. Actually the jobs are directed to specific slots in the robotic library and I confirmed that the tape in the slot is appendable.
The tape library has only one drive and I have 2 media servers sharing the tape library and are using the SSO option. The B2D jobs are set to run in parallel on both servers, but then the B2T are set to run as soo as the B2D has finished. I assume that using the SSO option the media servers will wait for the tape library to become avaialble before starting the B2T.
Is there a way to check whether a job is looking for appendable media or has switched to overwritable media as this would help with the investigation.
Are your jobs trying to use the same tape/slot?
If the problem only happens intermittently, then I will assume that your jobs are correctly configured to use appendable media.
Basically, as soon as BE prompts for overwritable media you know that BE thinks there are no appendable media available AT THAT TIME, so then you need to determine why your belief and BE's belief are different. Unfortunately, if this is happening in the middle of the night, it is hard for you to see exactly what BE sees at that time.
You say you are doing B2T, but if you have them configured to run immediately after the B2D jobs, then I suspect that you are really doing a duplication to tape job? The distinction probably doesn't matter, but just trying to make sure we all are on the same page.
I would avoid doing B2D, then B2T as the backups could be different. B2D2T is the right way to go.
But I would suggest that you schedule the duplicate to tape jobs at a specific time (rather than immediately after) as I suspect that intermittent timing is why you are getting prompted for overwritable media sometimes.
Yes both servers are backing up to the same slot. I have configured each slot to hold the tape for a day e.g. Slot 1 = Monday, Slot 2 = Tuesday, etc...
Your are correct I am using a template to do a B2D and then a duplication to tape so B2D2T
The only servers that are doing B2T are the backups of the backup servers themselves.
Altough the problem happens at night the queued jobs remain stuck, so in the morning I can see that the server keeps prompting to insert overwritable media. Also if I run an inventory on the same tape, then the job that was originally asking for the overwritable media continues running as normal.
Altough the problem happens at night the queued jobs remain stuck, so in the morning I can see that the server keeps prompting to insert overwritable media.
As previously stated, once BE starts looking for overwritable media, that is ALL that job will be looking for. But that doesn't answer the question if APPENDABLE media was available when the job started looking for media.
Also if I run an inventory on the same tape, then the job that was originally asking for the overwritable media continues running as normal.
Running an inventory shouldn't change anything. After the inventory, does the job continue and run as an append or as an overwrite? If it runs as an append, then I am stumped. If it runs as an overwrite, then your overwrite protection period is probably shorter than you wanted and you destoryed fresh backups.
You may be running into some weird resource issues. I don';t think that it is very common to have a single drive library partitioned into single slots and then have that shared between two SSO servers. I am not saying it is a bad idea, but it isn't common. Therefore, I think that my suggestions to make sure that the tape jobs start at fixed times (instead of immediately after the first phase) might be key. they second job needs to know that an appendable media is available, otherwise it tries to switch to overwrite mode and <game over>.
Partitioning your tape library into single slots is very restrictive and inevitably leads to problems like yours, even for tape libraries which are not shared. By sharing the library, the problem just gets compounded.
See my article below on why your jobs are not appending to the tapes especially the part about overlapping jobs.
I would strongly suggest that you remove the partitions and let BE and the tape library manage the tapes for you. See my article below
The idea of partitioning the library into single slots was done to force the backup to a particular tape. I had read this in some thread in the forums.
I am pretty sure that after the inventory the backup continued as an append, but better if I check this again. In one instance I had the following.
Backup for Server 1 started on 14/04/2015 5:29:55 A.M. The job was asking for overwritable media and the tape which was a Monday had the following settings
Modified: 14/04/2015 2:23 A.M
Overwrite: 19/04/2015 2:23 A.M
Append: 15/04/2015 11:00 A.M
Based on the above the tape should be appendable and the job should have continued. However it was queued and after I ran an inventory the job continued. I got the impression that the Job was not recognising the tape as appendable. As you mentioned it is very strange.
The idea of partitioning the library into single slots was done to force the backup is done to a particular tape. I had read this on the forums and it seemed a good idea at the time.
I am pretty sure that after the inventory the backup that is queued continues in Append mode but will check again to be 100% sure. In a particular case I had the following:
The duplicate to tape for a particular job started on14/04/2015 5:29:55 am. The job was queued and awaiting overwritable media. The tape in the slot was as follows
Modified: 14/04/2015 2:23 am
Overwrite: 19/04/2015 2:23 a.m.
Appendable: 15/04/2015 11:01 a.m.
Based on the above the job should have been able to append to the tape but was instead asking for ovewritable media. I ran the inventory and after the inventory the job continued running in Appendable mode. I will check again as I agree that it is very strange. Also I am not sure if I could have run an inventory if a job was already waiting for the tape.
Looking at the article from @pkh I might be having issues that the jobs from the 2 media servers are overlapping. I assumed that the SSO would not allow the backups to tape on media server 2, to start if media server 1 was already using the tape.
I have actually disabled the tape backups for BK2 (so I am only doing B2D) and only left the tape backups running only on BK1 (B2D2T). The same issue has happended again. The backup to tape was queued could not be done and the server keeps asking to enter Overwritable Media.
When I went to check the backups I noctied that the tape was actually Overwritable and Not Appendable, but the job kept asking to Insert Overwritable media.
I then moved the tape to scratch media and Reponded OK to the prompt to insert overwritable media and the job ran sucesfully.
Any ideas of what the issue might be?
The tape could become overwritable between the time the job needs it and the time that you check. You need to ensure that there are enough overwritable tapes BEFORE the start of your jobs.
I confirmed that the tape was overwritable before the job started. Actually I found the following. The jobs for Saturday were Queued and asking to insert the overwritable media. The jobs for Sunday were all marked as Ready, no idle devices (this is correct as the other jobs were queued).
When I cancelled the Saturday jobs, the Sunday job immediatley asked me to insert Overwritable media even though I had checked that the media was overwritable. I then assigned the tape to scratch media and the Sunday job could continue.
Any ideas of what the problem might be or if there are any logs that might help
Not yet as this ia a live system in a controlled environment and making changes is quite complex.
I would like to undertsand why it is not working given that up to 2 months ago it was working correctly with both servers peforming B2D2T.