07-30-2013 02:45 AM
Morning gents,
I am experiencing a really weird and inconsistent issue with my backups. The error revolves around the following:
Backup- GUINNESS-FAS02:10000
An unknown error occurred on device "HP 0007".
V-79-57344-34035 - The block size being used is incorrect.
So our setup is simple:
BackupExec 2010 R3
1 HP Robotic Tape Library called HP003. The model of this library is MSL 4048 / MSL G3 Series
4 tape drives as part of the library called HP004, HP006. HP0007 and HP0008. The model of each of these drives is HP Ultrium 5-SCSI
We have two NetApp devices each of which have 2 controllers (NetApp1 consists of controllers Netapp-A and NetApp-B and NetApp 2 device consists of controllers NetApp-C and NetApp-D)
We have 2 device pools created where the first device pool consists of 3 drives (HP0004, HP0006 and HP0007) and backs up data from the NetApp1 device. We have a second device pool that consists of 1 drive (HP0008) that serves backups from the NetApp2 device.
The drives all have the same settings:
Enable Compression
Block Size (per device) = 64kb
Buffer Size (per device) = 256kb
Buffer count = 20
High Water count = 10
Read single block mode = disabled
Write single block mode = disabled
Read SCSI-passthrough mode = disabled
Write SCSI-passthrough mode = disabled
So this error is really strange........when a daily incremental fails, I leave it and do absolutely nothing. The very next day, when the scheduled job runs , it completes fine!!! What gives?
I think I recall a backup job failing during the early hours with the same error and when I came in the morning and hit retry job, it worked and completed fine!
I can't understand these random success/failures and was hoping you guys might be able to chime in your expertise. Is there a way that I can avoid any future failures?
Thanks,
Solved! Go to Solution.
08-01-2013 04:56 AM
...if backing up to tape or another B2D folder (read: NOT a dedupe folder within BE), then your data is rehydrated. If the NetApp is deduping your data, the data backed up is inflated to the original size.
If you have the option enabled to use a dedupe folder, then backup to that...run a test and see how this works in your situation.
Thanks!
07-30-2013 02:52 AM
Hi,
Have you tried to default your settings again and run the backups to see if the error is recreated?
Thanks!
07-30-2013 02:57 AM
HP management tools can also cause this issue
http://www.symantec.com/docs/TECH61192
Religiously follow this and your issue will be sorted including the busyretrycount setting part
07-30-2013 03:03 AM
Actually, the error message relates to incorrect block size which is addressed below:
http://www.symantec.com/business/support/index?page=content&id=TECH4903
Thanks!
07-30-2013 03:03 AM
Religiosly following!!!
Craig, setting back to default wouldn't really answer the issue as the problem is random with the current settings. What I am hoping to understand is why the system is performing mixed results - why do some fail and some succed.
07-30-2013 03:06 AM
...who knows. Are there any maintenance jobs running on the storage/media server during these errors? Is BE 2010 R3 fully patched?
07-30-2013 03:11 AM
Kunal,
In the first step of the solution it asks the person to install the latest version of HP Insight Agents. I can confirm this software does not exist on the windows server 2008 backup server. What I can confirm is that HP StorageworksLibrary and Tape Tools is installed.
Am I right in saying that I substiture step 1 and simply install the latest version of HP StorageWorks Library? I believe the latest version of this product is 4.16 available here:
Can you verify this is correct please Kunal?
Thanks
07-30-2013 05:00 AM
...you would install the HP agents IF you have a ProLiant server. If you don't then that isn't going to do anything.
However, HP LTT is going to help you assess that drive so continue with those steps.
Thanks!
07-30-2013 05:09 AM
No I dont have ProLiant server mate. The VM is installed on a UCS Blade Server. I will update the Storagworks and go fromt there,
07-30-2013 05:23 AM
Yes go ahead with the latest version of HP StorageWorks Library and ignore HP Insight Agents as its not present on your server
07-30-2013 05:35 AM
...why have you repeated what I said?
Thanks!
07-30-2013 05:36 AM
Aha...so it is a media server running on a VM using SCSI pass-through. You know this is not a supported solution from Symantec and VMware right?
07-31-2013 12:03 AM
Hi Craig,
I can answer half of that question.
Yes it is a media server running on a VM. I cannot say for sure if it is using SCSI pass-through. All I know is the tapes are connected to a cisco switch via Fibre. And the fibre commands have zoned in the drives.
Would one describe this as SCSI pass-through?
07-31-2013 12:07 AM
Nope, I doubt that.
07-31-2013 12:21 AM
As a test, is it possible to use a physical media server with the library instead & check if these random errors appear or not.
07-31-2013 12:21 AM
If memory serves me correctly, i believe the protocol is Fibre Channel.
I just dont get this incorrect block size espeicially when I know 64KB isnt really incorrect. Last night 2 backups failed with the same error but this time it wasn't the same two drives that had failed the night before!
The ones that failed 2 nights ago completed fine last night. It just so happens I was lucky enough to log on last night and see two incremental backups fail on two other drives. Upon restarting them, hey presto, they complete fine.
Im thinking of setting the compression settings back to default to see if that changes anything. If it doesnt then I have no way ut to disable compression.
I can confirm one thing (or at least if I recall correctly, I think I can) - when you select read/write single block mode the error doesnt come up. So in summary
- Jobs that fail one day work fine on another
- tapes that fail one day work fine on another
- read/write single block mode for each drive resolves the issue (99% sure)
Thanks Craig
07-31-2013 12:38 AM
I would suggest that you change your block size to 256k to match your buffer size. I had problems when I was tuning my tape drives and my blocksize was different from my buffer size. Don't worry about changing your block size to 256k. Your tape drive can handle it.
07-31-2013 01:55 AM
Ahhh ok. Now that is interesting. The only issue is that I receive a warning message (attached).
Is this something to worry about? We generally process a number of restore requests from tape and the tapes will have been written to using the following settings:
Enable Compression = Disabled
Block Size (per device) = Disabled
Buffer Size (per device) = Disabled
Buffer count = Disabled
High Water count = Diabled
Read single block mode = Enabled
Write single block mode = disabled
Read SCSI-passthrough mode = Enabled
Write SCSI-passthrough mode = disabled
By increasing the Block size to 256, will this cause a problem with any future issues? If it does, can I revert back and get restores working again.
I have attached the warning message.
07-31-2013 02:00 AM
Read this to know this impacts
http://www.symantec.com/docs/TECH18486
07-31-2013 02:10 AM
Read that. If there are any issues, is it a simple case of reverting back the older settings?
Thanks,