We can't officially support SCSI or USB pass through into VMs with Backup Exec and the symptoms you describe are typical of the random issues we have had reported by customers that have tried it.
We will support you if you can prove to us that this same fault exists when the library is connected via the same interface card into a physical server instead of a virtual server (although we would want to research the condition on the physical server )
Recommendation is therefore switch your media server to a Physical machine
As an aside (we had a partner point out something else about this type of configuration earlier this week)
If your media server is virtualized and backs up other VMs in the same ESX environment then should anything break ESX, your media server will also be broken. As an English idiom this is known as "Putting your eggs in one basket" i.e. if you drop the basket everything inside is broken.
As such best practice is to run your backup server outside the ESX environment anyway. That way if ESX has a failure, your backup server should still be available, and if your backup server does something nasty - it won't effect ESX.
