I've been trying to get this working right lately and have not had much success either.
I would think that since each drive can be identified uniquely by it's serial number (under any BE supported OS afik), BE would treat removeable drives like tapes (like Ken observed above). Apparently BE treats removeable storage like normal disk storage and just assumes you have an unreliable or temporary connection to a single target storage folder (drive), and not multiple physical drives (like many of us seem to).
Part of the problem may be BE treating B2D targets like a virtual tape library behind the scenes (literaly), as if each "Drive" is a virtual library with one "drive" and a large number of "slots" and "tapes" of the specified capacity. This would explain some of the difficulty, as each physical drive (or B2D target) would be treated as a VTL device instead of storage media (hope that makes sense). I've come to this conclusion after using NetVault to create a VTL on a NAS drive last month for B2D (what a pain BTW).
The only thing I've found so far that seems to 'mostly' work so far is to create a pool of unique B2D drives and adding in some logic to handle 'missing' drives to the BE settings (auto accept media alerts for example). But this only works for a low number of drives as each drive would consume one drive letter (v,w,x,y,z for example), which makes it of limited utility in the end. The drive sometimes being unavailable still crops up randomly though (again can be handled somewhat grudgingly with pre-post scripts).
As many of my more frugal and less 'backup aware' clients keep asking for USB backup solutions this makes BE somewhat of a hard sell (If backing up to tape, it can be pretty easy).
[Idea: Single/Multi drive virtual tape library where each drive is treated as a tape]
What BE needs is to create a logical VTL device on the media server and treat an assigned drive letter (Z: for example) like a single drive slot in the VTL (same as it would treat a physical library with one tape drive). Then just treat each removeable drive (USB/FireWire/eSATA/hot-swappable) just like tapes (prepare, inventory before use, etc).
They could even impose certain restrictions initially to make this work, like reformatting the drive to a specific format, or assigning physical drives to virtual slots (can create an upper limit, but 16 should be plenty for practical purposes, any more and you might as well just use tapes and a library anyway). Done this way you could easily use multiple drives for the same backup job (spanning), etc. Even better, add a second drive letter as another "tape" slot to the VTL and use multiple drives at once.
BE could even handle media swapping by listening for device arrival/removal events from the OS and check to see if the device is one of the removeable drives ("tapes") in the B2D media pool, then assign it the correct drive letter as needed. (I've done this using WMI and VBS by the way, so I know it can at least be done under Windows)