Matt,
In regards to the networking suggestion, 1 quick question: are all your NICs and port speeds hard set to their appropriate setting? Ie: is anything set on 'auto negotiation'?
Ok, 2 questions: are all drivers for NICs up to date?
Sorry, 3: are the failures on all backups? differentials only? If it's just on differentials (I had a similar problem with a >1TB system) then the client may be timing out while looking for the next file to backup.