BE2010 job fails mid week, every week

Usually if a BE service stops in the middle of a job or if BE loses network connectivity with the data server, then this error message appears. This error message can appear due to a hardware issue, but it seems unlikely that such an issue would occur only on Wed and not other days of the week unless you use a specific tape for Wed backups only. I would recommend to focus the troubleshooting on the first 2 causes.

Check the event viewer logs on the BE media server and the remote server if there are any occurrences of a BE service stopping or restarting.

Also, are there any maintenance tasks running on Wed night which can interrupt the backup process or network connectivity ?

End marker unreadable error is associated with the hardware. As mentioned by vjware check the event id. For the hardware troubleshooting of RL. Kindly follow the below technical article.

https://support.symantec.com/en_US/article.TECH24414.html

https://www-secure.symantec.com/connect/blogs/how-troubleshoot-issues-robotic-library-autoloaderchanger-andor-tape-drives

No SQL maintenance plans. 
I don't see the service crashing in the logs.  

On the exchange server:  I see a bunch of schannel errors: 
The following fatal alert was received: 70. (also see 48) 
I see the VSS service stop, but I think that's normal.  
I see storage group consistancy checks in an error state, but not every time the job fails.  
I see this: Information Store (5816) Shadow copy instance 81 aborted. 
 

Enable shadow copies on the volume on which the Exchange database resides and specify no disk limit.

Done.  What will this do? 

This seemed to have worked.  It could be a fluke.  I'll feel better at the end of this week when it doesn't happen again.  

Failed again last night.  Now getting failures on no specific day, but not every day. 

Persuing flaky hardware as a cause.  Testing... 

The consistency check of the snapshot for the Microsoft Exchange database Database was successful.
Storage device "IBM 1" reported an error on a request to write data to media.
Error reported:
A device attached to the system is not functioning.
V-79-57344-34036 - An unknown error has occurred.

After this, the next two jobs run OK, but to another tape in the library.  This job's tape has an unreadable end-marker.  I asked my onsite contact how old the cleaning tape was, and he didn't know so we are getting him a new one asap.  Hopefully that does it. 

relevant bengine log snippet. 

[3076] 09/21/15 23:34:59 TF_NDMPStartWrite(): Started MediaServerWriteThread 4680
[1840] 09/21/15 23:36:18 DeviceManager: timeout event fired
[1840] 09/21/15 23:36:18 DeviceManager: processing pending requests
[1840] 09/21/15 23:36:18 DeviceManager: going to sleep for 900000 msecs
[1840] 09/21/15 23:51:18 DeviceManager: timeout event fired
[1840] 09/21/15 23:51:18 DeviceManager: processing pending requests
[1840] 09/21/15 23:51:18 DeviceManager: going to sleep for 900000 msecs
[4680] 09/21/15 23:54:48 ProcessMoverError default case!  mvrRet = 0x0000001f.  Returning 0xe00084f4.
[3076] 09/21/15 23:54:49 TF_NDMPGetResult(): MediaServer thread done, returning TFLE 0xE00084F4
[3076] 09/21/15 23:54:49 BaseBSDProcessor::MessagePumpAndWaitForResults(): TF_NDMPGetResult() returned 0xE00084F4
[3076] 09/21/15 23:54:49 TF_FreeTapeBuffers: from 0 to 0 buffers
[3076] 09/21/15 23:54:49  data halted: SUCCESSFUL
[3076] 09/21/15 23:54:49 BaseBSDProcessor::PollDataServer: Shutting down.
[3076] 09/21/15 23:54:49 RABackupBSDProcessor: MessagePumpAndWiatForResults() returned -536836876 (0xe00084f4)
[3076] 09/21/15 23:54:49 ERROR: 13 Error: Invalid state to process request
[3076] 09/21/15 23:54:49 BackupMessageHandler::CDB_EndCurrentCatalogOperation(  ) -- entered
[3076] 09/21/15 23:54:49 BackupMessageHandler::CDB_EndCurrentCatalogOperation(  ) is calling PerBSDDataVector[engine_id]->EndCatOperation( )
[3076] 09/21/15 23:54:49 EndCatOperation succeeded
[3076] 09/21/15 23:54:49 BackupMessageHandler::CDB_EndCurrentCatalogOperation(  ) -- returning
[3076] 09/21/15 23:54:49 BackupMessageHandlerForBackupJob::process( eng_msgs::close_catalog & msg ) called
[3076] 09/21/15 23:54:49 Not: 'm_bCatalogDrivenBackup && CatBuildVector[engine_id] && GetCarryFwdCatalogInfoForRemainingObjs( )'
[3076] 09/21/15 23:54:49 ERROR: CatBuildVector[engine_id] is NULL!!!!!
[3076] 09/21/15 23:54:49 TF_FreeTapeBuffers: from 0 to 0 buffers
[3076] 09/21/15 23:54:49 TF_CloseSet
[3076] 09/21/15 23:54:49 Error detected
[3076] 09/21/15 23:54:49 FreeFormatEnv( cur_fmt=0 )
[3076] 09/21/15 23:54:49 PDI backup - done. result = e00084f4
[3076] 09/21/15 23:54:49 Enter NDMPSnapHostBackupEngine::TrackSnappedBSD
[3076] 09/21/15 23:54:55 Exit NDMPSnapHostBackupEngine::TrackSnappedBSD
[3076] 09/21/15 23:54:55 NDMPSnapHostBackupEngine::ProcessBSDs Top of the loop
[3076] 09/21/15 23:54:55 snapResult = 0x0, result = 0xe00084f4
[3076] 09/21/15 23:54:55 NDMPSnapHostBackupEngine::ProcessBSDs Top of the loop
[3076] 09/21/15 23:54:55 snapResult = 0x0, result = 0xe00084f4
[3076] 09/21/15 23:54:55 Exit NDMPSnapHostBackupEngine::ProcessBSDs
[3076] 09/21/15 23:54:55 @@@@@@@MyCloseSocket called with sockfd = 1904(0x770)    retval = 0
[3076] 09/21/15 23:54:55 NDMPAgentConnector::EndConnection(): Data Server = LIEX-MAIL
[3076] 09/21/15 23:54:55 Setting rewind flag to true for older device.  This will likely degrade the performance of the current job.