Exchange Backup Failing 'file read failed(13)'

Question

Our Netbackup 7.5.0.5&nbsp;Exchange backups have started failing with 'file read failed(13)' errors.&nbsp; Both servers (Exchange and Netbackup) are running on Server 2008 R2.&nbsp; The mailstores (database &amp; logs)&nbsp;are residing on&nbsp;a Netapp FAS2040 with SnapDrive managing snapshots which are created frequently without error.&nbsp; The Netbackup policy itself is set to perform snapshot backups using VSS Provider Type 1 (System), without a DAG database source, has been backing up without any problems until some Windows updates including SP3 for Exchange 2010 were installed.

The job fails as follows:

28/11/2014 10:32:33 - Info nbjm(pid=5940) starting backup job (jobid=186119) for client exchange, policy EXCHANGE, schedule Full_Disc&nbsp;
	28/11/2014 10:32:33 - estimated 0 Kbytes needed
	28/11/2014 10:32:33 - Info nbjm(pid=5940) started backup (backupid=exchange_1417170753) job for client exchange, policy EXCHANGE, schedule Full_Disc on storage unit NexsanDSU
	28/11/2014 10:32:34 - started process bpbrm (484288)
	28/11/2014 10:32:40 - Info bpbrm(pid=484288) exchange is the host to backup data from&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:32:40 - Info bpbrm(pid=484288) reading file list from client&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:32:40 - connecting
	28/11/2014 10:32:43 - Info bpbrm(pid=484288) starting bpbkar32 on client&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:32:43 - connected; connect time: 00:00:03
	28/11/2014 10:33:27 - Info bpbkar32(pid=11548) Backup started&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:27 - Info bptm(pid=484816) start&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:28 - Info bptm(pid=484816) using 262144 data buffer size&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:28 - Info bptm(pid=484816) setting receive network buffer to 1049600 bytes&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:28 - Info bptm(pid=484816) using 30 data buffers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:29 - Info bptm(pid=484816) start backup&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:30 - Info bptm(pid=484816) backup child process is pid 482596.479660&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:30 - Info bptm(pid=482596) start&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 10:33:30 - begin writing
	28/11/2014 11:07:53 - Error bpbrm(pid=484288) from client exchange: ERR - Terminating backup.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 11:07:53 - Error bpbrm(pid=484288) from client exchange: ERR - failure reading file: Microsoft Information Store:\1GB Limit Mailbox Database\Logs_1417170539 (BEDS 0x0: )
	28/11/2014 11:08:03 - Error bpbrm(pid=484288) could not send server status message&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 11:08:04 - Error bpbrm(pid=484288) cannot send mail to&nbsp;...&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 11:08:05 - Info bpbkar32(pid=11548) done. status: 13: file read failed&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
	28/11/2014 11:08:05 - end writing; write time: 00:34:35
	file read failed(13)

I have checked the VSS Writers on Exchange and all are showing as stable with no errors.&nbsp; The Event Viewer on Exchange shows the following events just before the Netbackup&nbsp;job fails:

Log Name:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Application
	Source:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; MSExchangeIS
	Date:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28/11/2014 11:06:35
	Event ID:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9606
	Task Category: Exchange VSS Writer
	Level:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Information
	Keywords:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Classic
	User:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; N/A
	Computer:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exchange
	Description:
	Exchange VSS Writer (instance 8315289c-83fc-4e18-95c6-2f9e76f0f821) has prepared for backup successfully.

Log Name:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Application
	Source:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; SnapManager for Exchange
	Date:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28/11/2014 11:06:36
	Event ID:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 200
	Task Category: Backup
	Level:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Information
	Keywords:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Classic
	User:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; N/A
	Computer:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exchange
	Description:
	SnapManager VSS asynchronous DoSnapshotSet operation started.

Log Name:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Application
	Source:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ESE
	Date:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28/11/2014 11:06:39
	Event ID:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2007
	Task Category: ShadowCopy
	Level:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Error
	Keywords:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Classic
	User:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; N/A
	Computer:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exchange
	Description:
	Information Store (6236) Shadow copy instance 3 aborted.

Log Name:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Application
	Source:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; MSExchangeIS
	Date:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28/11/2014 11:06:39
	Event ID:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9609
	Task Category: Exchange VSS Writer
	Level:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Error
	Keywords:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Classic
	User:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; N/A
	Computer:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exchange
	Description:
	Exchange VSS Writer (instance 8315289c-83fc-4e18-95c6-2f9e76f0f821:3) failed with error code -2403 when preparing for Snapshot.

I appreciate this is probably an Exchange issue rather than a Netbackup one, but if anyone has an ideas as to why it is failing I'd be grateful for some help.

I have attached some log files, let me know if there is any other information that may help.

Thanks

&nbsp;

marianne · Answer

The important log that is missing here is bpbkar on the Exchange server.

You can see that bpbkar reported the error:

Info bpbkar32(pid=11548) done. status: 13: file read failed&nbsp;

&nbsp;

Oh, and you need to log a call with Microsoft about the VSS errors.
	This is probably the cause of the failure.&nbsp;

ultra317 · Answer

Thanks Marianne

I ran the job again and collected the bpbkar log (see attached) which contains&nbsp;'ERR - invalid STRM header'.&nbsp;After a little searching&nbsp;I found out this may be related to SnapManager for Exchange, where it&nbsp;changes the log location&nbsp;after creating a snapshot&nbsp;to a softlink that Netbackup cannot follow. It recommended removing the Netapp SnapManager for Exchange, which I don't think is an option.

Could this be the main cause of the failure or shall I continue down the VSS error route?

Thanks

ramwbp · Answer

I am also seeing the same issue. Any solution yet?

Forum Discussion

Exchange Backup Failing 'file read failed(13)'

Related Content

file read failed(13)

(13) file read failed

backup fail with status 13

Vmware backup fails code 13

File Read Failed (13)