EC13

[NBU35]

Hi we are taking backup of windows box , OS win 2008, NBU 6.5.4, we are taking its backup by applying multistreaming, all streams completing successfully , except shadow copy components, its failing with EC 13.

nelow are the logs

Nov 28, 2011 12:32:07 PM - Error bpbrm (pid=20441) socket read failed: errno = 131 - Connection reset by peer
Nov 28, 2011 12:32:07 PM - Error bptm (pid=21773) system call failed - Connection reset by peer (at child.c.1296)
Nov 28, 2011 12:32:08 PM - Error bptm (pid=21773) unable to perform read from client socket, connection may have been broken
Nov 28, 2011 12:32:08 PM - Error bptm (pid=21293) media manager terminated by parent process
Nov 28, 2011 12:32:19 PM - Error bpbrm (pid=20441) could not send server status message
Nov 28, 2011 12:32:22 PM - end writing; write time: 0:08:04

master and media both are solaris 10 and nbu 6.5.6 & 6.5.4

please suggest.

[Will_Restore]

Solution

Backup should work again once VSS default provider is set to "system provider". The steps are as follow:

1.

For NetBackup older than version 6.5.6: Upgrade the NetBackup client to 6.5.6 and obtain the EEB mentioned below.

For NetBackup version 6.5.6: Contact Symantec Technical Support to obtain EEB 2370957.  Install the EEB on the client.

For NetBackup version 7.0: Contact Symantec Technical Support to obtain EEB 2077740. Install the EEB on the client.

For NetBackup version 7.1.x: Proceed to step 2.

2. Log in to the affected client, and run below command:

<install dir>\Veritas\NetBackup\bin\w2koption -backup -snapshotprovidertype 1

3. Retry the backup.

http://www.symantec.com/business/support/index?page=content&id=TECH161331

[Marianne]

Which version of W2008? Windows Server 2008 R2 is supported as from 6.5.5 and Windows Server 2008 R2 Core as from 6.5.6.

Please ensure that you have the following logs enabled:

On media server: bpbrm and bptm
On client? bpbkar and bpfis

[NBU35]

On checking again i found all master, media and clients are on 6.5.6, now please suggest.

is Installing the EEB on the client the only way to eliminate this error ?

[Marianne]

Bill's post seems to hit the nail on the head.

bpbkar log on your client will confirm if you have the same problem.

[NBU35]

OS of client is Windows 2008 R2 SP1.0 and we enabled the bpbkar and bpfis logs on it

[NBU35]

please find the attached bpfis logs of client, nothing i can find in the bpbkar logs of client related to the shadow copy componenets.

[Marianne]

All snapshots were successfully done by bpfis:

bpfis Exit: INF - EXIT STATUS 0: the requested operation was successfully completed

You need to look at the rest of the logs (see my earlier post).

[NBU35]

i found this in event logs

Faulting application name: bpbkar32.exe, version: 6.5.2010.423, time stamp: 0x4bd26203

Faulting module name: bpbkar32.exe, version: 6.5.2010.423, time stamp: 0x4bd26203

Exception code: 0xc0000005

Fault offset: 0x00000000000d177a

Faulting process id: 0x958

Faulting application start time: 0x01ccaebcfb7c3dec

Faulting application path: D:\Program Files\Veritas\NetBackup\bin\bpbkar32.exe

Faulting module path: D:\Program Files\Veritas\NetBackup\bin\bpbkar32.exe

Report Id: 1d455f87-1ab2-11e1-8461-a4badb4bf5dd

[NBU35]

bprm , bpbkar and bptm logs are attached , please suggest..

[Marianne]

Found this in bpbrm:

12:46:58.589 [14645] <16> bpbrm readline: socket read failed: errno = 131 - Connection reset by peer

These 2 TN's point to idle socket being closed prematurely.
Please ignore reference to database agents and look at Resolution section of both:
http://www.symantec.com/docs/TECH43919
http://www.symantec.com/docs/TECH70670 

[NBU35]

but hre problem is this , we are taking backup by applying multistreams and only stream of shadow copy compnents is failing in this way , rest are completing with out failing. Please suggest .

[Marianne]

I somehow missed your 'event logs' post last night.

What was the timestamp on the errors in event log?

Maybe that explains the gap in bpbkar log between 12:45 and 12:54.

12:45:54.556 PM: .....
12:54:22.048 PM: 

bpbrm on the media server reported 'socket read failed' at 12:46. Probably as a result of 'faulting bpbkar32.exe'?

Best to log a support call. Symantec will need level 5 logs.

[Mark_Solutions]

This looks like it has been seen before - based on the version of bpbkar32.exe shown in your event viewer

It was caused in that case by a huge number of scheduled tasks

The link is here:

http://www.symantec.com/docs/TECH153780

Hope this helps - worth checking

[NBU35]

on checking we found the no task is scheduled as fo now , only 46 tasks were scheduled and all completed successfully.

[Marianne]

We have REALLY tried to help with everything we could find/think of.

Maybe time open a support call with Symantec?

[NBU35]

We have logged a case with symantec, will update its progress here :)

[Mark_Solutions]

One other thought as you are having an application crash ... have you excluded the Veritas and all sub folders from Anti Virus scan (and all apps as well if needs be - bpfis, bpbkar32, bpcd, vnetd etc.)?

[NBU35]

Symantec has suggested to open a case with microsoft bcz as symantec checked teh ouput of

vssadmin list writers
vssadmin list providers

vssadmin list writers shows few Failures which need to be fixed by Microsoft , so that all writers show Stable State. 

we ahve opened a case with the microsoft, they are loking in to it , i will update it further here.