System state backup is partially completing

[Dharsana]

Netbackup 7.1.0.3

Master and client: Windows server 2008 R2 Standard

System state backup partially completing.

Job Details:

15/01/2016 19:00:21 - begin writing
15/01/2016 23:15:27 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open object: System State: (WIN32 -536805685: Unknown error)
15/01/2016 23:15:55 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\Task Scheduler\TasksStore (WIN32 -536805667: Unknown error)
15/01/2016 23:15:56 - Error bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: ERR - Error encountered while attempting to get additional files for System State:\
15/01/2016 23:15:57 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\VSS Express Writer Store\WriterMetadataStore (WIN32 -536805667: Unknown error)
15/01/2016 23:15:58 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\Performance Counter\PerformanceCounters (WIN32 -536805667: Unknown error)
15/01/2016 23:15:59 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\Automated System Recovery\BCD\BCD (WIN32 -536805667: Unknown error)
15/01/2016 23:16:17 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\System Files\System Files (WIN32 -536805667: Unknown error)
15/01/2016 23:16:18 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\Registry\Registry (WIN32 -536805667: Unknown error)
15/01/2016 23:16:19 - Warning bpbrm(pid=14656) from client EUAYLVCB02.accoeu.local: WRN - can't open file: System State:\COM+ Class Registration Database\COM+ REGDB (WIN32 -536805667: Unknown error)
15/01/2016 23:17:21 - Info bptm(pid=15788) waited for full buffer 20415 times, delayed 113205 times    
15/01/2016 23:17:42 - Info bptm(pid=15788) EXITING with status 0 <----------        
15/01/2016 23:17:44 - Info bpbrm(pid=14656) validating image for client EUAYLVCB02.accoeu.local        
15/01/2016 23:18:04 - end writing; write time: 04:17:43
the requested operation was partially successful(1)

The job was successfully completed, but some files may have been
busy or unaccessible. See the problems report or the client's logs for more details.
15/01/2016 23:18:09 - Info bpbkar32(pid=5848) done. status: 1: the requested operation was partially successful    

VSS writers are stable.

Found snapshot is disabled. Does this is the problem for partially completing?

Attached screensot, bpbrm and bpbkar logs

[sdo]

Looks like you have a VSS stability issue:

HOWTO - How to verify Windows Open File Backup operation using VSSADMIN and BPFIS before contacting Symantec Technical Support
http://www.veritas.com/docs/000031547

[sdo]

Your Win32 error code is:  -536805667

...which is: 0xE000FEDD

...and Google finds these:

https://www.veritas.com/support/en_US/article.000009278

https://www.veritas.com/support/en_US/article.000086392

You may found others with that hex code.

[Baski]

Hi ,

Ask to Intel admin to take windows NT backup. If they are unable to take backup. Let them fix any windows patches.It will help to fix VSS issue.

[SujayDesai]

Check event veiwer logs are there any errors in it.

Check with below command on client are your writers stable when no backup is running for that client:-

vssadmin list writers

Check are we able to take the backup of System state using wbadmin if your client server is 2008 else if its 2003 then try with NT backup.

Below technet of microsoft for  your refernece:-

https://technet.microsoft.com/en-us/library/cc753201.aspx

If above backup works fine then check with below actions:-

Check that windows open file backup is enabled.

To enable it below are the steps:-

Admin console->Master server host properties->Client Attribute->Select the client->Windows Open File Backup->Check the box for Enable Windows Open File Backups for this client and select Use Microsoft Volume Shadow Copy service.

If any antivirus installed on client disable the antivrus or exclude netbackup.

Then initiate the ackup and check.

[Dharsana]

Thanks All.

I will work with Windows Administrators as we donot have access to clients

[Dharsana]

NT Backup is successful

   Backup Status
Operation: Backup
Active backup destination: File
Media name: "System State.bkf created 1/19/2016 at 3:37 PM"

Volume shadow copy creation: Attempt 1.
Backup (via shadow copy) of "System State"
Backup set #1 on media #1
Backup description: "Set created 1/19/2016 at 3:37 PM"
Media name: "System State.bkf created 1/19/2016 at 3:37 PM"

Backup Type: Copy

Backup started on 1/19/2016 at 3:37 PM.
Backup completed on 1/19/2016 at 3:40 PM.
Directories: 191
Files: 2887
Bytes: 558,092,896
Time: 2 minutes and 16 seconds

[Dharsana]

Hi Sujay,

Windows anti virus is out of date, so Windows administrators will work with respective team to fix it and then they will exclude netbackup, if required.

[Marianne]

Can you confirm that the client has been patched to 7.1.0.3 or 7.1.0.4?

I remember numerous patches for System State backups in 7.1.0.x patches.

Oh - you forgot to attach client's bpbkar log.
Level 0 logging may not be sufficient to log actual issue.
I usually recommend level 3.
Veritas Support normally wants level 5 (please only enable level 5 logs when you want to open a Support call).

PS:

Any reason why you are on such an old NBU version?

Current NBU version is 7.7.1.
Support for all 7.0.x - 7.6.x versions will end in a year from now.

[Dharsana]

Hi Marianne,

Master Version is 7.1.0.3. So we have patched client to 7.1.0.3

It is customer requirement to hold in 7.1.0.3 as they have different plans for these servers.

Will upload the bpbkar logs with verbose 5 with next full backup. as only full backup is partially completing.

[Dharsana]

created case with Veritas.

and they suggested to the solution No. 4 and change the maximum size setting accordingly.

http://www.veritas.com/docs/000010988