Windows Clients exit status 1 - BMR issue?

[tod_jackson]

**** Moved to new discussion from https://www-secure.symantec.com/connect/forums/bmr-master-server-setup-giving-error ****

We have a few clients (Windows 2008 Ent R2) that are failing with Status Code 1. Investigating the logs I had noticed that the BMR boot server is not configured. Talking with the lead engineer he had mention clicking the option within the policy to activate BMR. I did. Being nosey I was looking around on the Master Server and notice that BMR Boot Server and BMR Master Server services weren't running. I am guessing that these services are only active during the BMR running during the actual backup. I tried to start the service and it stated that "failed, due to NBU Relational Database Manager". I had looked and when researching all things about BMR there were no BMRDB.db, or BMRDB.log, BMR_DATA.Db or BMR_INDEX.db. I also did not locate the file "databases.conf within C:\Program Files\Veritas\Netbacup\DB\.

Could the BMR Master Server configuration have not been completed correctly. Any information in this is greatly appreciated.

Tod Jackson

[Marianne]

As a start - I'm not sure why the Support engineer believed that the Status 1 is a result of BMR issue?

Please show us all text in Details tab of status 1 job. This should at least give us a starting point.

Also tell us NBU version and patch level on the NBU master and Windows clients.

Do you actually need to configure BMR or simply trying to troubleshoot status 1 exit code?

[Marianne]

ERR: failure reading file; System State:\System files:\System files (Win:32 -536805665: Unknown error

ERR: encountered while attempting to get additional files for System State:\

These errors have nothing to do with BMR.

Can you use Windows backup utility to backup System State and/or Shadow Copy Components?

What is in Backup Selection for this policy?

Similar issue in this post: https://www-secure.symantec.com/connect/forums/backup-failed-cant-open-object 

Again - do you need BMR or are you simply trying to resolve status 1 exit code?

[tod_jackson]

Thank you for replying back Marianne.

Version - NBU 7.5 Build 20120207

I had to transpose the details -

ERR: failure reading file; System State:\System files:\System files (Win:32 -536805665: Unknown error

ERR: encountered while attempting to get additional files for System State:\

====================================================================================

Within the bmrd log

\Veritas\Netbackup\logs\bmrd

Error while attempting to update Boot Server Record

Check whether the BMR Master Server has been setup and whether Boot Server has been registered with teh Master Server

Failure in initializing the boot server. Check if the boot server is registered with the BMR Master Server.

In seeing the above error I had thought that BMR had to be installed.

[tod_jackson]

Thank you for your reply and for clarifying the errors are not being related to BMR.

The Windows backup utility is not installed on this client surprisingly. The backup selection is as follows:

New Stream

Shadow Copy Components

c:\

New Stream

F:\

I am trying to resolve if possible the Status Code 1's. There are similar clients that have the same error and looking for a recommended fix if possible.

The team lead would like to activate BMR

[tod_jackson]

Thank you. I am going to try the recommendations from the TECHNOTE 147574.

[SymTerry]

Just curious. When you said  lead engineer, do you have a case open with NetBackup support?

[Marianne]

We can tell you for sure that the status 1 errors will not be solved by enabling BMR.
So, best to forget about BMR for now and let's see if TECH147574 helps to solve the problem. Or one of the other TNs mentioned in that discussion.

If I look at your Backup Selection, can we assume that 'Allow multiple data streams' is selected in the Policy attributes and that 3 jobs are created for :
Shadow Copy Components 
c:\
F:\

and that only job for Shadow Copy Components is failing with status 1?
C and F backups are fine?

[tod_jackson]

Thank you for your reply and recommendations. I do not believe the Shadow Copy Components is the cause of the Status 1. I had noticed something new since I selected the VSP option for the client. Now I am seeing the TIR error. When I look at where the error is it states that E:\ArchiveEventlogs\Netbackup_file_info.xxxx-xxxx-xxx. The TECHNOTE 84583 states that if you see the word NEW within the file name Netbackup_file_info rename the file. That isn't the case looking at the file in the tir_info folder. I had seen this before at another job and the word NEW was present and I renamed the file and everything was fine. Should I rename the file to something else maybe?? Your thoughts when you have a chance.

Thank you.

[tod_jackson]

Sorry SymTerry, when I had said Lead Engineer I meant my supervisor. I should have said Team Lead. Sorry for the confusion. I am almost at the point of opening a ticket, but the TECHNOTES and yourself and Marianne are being very helpful.

[Marianne]

The error that you posted above clearly shows System State error which is part of Shadow Copy Components.

Can I ask again that you check if 'Allow multiple data streams' is selected in Policy attributes?

This will ensure that 3 separate backup jobs are started and will therefore allow us to troubleshoot status 1 of each job separately. The reasons are normally different.

You are mentioning VSP? Really? 

As far as I am aware VSP was discarded by Symantec as from NBU 7.0.

Only VSS should by used for W2003 and later.

Please show us exact TIR error you are seeing to see if TECH84583 is applicable. 
I clicked on 'Product' in the TN and it seems to apply to NBU 6.x only.

Please create bpbkar and bpfis log folders under ...NetBackup\logs\ on problematic clients for further troubleshooting.

PS: 
Hopefully you have de-selected BMR option in the policy by now? 
We need to first see if we can get status 1's resolved before you attempt BMR backups again (if you really need it).
TIR with BMR selected is probably creating new issues as per this discussion: 

TIR backup issue.

[tod_jackson]

Thank you for replying. I know this discussion is getting tiring especially since I am not providing all of the details. Learning process.  Sorry for that.

4/3/2014 12:06:49AM - Warning bpbrm (PID=5252) from client xxxx-xxxxx-xxx:WRN - old tir info file 'C:\Program Files\veritas\Netbackup\tir_info\E\ArchiveEventLogs\Netbackup_file_info.xxxx-xxxxx-xxx' is missing. Backup everything in 'E:\ArchiveEventLogs'.

You are correct regarding only applying to NBU 6.5 which is what I had used at a previous job and changing the file name worked. It seemed my only reference at the time, is there anything new out there for this?

Allow multiple data streams is enabled

Also enabled is Collect true image restore info and with move detection

I read the option incorrectly regarding VSP. Windows Open File backup is enabled and the VSP was not selected but VSS is. The selection I have Snapshot error control is currently set to disable instead of abort. Snapshot usage is set for Individual snapshot. cannot unselect either of these.

The client I am dealing with is a VM running Windows 2008 R2 and the Master Server is running the same thing.

Your help is greatly appreciated.

[Marianne]

The post that I have mentioned above: TIR backup issue.  describes an issue with TIR and BMR.
You will see that backline engineer Jaime told the customer to log a Support call as the issue has been identified as a bug.

Please check that log folders exist on problematic clients to assist with troubleshooting status 1 for each backup stream separately.

*** EDIT ***

Have a look at this TN to manually test VSS snapshots on problematic clients:
http://www.symantec.com/docs/TECH47808 

[SymTerry]

If you do open a case, let me know as well. I will get a few key people, including Jaime, to look at it and help the Tech assigned.

[Jaime_Vazquez]

A client backup can get a RC=1 status code for situations where the BMR databases are not present and the bmrd process is not running. Use this TECH article to help diagnose the root cause, if it really is BMR:

BMR enabled backup jobs fail with status code 1
http://www.symantec.com/docs/TECH207639

Now, for the TIR stuff.  If the new TIR info file is created, the backup job (bpbkar status) is zero, but the newly created TIR info file( *info.new* )  is not being renamed to the correct format, then this a situation I am working on to get resolved. Look in the bpbkar debug log for an entry that looks like:

<2> tar_base::V_vTarMsgW: INF - EXIT STATUS 0: the requested operation was
successfully completed
<16> dtcp_read: TCP - failure: recv socket (360) (TCP 10054: Connection reset
by peer)

The tipoff is the "dtpc_read: TCP -failure" message.  Based on some empirical data, a possible solution for the problem is to set Resilient Network host property on the Master and include the client name in the list.

Please try this out and let me know your results. This looks like a TECH article in the making.

[tod_jackson]

Good Morning - I created the bpbkar, bpcd, bpfis folders. Set the Global setting to 2. The bpbkar log is huge 67kb and I cannot upload the log due to security reasons to support. Even trying to capture parts of it the screen for notepad or word becomes unresponsive. The bpcd, bpfis logs are much smaller. So I am at a lossas what to do next. Thinking I can call support and try to plod through the log hoping it won't lock up in the process. What I can see in the logs is host_cache issues. Strill troubleshooting a way to get the bpbkar logs up to support.

[Marianne]

I suggest you get permission to supply the log file to Symantec support or send the file via Private Message to one of us.

Symantec will need to see the log if you open a Support call. It is impossible to assist without seeing the logs.

You can try do download and install TextPad to open the file. Identify the PID for each stream - in square brackets after the timestamp [ ] - and filter all entries for particular PID.

If you search and replace all hostname and IP info, will you then be able to post log info?
You may also want to edit/delete file/folder info.