Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Dear Frnds

I am having an issue with my exchange backup policy, Each time when the backup started it got failed.

Logs are attached. Please help me as my veritas support is expired, therefore, I am unable to take support from them. 

Your help will be highly appreciated.

Nebackup version: 8.1

Plateform:  Windows Server 2008

 

Regards

Zuhaib Siddiqui

Tags (2)
14 Replies

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

1) Exact version of MS Exchange?

2) Definitely Windows 2008, and not Windows 2008 R2?

3) Has this ever worked, or is this a new backup policy or new backup client or new database?

4) Is this your first ever configuring an MS Exchange backup policy?

5) Is Exchange a DAG or CCR or other or standalone?

6) Please attached as a text file the full detailed backup policy listing of the problem backup plicy by using bppllist -U policyname

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

1) Exact version of MS Exchange? 2016

2) Definitely Windows 2008, and not Windows 2008 R2? Windows 2008 R2

3) Has this ever worked, or is this a new backup policy or new backup client or new database? Yes ever worked

4) Is this your first ever configuring an MS Exchange backup policy? No

5) Is Exchange a DAG or CCR or other or standalone? DAG

6) Please attached as a text file the full detailed backup policy listing of the problem backup plicy by using bppllist -U policyname. command is not working, please help me

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

sorry, the command is:

bppllist policyname -U

 .

Still need exact version of MS Exchange, i.e. CU level

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

CU level is 8

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Logs are mention below:

Nov 8, 2019 7:24:51 PM - Info nbjm (pid=7588) starting backup job (jobid=47963) for client exchsrv.ablamc.com, policy Exchsrv, schedule FullBackup_Daily
Nov 8, 2019 7:24:51 PM - Info nbjm (pid=7588) requesting MEDIA_SERVER_ONLY resources from RB for backup job (jobid=47963, request id:{83226E2C-C6CC-41CE-AE37-4C2D2F02E372})
Nov 8, 2019 7:24:51 PM - requesting resource  NBU_DP-stu
Nov 8, 2019 7:24:51 PM - requesting resource  nbpsrv.ablamc.com.NBU_CLIENT.MAXJOBS.exchsrv.ablamc.com
Nov 8, 2019 7:24:51 PM - requesting resource  nbpsrv.ablamc.com.NBU_POLICY.MAXJOBS.Exchsrv
Nov 8, 2019 7:24:51 PM - requesting resource  EXCHANGE_RESOLVER.nbpsrv.ablamc.com.Exchsrv.exchsrv.ablamc.com
Nov 8, 2019 7:24:51 PM - granted resource  nbpsrv.ablamc.com.NBU_CLIENT.MAXJOBS.exchsrv.ablamc.com
Nov 8, 2019 7:24:51 PM - granted resource  nbpsrv.ablamc.com.NBU_POLICY.MAXJOBS.Exchsrv
Nov 8, 2019 7:24:51 PM - granted resource  EXCHANGE_RESOLVER.nbpsrv.ablamc.com.Exchsrv.exchsrv.ablamc.com
Nov 8, 2019 7:24:56 PM - estimated 263813545 kbytes needed
Nov 8, 2019 7:24:56 PM - begin Parent Job
Nov 8, 2019 7:24:56 PM - begin Exchange14 Resolver: Start Notify Script
Nov 8, 2019 7:24:56 PM - Info RUNCMD (pid=3260) started
Nov 8, 2019 7:25:00 PM - Info RUNCMD (pid=3260) exiting with status: 0
Operation Status: 0
Nov 8, 2019 7:25:00 PM - end Exchange14 Resolver: Start Notify Script; elapsed time 0:00:04
Nov 8, 2019 7:25:00 PM - begin Exchange14 Resolver: Step By Condition
Operation Status: 0
Nov 8, 2019 7:25:00 PM - end Exchange14 Resolver: Step By Condition; elapsed time 0:00:00
Nov 8, 2019 7:25:00 PM - begin Exchange14 Resolver: Read File List
Operation Status: 0
Nov 8, 2019 7:25:00 PM - end Exchange14 Resolver: Read File List; elapsed time 0:00:00
Nov 8, 2019 7:25:00 PM - begin Exchange14 Resolver: Resolver Discovery
Nov 8, 2019 7:25:01 PM - started process bpbrm (pid=9480)
Nov 8, 2019 7:25:08 PM - Info bpbrm (pid=9480) exchsrv.ablamc.com is the host to restore to
Nov 8, 2019 7:25:08 PM - Info bpbrm (pid=9480) reading file list for client
Nov 8, 2019 7:25:15 PM - Info bpbrm (pid=9480) client_pid=25000
Nov 8, 2019 7:25:16 PM - Info bpbrm (pid=9480) from client exchsrv.ablamc.com: TRV - BPRESOLVER has executed on server (EXCHSRV)
Nov 8, 2019 7:25:44 PM - Info bpresolver (pid=25000) done.  status: 2
Nov 8, 2019 7:25:44 PM - Error bpbrm (pid=9480) no worklist items returned from bpresolver on client  exchsrv.ablamc.com
Nov 8, 2019 7:25:44 PM - Info bpresolver (pid=25000) done. status: 2: none of the requested files were backed up
Nov 8, 2019 7:25:44 PM - end Exchange14 Resolver: Resolver Discovery; elapsed time 0:00:44
Nov 8, 2019 7:25:44 PM - begin Exchange14 Resolver: Stop On Error
Operation Status: 0
Nov 8, 2019 7:25:44 PM - end Exchange14 Resolver: Stop On Error; elapsed time 0:00:00
Nov 8, 2019 7:25:44 PM - begin Exchange14 Resolver: End Notify Script
Nov 8, 2019 7:25:44 PM - Info RUNCMD (pid=7044) started
Operation Status: 2
Nov 8, 2019 7:25:51 PM - Info RUNCMD (pid=7044) exiting with status: 0
Operation Status: 0
Nov 8, 2019 7:25:51 PM - end Exchange14 Resolver: End Notify Script; elapsed time 0:00:07
Operation Status: 2
Nov 8, 2019 7:25:51 PM - end Parent Job; elapsed time 0:00:55
none of the requested files were backed up  (2)

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Did anything change recently?  Was MS Exchange recently patched with a new CU?  Any OS patches in the days/week before backups broke?

When was the server last rebooted?  Did backups break immediately since most recent reboot?

Have the DAG managed databases recently moved from one member node to another?

The backups were definitely working right?  I ask because, officially, it looks like NetBackup v8.x is not officially supported for use with MS Exchange 2016 on Windows 2008 R2, see:

https://download.veritas.com/resources/content/live/OSVC/100032000/100032809/en_US/nbu_80_db_scl.htm...

...many times we find that some interop scenarios which are not officially supported might work for quite some time or only for a short while, but then break when something else very minor and seemingly unlrelated elsewhere changes.

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

You are going to have to dig deeper into the logs.

Search this forum for other examples of advice on which NetBackup logs to enable for MS Exchange backups and how to reach those logs, and or see here:

https://www.veritas.com/content/support/en_US/doc/19475139-127283815-0/id-SF930127341-127283815

...but... at the end of the day, you might never get it to work again, because it is not an officially supported interop configuration.

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Dear SDO

Did anything change recently?  Was MS Exchange recently patched with a new CU?  Any OS patches in the days/week before backups broke?

Ms. Exchange not yet patched with any CU. Current CU is CU8, OS on which Exchange is installed is Windows Server 2012 R2 and also OS is not patched.

When was the server last rebooted?  Did backups break immediately since the most recent reboot?

Server rebooted recently

The backups were definitely working right? Yes, before that backups were working right. I ask because, officially, it looks like NetBackup v8.x is not officially supported for use with MS Exchange 2016 on Windows 2008 R2, see. Ms. Exchange installed on Windows Server 2012 R2

https://download.veritas.com/resources/content/live/OSVC/100032000/100032809/en_US/nbu_80_db_scl.htm... 

Link is not working

Regards

Zuhaib Siddiqui

 

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

@zuhaib 

Have you tried to check bpresolver log on all DAG nodes?
If the folder does not exist, please create it before next backup attempt.

 

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Dear Marianne

Thanks for response

We have only one DAG named "DAG01.<companydomain>.com.

How I access DAG because it is IP less DAG.

 

Regards

 

 

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Exchange DAG is a clustered configuration that concists of physical nodes. 
Speak to your Exchange Admin for access to logs on the nodes.

We see this hostname in the job details:

Nov 8, 2019 7:25:16 PM - Info bpbrm (pid=9480) from client exchsrv.ablamc.com: TRV - BPRESOLVER has executed on server (EXCHSRV)
Nov 8, 2019 7:25:44 PM - Info bpresolver (pid=25000) done.  status: 2
Nov 8, 2019 7:25:44 PM - Error bpbrm (pid=9480) no worklist items returned from bpresolver on client  exchsrv.ablamc.com

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

Dear Marianne

Please find attached bpresolver logs.

Please help me.

Regards

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

these look self explanatory :

10:30:35.175 [14704.25252] <16> set_exchange_impersonation: Login failed for user:

10:30:57.502 [14704.25252] <16> Resolver_Exchange::generate_TopologyList: No databases in the file list are mounted - no databases with active servers found

10:30:57.518 [14704.25252] <16> bpresolver_main(): bpresolver could not resolve the passed directive set.

.

My advice is to go back to the documentation and start again - as you appear to have missed at least one step out - i.e. bpresolver might be saying that it cannot find any databases because it was unable to logon, and it says this right at the very beginning of the logs - so, check the account that you are using.  If you don't know what to check, then go back to the NetBackup documentation and start the configuration again from the beginning.

Highlighted

Re: Error bpbrm (pid=1264) no worklist items returned from bpresolver on client

You need to set Exchange credentials in the client host properties for each node of the DAG.

If you set client host properties for the DAG, they only get set on the Exchange server to which NetBackup resolves the DAG at the moment. To set the host properties on every Exchange server if they are not already visible as clients in the NetBackup admin console, I create a fake MS-Windows policy on each client. You can delete the policy once the master server has added the server to its client list.

Without the Exchange credentials, bpresolver can't run PowerShell commands to learn the DAG layout.