cancel
Showing results for 
Search instead for 
Did you mean: 

Netbackup GRT Client Partially Successful

2013
Level 5

Netbackup Client GRT Partially Successful with Error1.

Master Server 7.7.3 /Media Server 7.7.3 / Client 7.7.3

i have attached bpbkar client logs.

Please check what needs to be done to fix.

18 REPLIES 18

2013
Level 5

If any other log needs to be checked please update me.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@2013 

Please show us all text in Job Details before we download and unzip the attachment.

Tell us what the policy type is for GRT backup - Exchange? SharePoint? Something else?

What is the Application (Exchange, Sharepoint, etc) version of the client?
Which OS on Master, Media, and Client?

Have you followed NBU documentation to configure GRT backup for the relevant policy/client type? 

02/17/2020 06:38:17 - Info nbjm (pid=58463) starting backup job (jobid=34680353) for client client, policy policy, schedule Daily
02/17/2020 06:38:17 - Info nbjm (pid=58463) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=34680353, request id:{5F9A6A9A-5182-11EA-B527-09DEF5189302})
02/17/2020 06:38:17 - requesting resource DataDomain
02/17/2020 06:38:17 - requesting resource materserver.NBU_CLIENT.MAXJOBS.client
02/17/2020 06:38:17 - requesting resource masterserver.NBU_POLICY.MAXJOBS.GRTPolicy
02/17/2020 06:38:18 - granted resource  masterserver.NBU_CLIENT.MAXJOBS.client
02/17/2020 06:38:18 - granted resource  masterserver.NBU_POLICY.MAXJOBS.GRTPolicy
02/17/2020 06:38:18 - granted resource  MediaID=@aaaaf;DiskVolume=disk volume;DiskPool=pool;Path=path;StorageServer= storage server;MediaServer=media server
02/17/2020 06:38:18 - granted resource  DataDomain
02/17/2020 06:38:18 - estimated 34759394 kbytes needed
02/17/2020 06:38:18 - Info nbjm (pid=58463) started backup (backupid=client_1581943098) job for client client, policy GRTpolicy, schedule Daily on storage unit DataDomain
02/17/2020 06:38:19 - Info bpbrm (pid=29377) client is the host to backup data from
02/17/2020 06:38:19 - Info bpbrm (pid=29377) reading file list for client
02/17/2020 06:38:19 - started process bpbrm (pid=29377)
02/17/2020 06:38:20 - Info bpbrm (pid=29377) starting bpbkar on client
02/17/2020 06:38:20 - connecting
02/17/2020 06:38:20 - connected; connect time: 0:00:00
02/17/2020 06:38:21 - Info bpbkar (pid=6292) Backup started
02/17/2020 06:38:21 - Info bpbrm (pid=29377) bptm pid: 29394
02/17/2020 06:38:21 - Info bptm (pid=29394) start
02/17/2020 06:38:21 - Info bptm (pid=29394) using 1048576 data buffer size
02/17/2020 06:38:21 - Info bptm (pid=29394) using 512 data buffers
02/17/2020 06:38:22 - Info bpbkar (pid=6292) change time comparison:<disabled>
02/17/2020 06:38:22 - Info bpbkar (pid=6292) archive bit processing:<enabled>
02/17/2020 06:38:22 - Info bptm (pid=29394) start backup
02/17/2020 06:38:23 - Info bptm (pid=29394) backup child process is pid 29399
02/17/2020 06:38:23 - begin writing
02/17/2020 06:59:23 - Info bpbrm (pid=29377) from client *********: TRV - Starting granular backup processing for (System State\Active Directory). This may take a while... 
02/17/2020 07:15:34 - Info bpbrm (pid=29377) from client *******: TRV - Granular processing failed! 
02/17/2020 07:15:34 - Error bpbrm (pid=29377) from client *********: ERR - Error encountered while attempting to get additional files for System State:\
02/17/2020 07:19:28 - Info bptm (pid=29394) waited for full buffer 4451 times, delayed 149817 times
02/17/2020 07:19:28 - Info bptm (pid=29394) EXITING with status 0 <----------
02/17/2020 07:19:29 - Info bpbrm (pid=29377) validating image for client ********
02/17/2020 07:19:30 - Info bpbkar (pid=6292) done. status: 1: the requested operation was partially successful
02/17/2020 07:19:30 - end writing; write time: 0:41:07
The requested operation was partially successful  (1)

Policy Type - :MSWindows / Enable grandular recovery is Enabled

GRT Backup : Domain Controller

OS : Master Server Linux

Media Server Linux

Client : Windows 2016

Name Change DiskVolume=disk volume;DiskPool=pool;Path=path;StorageServer= storage server;MediaServer=media server, storage unit DataDomain,Client: client, Policy : GRTPolicy

I found something in the logs:

06:38:48.977 [6292.4360] <2> ov_log::V_GlobalLog: INF - VSS snapstarted. SnapshotSetID = {6cdfe0bf-a46a-4d7a-8db6-5f5489822f5f}
06:38:48.977 [6292.4360] <2> ov_log::V_GlobalLog: INF - VssSnapshotVolume::GatherWriterMetadata()
06:38:48.977 [6292.4360] <2> ov_log::V_GlobalLog: INF - VssSnapshotVolume::GetWriterMetaData() - calling IVssBackupComponents::GatherWriterMetadata.
06:38:48.977 [6292.4360] <2> ov_log::V_GlobalLog: INF - VssSnapshotVolume::GetWriterMetaData() - waiting for IVssBackupComponents::GatherWriterMetadata to complete.
06:38:50.695 [6292.4360] <2> ov_log::V_GlobalLog: INF - VssSnapshotVolume::GetWriterMetaData() - IVssBackupComponents::GatherWriterMetadata complete.
06:38:50.695 [6292.4360] <4> dos_backup::tfs_scanstart: INF - starting scan for file directive:<System State:\>
06:38:50.695 [6292.4360] <4> dos_backup::tfs_scanstart: INF - starting scan using name:<System State:\>
06:38:50.695 [6292.4360] <4> dos_backup::tfs_scanstart: INF - in 'pre-backup' mode
06:38:50.727 [6292.4360] <4> dos_backup::tfs_startdir: INF - volume guid:<>
06:38:50.727 [6292.4360] <4> dos_backup::tfs_startdir: INF - volume mount point:<>
06:38:50.727 [6292.4360] <2> ov_log::V_GlobalLog: INF - v_beds::V_FindFirst() ENTER Name:System State: Mode:0
06:38:50.727 [6292.4360] <2> ov_log::V_GlobalLog: DBG - v_beds::V_FindFirst() Device:System State: Obj: Pattern:(null)
06:38:50.727 [6292.4360] <2> ov_log::V_GlobalLog: INF - BEDS Resource ID - 0xF36
06:38:50.727 [6292.4360] <2> ov_log::V_GlobalLog: INF - BEDS Resource ID - 0xF36
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - Successfully deattached from Device 'System?State' BackupReason:0x1
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - FS_FshGen: Failure to create debug trace object at line number 84
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - AD:Attach reason (0x400)
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - Setting bSystemStateBackup = true
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - AD:Status not performing account check. System is not a 2003 Domain Controller in SystemState::PerformAccountCheck:1141
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - Starting to generate file list based on SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshot. (Function: LoadExcludeFiles, Line: 32)
06:38:50.758 [6292.4360] <2> ov_log::V_GlobalLog: INF - Started enumerating all UserProfile directories in C:\Users\*. (Function: GetUsersProfiles, Line: 237)
 

Michal_Mikulik1
Moderator
Moderator
Partner    VIP    Accredited Certified

Hello,

pay attention to your configuration. You must have MS-Exchange-Server policy type. When you have MS_Windows policy type, then GRT is valid on domain controllers only (for AD data like users, computers, etc.)

Regards

Michal

Its for Domain Controllers. By mistake i mentioned Exchange. Sorry for that

Hamza_H
Moderator
Moderator
   VIP   

Hi,

are you sure it is an exchange backup? you said that the policy type is Ms-Windows with granular recovery enabled:

Policy Type - :MSWindows / Enable grandular recovery is Enabled

GRT Backup : Exchange

 

If it is an ms-windows backup with active directory backup, please refer to nbu administration for AD and follow the best practices.

 

BR

Policy Type - :MSWindows / Enable grandular recovery is Enabled

GRT Backup :Domain Controllers

In my environment there are other server where backup completed successfully but this server is Partially completed with Error 1. I have already attached bpbkar logs. If anything needs to be check please let me know.

Thanks.

 

Hamza_H
Moderator
Moderator
   VIP   

To perform Active Directory granular backups and restores, ensure that you meet the following requirements:

1-The Network File System (NFS) must be installed on the media server and all Active Directory domain controllers or ADAM/LDS hosts.

2- The NetBackup Client Service must be configured to log on as an account with domain privileges.

for further informations, please take a look on :

System requirements for Active Directory granular NetBackup backups and recovery: https://www.veritas.com/support/en_US/doc/18716246-126559472-0/v20990572-126559472

 

BR

I went through the logs which i have attached and i have found the Veritas TechNote https://www.veritas.com/support/en_US/article.100041839

I have some questions regarding this Technote:

1. I am not getting NBU 7.7.3 = ET 3940293. Can someone send me the direct link to download. Client in Win2016.

2. If i follow the procedure then it will take backup of the location which i provide in Regedit.

Give ADlocation a value of the new path location (Example: C:\tempad)

Lets suppose there is a addition in Active Directory data set (by default located within C:\Windows\ntds). Then what needs to be done. Will it take backup if AD increases? else we have to manually copy and paste C:\Windows\ntds to C:\tempad?

Please suggest.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

To download updates and hotfixes, you need to logon to Veritas Support site:  https://www.veritas.com/support/en_US

Click on Downloads.

Select NetBackup under 'Products' and 7.7.3 under 'Version, then click Explore. 

Hotfixes and EEBs should be under Updates, but I do not see the Article number or the ET number in the list.

This means that you need to log a support call with Veritas. Only problem is that 7.7.3 is already out of support.

According to the Article, the EEB is not needed for NBU 8.1.1 or later.

So, only when you have applied the EEB or upgraded NBU can you implement the alternate method of the Active Directory granular backup

You never need to copy and paste anything in C:\tempad. The 'offline copy' will be made at time of backup (that is my understanding from reading the Article). You just need to ensure that the temp location (e.g. C:\tempad) is big enough. 

Thanks Marrianne.

Lets suppose i have the EEB pack and i have implement it on Client and i followed the below steps which is mentioned in Article. After performing all the below steps.

To determine how much space is needed, identify the size of the Active Directory data set (by default located within C:\Windows\ntds).  You'll need to create a temporary folder for NBU to use, on a volume with sufficient space to hold the copy of AD.

  1. On the client, create a folder for NetBackup to place the temporary copy of AD (Example: C:\tempad)

NOTE: This folder must be local to the client

NOTE: This folder must have sufficient space to hold a copy of AD

NOTE: This folder must not reside on an NFS mounted volume

NOTE:  No Active Jobs should be running on the client at the time of this configuration

  1. Open the registry editor on the client
  2. Navigate into: HKLM\Software\Veritas\NetBackup\CurrentVersion\Config
  3. Inside Config, create a new REG_SZ (string) value called: ADlocation
  4. Give ADlocation a value of the new path location (Example: C:\tempad)
  5. The next backup should use the new AD location and the granular processing error should no longer occur.

-----------

So we donot have to worry about Active Directory data set (by default located within C:\Windows\ntds).?

As per the new setting in Registry AD backup will happen from (Example: C:\tempad). 

In future if AD inceases on default dir by default located within C:\Windows\ntds). Will (Example: C:\tempad) will have all the changes which took place in future?

NetBackup uses the local path you specify to temporarily copy AD files from the backup image so that the Jet engine doesn't have to access them over NFS. NetBackup deletes the local copies at the end of the backup.

You do still need to have Client for NFS on the client and Server for NFS on the media server, as they are involved in copying the files from the backup image to the temporary local path.

(Yes, this is the same Jet engine that Microsoft uses for Exchange.)

Are you sure the TechNote applies to your situation?

I don't see an error in the bpbkar log snippet you posted. Keep reading until you find the granular processing failure.

This comes from bpkar32.exe. You should be able to match it in the bpkar log:

>Info bpbrm (pid=29377) from client *******: TRV - Granular processing failed

Start there and look upstream for the error. In particular, look for a JET error to establish the applicability of the TechNote. In the EEB ETrack document that you referenced, I find the following error:

>JET_ERR DbTable::Open -> -1090(0xfffffbbe) = -1090 This instance cannot be used because it encountered a
fatal error.
>SES=112658720,DB=1 - Z:\inprogress\astdc1adc01.external.astad.qa\allusers\full\1516181350\_vv_ASTDC1ADC010\System State\Active Directory\ntds.dit ()

The Z:\ drive in this case was the NFS-mounted path to the image on the media server. The "_vv_" part of the path stands for "virtual view" which means nbfsd provides filtered I/O to the Jet engine.

The following are not errors:
>FS_FshGen: Failure to create debug trace object at line number 84
>AD:Status not performing account check. System is not a 2003 Domain Controller in SystemState::PerformAccountCheck:1141

I have attached bpbkar logs. Please review once.

01:29:24.988 [8336.6900] <2> ov_log::V_GlobalLog: INF - DSProv:Exception (-1090) opening [Z:\inprogress\DELLUTLEXDCCP01\allusers\full\1582094441\_vv_DELLUTLEXDCCP010\System State\Active Directory\ntds.dit] at B:248
01:29:24.988 [8336.6900] <2> ov_log::V_GlobalLog: INF - JET_ERR DbTable::Open -> -1090(0xfffffbbe) = -1090 This instance cannot be used because it encountered a fatal error. ! SES=80349472,DB=1 - Z:\inprogress\DELLUTLEXDCCP01\allusers\full\1582094441\_vv_DELLUTLEXDCCP010\System State\Active Directory\ntds.dit ()!
01:29:24.988 [8336.6900] <2> ov_log::V_GlobalLog: INF - Error on close -1090!
01:29:24.988 [8336.6900] <2> ov_log::V_GlobalLog: INF - Error on detach -1090!
01:29:24.988 [8336.6900] <2> ov_log::V_GlobalLog: INF - DSProv:Error -1090 closing db at B:308
01:29:24.988 [8336.6900] <2> ov_log::V_GlobalLog: INF - Terminating instance ZinprogressDELLUTLEXDCCP01allusersfull1582094441_vv_DELLUTLEXDCCP010System StateActive Directory!.
01:29:26.649 [8336.6900] <2> ov_log::V_GlobalLog: INF - DSProv:Failed to create the instance for [Z:\inprogress\DELLUTLEXDCCP01\allusers\full\1582094441\_vv_DELLUTLEXDCCP010\System State\Active Directory\ntds.dit] at P:93
01:29:26.649 [8336.6900] <2> ov_log::V_GlobalLog: INF - ADGran: Status SUCCESS (0x00000000) no provider enumerator in getADGInstanceName:702. Attach reason = 0x0001, Provider status = 0x000C
01:29:26.649 [8336.6900] <2> ov_log::V_GlobalLog: INF - ADGRAN : Attaching to ADGRAN with attach reason = 1, isLiveDB = 0, isADDomain = 0
01:29:26.649 [8336.6900] <2> ov_log::V_GlobalLog: INF - BEDS Resource ID - 0xF35
01:29:26.649 [8336.6900] <2> ov_log::V_GlobalLog: INF - ADGran: VFF error in ADGran_ChangeDir:104. Provider status = 0x000C
01:29:26.649 [8336.6900] <2> Root::start(): DBG - FS_ChangeDir() Failed! (0xE0008488:Access is denied.) (../Root.cpp:429)
01:29:26.649 [8336.6900] <2> ov_log::V_GlobalLog: ERR - raiPdiEnumerate():start() failed, error = 17

I was trying to encourage you on how to read the bpbkar log for yourself.

You found the Jet error -1009. Please proceed with the ADLocation setting in accordance with the TechNote.

Hamza_H
Moderator
Moderator
   VIP   

Hi,

as described by the technote and refering to the same error that you got, just go through the steps from the technote (I have already done it before and everything works just fine).

for the eeb, I can download it and share it with you, but I don't know if i'm allowed to do it.. can anyone confirm this please @Lowell_Palecek , @Marianne ?

 

I don't know the rules for downloading or distributing an EEB. I work on the other end of the pipe.

Thanks all valable comment and looking into it. I have downloaded the EEB. I will update you once done.