Forum Discussion

Mark_Solutions's avatar
Mark_Solutions
Level 6
12 years ago

AIR Import Of BMR Data Unsuccessfull For Some Clients

This is already with Support (case 03591776) but wanted to put it out here as well to see if I can get it cracked.

I am pretty sure that this is an AIR issue more than a BMR issue but obviously there is a relationship between the two.

So the setup:

All a brand new installation

2 sites, each has 2 N5220 appliances (one Master / Media / BMR Master) one just a Media

All appliances on 2.5.1b with AIR EEB Bundle applied

All clients on 7.5.0.4 and set to collect BMR Data

One site is the main site and does most backups, no real backups done on site 2 as yet.

Backups and BMR collection work perfectly on the main site (just 8 Windows clients being backed up so far, mix of 2003 and 2008) and all show under BMR Clients section

AIR Replication works fine

AIR Import works fine in general but since starting to take BMR data some imports end with a Status of 1 - these relate to imports on the Master and Media Server at site 2.

Typical details of the import job here (edited names slightly that is all):

03/02/2013 20:10:07 - requesting resource @aaaaf
03/02/2013 20:10:07 - granted resource MediaID=@aaaaf;DiskVolume=PureDiskVolume;DiskPool=dp_disk_nbu02;Path=PureDiskVolume;StorageServer=nbu02;MediaServer=nbu02
03/02/2013 20:10:08 - begin Import
03/02/2013 20:10:08 - started process RUNCMD (15552)
03/02/2013 20:10:18 - Error bpimport(pid=15552) The replicated image has been imported successfully.. But failed to import bmr configuration of client client04 for backup id client04_1359921621.. Either BMR Master server is not configured on this setup or the service is down.. Please refer BMR administrator guide to know how to configure BMR Master server and import client BMR configuration manually.
03/02/2013 20:10:18 - import failed for backup id client04_1359921621 with status 1
03/02/2013 20:10:18 - Error bpimport(pid=15552) Imported 0 of 1 images successfully.      
03/02/2013 20:10:18 - Error bpimport(pid=15552) Imported 1 of 1 images partially successfully.     
03/02/2013 20:10:18 - Error bpimport(pid=15552) Status = the requested operation was partially successful.    
03/02/2013 20:10:18 - end Import; elapsed time: 00:00:10
03/02/2013 20:10:20 - Info bpdm(pid=31836) started           
03/02/2013 20:10:20 - started process bpdm (31836)
03/02/2013 20:10:20 - Info bpdm(pid=31836) reading backup image         
03/02/2013 20:10:21 - Info bpdm(pid=31836) origin master server nbu01, backup id client04_1359921621     
03/02/2013 20:10:23 - Info bpdm(pid=31836) EXITING with status 0        
03/02/2013 20:10:23 - Info nbu02(pid=31836) StorageServer=PureDisk:nbu02; Report=PDDO Stats for (nbu02): read: 3940 KB, CR received: 3939 KB, CR received over FC: 0 KB, dedup: 0.0%
the requested operation was partially successful(1)
The job was successfully completed, but some files may have been
busy or unaccessible. See the problems report or the client's logs for more details.

So out of the 8 clients being backed up and AIR replicated I have 3 on the remote site showing under BMR Clients.

Following the import the automatic restore job runs (for /C/Program Files/Veritas/NetBackup/BareMetal/client/data/bundle.dat) runs successfully.

So BMR is configure and working as it is all visible on the remote site and three clients have worked fine - so i need to track down why the others do not.

Any ideas?

Thanks

7.5
Backing Up
Backup and Recovery
Configuring
Integration
NetBackup
Troubleshooting
  • Jaime_Vazquez's avatar
    Jaime_Vazquez
    12 years ago

    The issue in question can occur on any Unix/Linux based NetBackup Master but only  for Windows based BMR registration clients.  The client configuration file being used is called "bundle.dat".  Consider it just another tar file. For registration at AIR sites, the process will perform an "alternate client" restore (i.e. BMR will do a restore of this file from the newly duplicated and imported backup image itself. Because it came from a Windows backup image, it has some Windows based "baggage" which in this case is a Windows file header. The import process is supposed to strip off the header field before importing into the BMRDB.  The routine that is supposed to strip the header somehow failed to do so.  The resulting file can not be imported as a consequence.

    To "see" what this means, use the command "file $client.bundle.dat.tmp" which will not show it as a tar file, but as some other file format.( if I remember correctly, a VAX executable). Of interest, the original reporting customer also had the  issue disappear when they set the debug level to a higher value.

    This is a known issue on appliance 2.5.1 and NetBackup 7.5.0.4 .  Contact Symantec support and reference ET3081314 for an Engineering fix binary.  .The fix is included in the NetBackup 7.5.0.5 release.

     

     

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    Ok - an update - I have been running all vxlogviews for all BMR processes and have the following outputs if they help (again slightly edited for client names) - these are from the Master Server doing the imports.

    The issue does seem to be here - but what does it actually mean? 

    nbu01:/ # vxlogview -o 119 -t 20:00:00
    02/06/13 01:00:52.669 [winStreamHeader.cpp:removeWinStreamHeader()] Failure read
    ing to end of file while processing file /usr/openv/netbackup/baremetal/server/d
    ata/client05.bundle.dat.
    02/06/13 01:00:52.670 [bmrd.cpp:ImportUpdateClientCurrentConfiguration()] Unable
    to remove Windows Stream Header from bundle file /usr/openv/netbackup/baremetal
    /server/data/client05.bundle.dat.
    02/06/13 09:06:18.486 [Error] V-119-6 Server network failure
    02/06/13 14:50:44.009 [Error] V-119-6 Server network failure

    nbu01:/ # vxlogview -o 123 -t 20:00:00
    02/06/13 15:45:07.224 [Error] V-123-48 Invalid table name specified.
    02/06/13 15:45:07.251 [Error] V-123-49 Operation failed.

    nbu01:/ # vxlogview -o 128 -t 20:00:00
    02/06/13 01:00:47.450 [fileAttribsCommon.cpp:FileExistsCommon()] Could not do st
    at on file /usr/openv/netbackup/baremetal/server/data/client05.bundle.dat
    02/06/13 01:00:48.157 [fileAttribsCommon.cpp:FileExistsCommon()] Could not do st
    at on file /usr/openv/netbackup/baremetal/server/data/rename.client05
    02/06/13 09:06:18.470 [sendrecv.cpp:RecvPacket()] bad header, rc=0
    02/06/13 14:50:43.994 [sendrecv.cpp:RecvPacket()] bad header, rc=0
     

    nbu01:/usr/openv/netbackup/baremetal/server/data # ls -l
    total 14732
    -r--r--r-- 1 root bin 3224 Sep 17 16:00 BMRDB.load.xml
    -r--r--r-- 1 root bin 720756 Sep 17 16:00 PnPDB_2003.xml
    -r--r--r-- 1 root bin 524577 Sep 17 16:00 PnpDB_2000.xml
    -r--r--r-- 1 root bin 526143 Sep 17 16:00 PnpDB_2000SP1.xml
    -r--r--r-- 1 root bin 526630 Sep 17 16:00 PnpDB_2000SP2.xml
    -r--r--r-- 1 root bin 526665 Sep 17 16:00 PnpDB_2000SP3.xml
    -r--r--r-- 1 root bin 530528 Sep 17 16:00 PnpDB_2000SP4.xml
    -r--r--r-- 1 root bin 770990 Sep 17 16:00 PnpDB_2003SP1.xml
    -r--r--r-- 1 root bin 770996 Sep 17 16:00 PnpDB_2003SP2.xml
    -r--r--r-- 1 root bin 1442188 Sep 17 16:00 PnpDB_WinPE_2003SP1.xml
    -r--r--r-- 1 root bin 541349 Sep 17 16:00 PnpDB_WinPE_2003SP1_X64.xml
    -r--r--r-- 1 root bin 1650573 Sep 17 16:00 PnpDB_WinPE_2008.xml
    -r--r--r-- 1 root bin 382651 Sep 17 16:00 PnpDB_WinPE_2008_X64.xml
    -r--r--r-- 1 root bin 730137 Sep 17 16:00 PnpDB_XP.xml
    -r--r--r-- 1 root bin 741026 Sep 17 16:00 PnpDB_XPSP1.xml
    -r--r--r-- 1 root bin 745725 Sep 17 16:00 PnpDB_XPSP2.xml
    -r--r--r-- 1 root bin 28587 Sep 17 16:00 RuleDB_WinPE_Compatibility.xml
    -r--r--r-- 1 root bin 4284 Sep 17 15:06 banner.img
    -r--r--r-- 1 root bin 16 Sep 17 15:06 boot.msg
    -r-xr-xr-x 1 root bin 44408 Sep 17 15:06 discoverDevices.linuxR_x86
    -r-xr-xr-x 1 root bin 49548 Sep 17 15:06 discoverDevices.linuxR_x86_32
    -r-xr-xr-x 1 root bin 44496 Sep 17 15:06 discoverDevices.linuxS_x86
    -r-xr-xr-x 1 root bin 41236 Sep 17 15:06 discoverDevices.linuxS_x86_32
    -r--r--r-- 1 root bin 1474560 Sep 17 16:00 dos.uncooked
    -rwx------ 1 root root 1811152 Feb 5 20:41 client04.bundle.dat.tmp
    -rwx------ 1 root root 302816 Feb 6 01:00 client05.bundle.dat.tmp
    -r--r--r-- 1 root bin 3985 Sep 17 16:00 unattend.txt
    -r--r--r-- 1 root bin 9554 Sep 17 16:00 unattend.xml
  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Size of client05.bundle.dat.tmp looks way smaller than client04 file...

    Maybe something wrong with network transfer or import...

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Have you tried to compare size of 'bundle.dat' with source master server?

  • mandar_khanolka's avatar
    mandar_khanolka
    Level 6
    12 years ago

    bundle.dat is inside client/data folder. Have you backed it up via your BMR enabled backup policy on primary?

    Thanks.

    Mandar

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    Backed up fine on primary site and populated the BMR Clients section - no errors on the BMR Jobs

    Have looked on client04 and client05 in the primary site and there bundle.dat files are that size - so sizes are correct.

    Still getting backups do the same thing today and still only 3 out of 8 clients showing on the remote site - and those three all uodated after last nights replications.

    Awaiting some testing Support are doing ...

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    Just an update ....

    Increased logging for 119, 128 and 226 to 6/6

    Also setup a 64 bit Windows SRT yesterday on the main site while continuing with the site setups (have made 32 bit as well today and same on secondary site)

    So obviously, as always seems to happen when you ramp the logging levels up, it all worked perfectly last night!!

    All imports worked and all clients are populated on both sites

    As Bob would say - "Veritas - ain't it the Truth!"

    Logs are off for analysis (and another case created this morning for some reason - so now 3630993) so will update you all when and if any reason is establish for the behavoiur I have seen.

  • Jaime_Vazquez's avatar
    Jaime_Vazquez
    Level 6
    12 years ago

    The issue in question can occur on any Unix/Linux based NetBackup Master but only  for Windows based BMR registration clients.  The client configuration file being used is called "bundle.dat".  Consider it just another tar file. For registration at AIR sites, the process will perform an "alternate client" restore (i.e. BMR will do a restore of this file from the newly duplicated and imported backup image itself. Because it came from a Windows backup image, it has some Windows based "baggage" which in this case is a Windows file header. The import process is supposed to strip off the header field before importing into the BMRDB.  The routine that is supposed to strip the header somehow failed to do so.  The resulting file can not be imported as a consequence.

    To "see" what this means, use the command "file $client.bundle.dat.tmp" which will not show it as a tar file, but as some other file format.( if I remember correctly, a VAX executable). Of interest, the original reporting customer also had the  issue disappear when they set the debug level to a higher value.

    This is a known issue on appliance 2.5.1 and NetBackup 7.5.0.4 .  Contact Symantec support and reference ET3081314 for an Engineering fix binary.  .The fix is included in the NetBackup 7.5.0.5 release.

     

     

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Mr BMR himself!

    We are honoured to have you visit our community, Jaime!