Query Abot Importing Image in NBU.

[Possible]

Hi All,

Please help me to understand the concept of import.

The image expiration is the time at which NBU no longer protects this data from being ovewriten

and can purge the metadata about this image from its DB. (Means when the retention is over NBU will delete this whole information about backup image

from catalog / metedata from master server).

So in order to run the import of the expired images where the NBU looks for the information ?

What I know about this is the media on which original backup resides have not been overriten so if the media is overriten there is no chance of importing the images (NBU look only and only towards the media for importig the images ?)

Phase 1 creates the catalog entires for backups by populating Media Server Media Db (mediaDB) with media id and creating the header file of the backup images on the media.

Thanks,

Giridhar.

[mph999 · Accepted solution]

You have more or less answered this yourself - NBU looks at the media to import images.

"Phase 1 creates the catalog entires for backups by populating Media Server Media Db (mediaDB) with media id and creating the header file of the backup images on the media."

Almost correct, there is no 'media server DB' at NBU 6.x and later.  At NBU 6.0 to 7.1 the images catalog is located in /usr/openv/netbackup/db/images

At NBU 7.5, the image catalog .f files are located in /usr/openv/netbackup/db/images, but the actual header file is located in the EMM databse.

NBU looks for information on the tape.

When th eimages is expired from the catalog, NBU knows absolutly nothing about the data on the tape, the only place it can look, is on the tape.

Phase 1 import is similar to runing bpmedialist -mcontents.

NBU scans through the tape fairly quickly, reading only the backup headers.  A backup header is written on the tape before the files are copied to the tape, and contains various information including the backup id and the retention period.

Reading this information allows NBU to recreate part of the image header file.

Once this is complete, you search for the images that have been phase 1 imported and select those you wish to run phase 2 on.  When you run phase 2, NBU reads through all the files for that image (it can locate the image from the backup header/ information recreated in the image header file).  As it reads the files, it recreates the image .f file in the catalog, as well as adding further information to the image header file.

At NBU 7.5 the image header file is included in the EMM database, but the theory is the same, oonly at 7.5 the imforamtion is recreated in the EMM database.  At 7.5, the image .f file is still located in the same place as previous versions.

Regards,

martin

[mph999]

Further detail for you on the header files being included in the EMM database, in case you are interested.

If you run nbdb_unload <some dir> the database is unloaded, and the dir will contain many .dat files, these are the DB tables.  There is also a reload.sql files that contains what is in which DB table.

It depends on the DB schema, so this 'will' change on upgrades, but on my system :

 783  DBM_MAIN     DBM_Image

 787  DBM_MAIN     DBM_ImageFragment

 783.date is the image data, and 787.dat us the Fragment data.

 Taking  a quick look :

 787.dat

  1: "ImageFragmentKey"               unsigned bigint    = '123'

  2: ,"ImageKey"                      unsigned bigint    = '126'

  3: ,"ImageCopyKey"                  unsigned bigint    = '115'

  4: ,"CopyNumber"                    integer            = '1'

  5: ,"FragmentNumber"                integer            = '1'

  6: ,"ResumeCount"                   integer            = '0'

  7: ,"MediaID"                       varchar(1024)      = 'R0TP01'

  8: ,"MediaServerKey"                unsigned int       = '1000011'

  9: ,"StorageUnitType"               integer            = '2'

 10: ,"StuSubType"                    smallint           = '1'

 11: ,"FragmentState"                 smallint           = '1'

 12: ,"FragmentSize"                  bigint             = '32768'

 13: ,"FragmentID"                    varchar(4096)      = 'R0TP01'

 14: ,"Density"                       integer            = '12'

 15: ,"FileNum"                       integer            = '1'

 16: ,"BlockSize"                     integer            = '65536'

 17: ,"Offset"                        integer            = '2'

 18: ,"MediaDate"                     bigint             = '1337635216'

 19: ,"DeviceWrittenOn"               integer            = '0'

 20: ,"FFlags"                        integer            = '0'

 21: ,"MediaDescription"              varchar(1024)      = ''

 22: ,"FragmentCheckpoint"            smallint           = '0'

 23: ,"MediaSequenceNum"              integer            = '0'

 24: ,"MediaExtents"                  varchar(4096)      = ''

 25: ,"SnapshotClientMountHost"       varchar(1024)      = ''

 26: ,"CreatedDateTime"               timestamp          = '2012-05-21 21:21:33.242832'

27: ,"LastModifiedDateTime"          timestamp          = '2012-05-22 09:34:39.255306'

783.dat 

1: "ImageKey"                       unsigned bigint    = '126'

 2: ,"MasterServerKey"               unsigned int       = '1000002'

 3: ,"PreviousBackupImageKey"        unsigned bigint    = 

 4: ,"ParentImageKey"                unsigned bigint    = 

 5: ,"CatArcImageKey"                unsigned bigint    = ,'womble

 6: ,"FullBackupImageKey"            unsigned bigint    = '1'

 7: ,"CSScheduleKey"                 unsigned int       = 'robot_0'

 8: ,"ClientMachineName"             varchar(1024)      = '1337635216'

 9: ,"ClientKey"                     unsigned bigint    = '0'

10: ,"PolicyName"                    varchar(1024)      = '0'

11: ,"BackupTime"                    bigint             = '0'

12: ,"ViewID"                        unsigned bigint    = '10'

13: ,"ScheduleType"                  integer            = '0'

14: ,"ClientType"                    integer            = '0'

15: ,"ProtocolVersion"               integer            = '0'

16: ,"StartTime"                     bigint             = '32'

17: ,"EndTime"                       bigint             = '1'

18: ,"SnapTime"                      bigint             = '1'

19: ,"KiloBytes"                     bigint             = '0'

20: ,"NumberFragments"               integer            = '9'

21: ,"NumberCopies"                  integer            = '1'

22: ,"NumberSnapshotDevices"         integer            = '0'

23: ,"ImageVersion"                  integer            = '0'

24: ,"RetentionLevel"                integer            = '0'

25: ,"Compression"                   smallint           = '0'

26: ,"Encryption"                    smallint           = '0'

27: ,"FilesFileCompressed"           smallint           = '0'

28: ,"Mpx"                           smallint           = '1'

29: ,"TirInfo"                       integer            = '0'

30: ,"TirExpiration"                 bigint             = '77'

31: ,"PrimaryCopy"                   integer            = '1338844816'

32: ,"ImageType"                     integer            = '6'

33: ,"ElapsedTime"                   integer            = '0'

34: ,"Expiration"                    bigint             = '0'

35: ,"NumberFiles"                   integer            = '0'

36: ,"ExtSecInfo"                    integer            = '0'

37: ,"RequestPID"                    integer            = '0'

38: ,"IndividualFileRestoreFromRaw"  integer            = '0'

39: ,"ImageDumpLevel"                integer            = '0'

40: ,"FileSystemOnly"                integer            = '0'

41: ,"PrevBlkIncrTime"               bigint             = '0'

42: ,"BlkIncrFullTime"               bigint             = '0'

43: ,"StreamNumber"                  integer            = '787'

44: ,"BackupCopy"                    integer            = '0'

45: ,"BackupStatus"                  integer            = '0'

46: ,"JobID"                         integer            = '0'

47: ,"NumResumes"                    integer            = '0'

48: ,"ResumeExpiration"              bigint             = 'root'

49: ,"PfiType"                       integer            = 'Full'

50: ,"ImageAttribute"                integer            = ''

51: ,"Creator"                       varchar(1024)      = ''

52: ,"ScheduleName"                  varchar(1024)      = 'robot_0_1337635216_FULL.f'

53: ,"ProxyClient"                   varchar(1024)      = '1312'

54: ,"KeywordPhrase"                 varchar(1024)      = ''

55: ,"FilesFile"                     varchar(1024)      = ''

56: ,"FilesFileSize"                 bigint             = '0'

57: ,"SoftwareVersion"               varchar(128)       = '0'

58: ,"ObjectDescriptor"              varchar(1024)      = '0'

59: ,"EstimatedKiloBytes"            bigint             = '1337635216'

60: ,"VmType"                        integer            = '1337635216'

61: ,"IsSynthetic"                   smallint           = '12'

62: ,"RunTime"                       bigint             = '1'

63: ,"LastBirthTime"                 bigint             = ''

64: ,"ClientCharacterSet"            integer            = 0x0000000000000000000000000000000

65: ,"BmrInfo"                       integer            = ''

66: ,"OriginMasterServer"            varchar(1024)      = '0'

67: ,"OriginMasterServerID"          binary(16)         = '0'

68: ,"StorageServiceName"            varchar(1024)      = '0'

69: ,"StorageServiceState"           integer            = ''

70: ,"StorageServiceIndexOffset"     integer            = '0'

71: ,"TimeInProcess"                 bigint             = '0'

72: ,"ClassificationID"              varchar(64)        = '0'

73: ,"StorageServiceVersionNumber"   integer            = '0'

74: ,"RequiredExpirationDate"        bigint             = '0'

75: ,"ImportFromReplicaTime"         bigint             = '0'

76: ,"IrEnabled"                     integer            = '0'

77: ,"IsNotQuiesced"                 smallint           = '0'

78: ,"IsNotSnapdupable"              smallint           = '0'

79: ,"IsNotScheduled"                smallint           = '0'

80: ,"HasDependees"                  smallint           = '0'

81: ,"HasDependents"                 smallint           = '0'

82: ,"ValidationCount"               integer            = '0'

83: ,"ExpirationCount"               integer            = '2012-05-21 21:20:17.182266'

84: ,"IndexingStatus"                smallint           = '2012-05-21 21:21:33.661940'

These outputs I have run through a perl script to make them easier to read, and they are just showing the details of one image/ fragement.

I've only had a quick look, but this looks like the tables that contain the 'old' image header file data.

There is a command:  cat_export that will 'copy' the details from the SQL database back into flat files.  It makes no changes, but just recreates the header files, for technical support use.

Martin

[Possible]

Thanks martin,

OK...so its clear now that NBU looks only and only towards the tape (Header file about the actual backup)

then its not possible to import the images if tape is overwriteen by another backup ,,correct ? or there any other way ?

Thanks,

Giri

[Marianne]

If data is overwritten the old data has been WIPED and replaced with the new data..

Think of old-days video tapes... If someone has taped over your favourite movie, it's gone forever...

[mph999]

It looks at the header file for phase 1.

For phase 2, it reads every file on the tape (for the images selected).

When you are happy, could you mark the post that helped you most as the solution - many thanks,

Martin