Forum Discussion

danbert73's avatar
danbert73
Level 3
8 years ago

Large restore challenge

I've been asked to complete a restore of multiple user drives, across multiple servers, over multiple years....I know...sounds like fun, right? Here are my challenges...

1. Generate a list of files to be restored. I have the user list, but they could be on any of 20 different client servers. I was thinking of using bplist to find them, but can't for the life of me get past the dreaded 227 error when I run it from the Master server. Oh, and some of those clients don't exist anymore....just the backups.

 - I've tried variations for bplist such as: bplist -S master -C client -t 13 -R c:\pathname on client

2. Use powershell, to take the list from above and generate the restores with the files being directed to another location using bprestore is my guess.

 

I'm running NBU 7.6.0.1 on W2K8 and my install path is not the default. 

 

Any help would be really appreciated as this is probably going to take weeks as it is.


  • cd to your working folder where your script resides...

    you should have two files already:
    list-of-source-clients.txt (contains 7 to 20 old server - i.e. backup client names)
    list-of-usernames.txt (contains 50 to 51 old usernames, partial strings, i.e. not full paths)


    ...then use this to generate a list of all MS-Windows type backup images in a two year date range:

    for each record in list-of-source-clients.txt
      extract clientname
      bpimagelist -idonly -d twoyearsagodatetime -e todaydatetime -pt MS-Windows -client $clientname > images.txt
      for each record in images.txt
        extract imagename
        extract imagedatetime
        for each record in list-of-usernames.txt
          extract username
          bplist -C $clientname -s $imagedatetime -e $imagedatetime -t 13 -I -PI -R 999 -nt_files *$username* >>files-$imagename.txt
        next
      next
    next

    ...then you would end up with a bunch of files named "file-*.txt" which contain a list of files to be restored from each image.
    The above was the easy part.


    Here comes the tricky part:

    dir /b files-*.txt > list-of-files.txt

    set targetserver = blahblah (i.e. the name of the server that restores will be written to)

    delete restore-commands.txt

    for each record in list-of-files.txt
      extract backupfilename
      extract imagename (from $backupfilename)
      extract clientname (from $imagename)
      extract imagectime (from $imagename)
      convert imagectime to imagedatetime

      set renamefile = restore-$imagename.txt

      for each record in $backupfilename
        extract sourcefilename
        set sourcelen = len( sourcefilename )

        set targetfilename = "F:\$imagename\" + Mid( $sourcefilename, 4, 999 )
        set targetlen = len( targetfilename )

        write $renamefile "rename $sourcelen $sourcefilename $targetlen $targetfilename"
      next


      write restore-commands.txt "bprestore -B -R $renamefile -C $clientname -s $imagedatetime -e $imagedatetime -f $sourcefilename -D $targetserver"
    next


    FYI - it is important that there is a one-to-one relationship between each file name entry in:
    -f sourcenames
    -R renamenames
    ...and so the above should achieve:
    - separate restore folder for each file in backups
    - restored folder structure maintainted underneath F:\imagenames\

    - and then you can run a tool to detect duplicate files and delete, so that only unique files are left.

    - and a list of restore commands in "restore-commands.txt"

    ...and so could pick the first one to test an actual restore.

    It may be possible to also capture and calculate the total size of what is to be restored. But I'll let you work out how to do that :)

    HTH.

    • danbert73's avatar
      danbert73
      Level 3

      Hey Nicolai, thanks for the link. It definitely did help. I'm still trying to figure out how to browse a list of clients to  figure out which one the User's drive resided on. 50 users out of 10,000 isn't that easy for sorting.

  • Do you have Opscneter in place...if yes use the restore option in Opscenter.. it will search for you if you give the directory or file name for restore  

    you might need to give the client names and dirctory or file names to search.. you can trigger the restore from Opscneter as well.. 

    even if you get issues triggering the restores from Opscneter.. at lest you will get to know what are the clietns havign the data and date details... to trigger them from Master server... 

  • hey, arent you the lucky one !!!!!

    i have underwent similar case, first things first

    are you restoring from tapes? or disk (may be DataDomain or something), if its tape then you have more trouble in identifying the tapes needed for restore, if its DISK then its lots simpler

     

    1) is your source files under diffrent servers same ? (with user name diffrents)

    2) do you have the dates from which your planning to restore

    3) chose same destination path (e:\restore\) for all restoer you want to perform.

    4) use bprestore command to restore the file 

    bprestore -s <start date> -e <end date> -t 13 -C <source client> -D <dest. client> -L c:/temp/restore/logs/Restore1.log -R <path to file name where you have defined restore destiantion> -f <path of teh file name where you defined list of files> 

    restore command example : from which i did (it was unix though)

    /usr/openv/netbackup/bin/bprestore -s 3/26/2015 -e 3/26/2015 -t 13 -C Source_Client -D Dest_client -L /tmp/restore/logs/Restore_`date +%h%d_%R`.log -R /tmp/restore/alt_location -f /tmp/restore/filelist

    • danbert73's avatar
      danbert73
      Level 3

      Thanks for the info....but still looking for more. 


      I need to figure out which client (out of 7) the user existed on during the time frame. My date range is everything that was created or modified over a two year period, and the destination will be a different server entirely. The date range can be extrapolated via bplist, so really it will just be finding their original client. Powershell will end up being my friend by the end of this.

      Tapes are tapes....we'll be loading lots of them for this restore.

      • sdo's avatar
        sdo
        Moderator

        for each record in list-of-7-clients

          read clientname

          bpimagelist -idonly -s twoyearsago -e today -pt MS-Windows > images-$clientname.txt

          for each record in images-$clientname.txt

            extract imagedatetime

            for each record in list-of-usernames.txt

              extract username

              bplist -C $clientname -t 13 -s $imagedatetime -e $imagedatetime -nt_files -I -PI -R 999 *$username* >> files-$clientname.txt

            next

          next

        next

        for each filename in files-*.txt

          extract clientname from filename

          create a restore-$clientname.txt

          call bprestore

        next

         

         

         

  • Is the list of users in a form that actually matches actual known folder names, or are they a bit of a guess?
    Are you quite skilled with PoSh?
    • sdo's avatar
      sdo
      Moderator
      Are all of the username based folders at the same level of folder depth no matter which server they were on at the time of backup?
    • danbert73's avatar
      danbert73
      Level 3

      Hey SDO,

      Thanks, for the most part they should be known users with known folder paths. I have some PoSh knowledge and can get help as necessary. PowerwShell was the direction I was thinking of going, so any help will be appreciated.

       

      Dan

  • Have you confirm that the Client name used in bplist corresponds with client name used at time of backup?

    Can you browse ...netbackup\db\images folder and locate correct client name in subfolders?
    Bear in mind that NBU is case sensitive and you need to determine if clients were baked up using shortname or FQDN.
    Client1 != client1 != client1.fqdn

    Once you have located the client folder under images, can you browse the 10-digit-number-folders in the client folder and see valid catalog entries?

    Once you have correct client folder name and 10-digit folder containing valid images, you can convert the timestamp for one of the oldest images with bpdbm command:
    bpdbm -ctime <10-digit-number>

    e.g. 

    C:\Program Files\VERITAS\NetBackup\bin>bpdbm -ctime 1165000000  
    1165000000 = Fri Dec 01 13:06:40 2006
     
    So, now we have a client name and timestamp.
    Try to browse top 3 levels of files/folders for this client for a given period (like 6 months), e.g:
    bplist -C <client-name> -s 12/01/2006 -e 06/01/2007 -R 3 / 
     
    Once you have output other than status 227 we can proceed to dig deeper... 
    • danbert73's avatar
      danbert73
      Level 3

      Hey Marianne,

      After a small amount of fighting, I have  been able to rid myself of the nasty 227 error, thanks.So now I can get an output list of all the files across all the backups that i'll need. So now, it comes to how do I restore all the files, across all the versions of backups, for the 51 users that are required?