Forum Discussion

spying's avatar
spying
Level 3
12 years ago

Upgraded from 7.1.0.1 to 7.5.06 - Prev images not found in BAR

Hi,

With the help of symantec technical support, we upgraded our microsoft clustered master server from 7.1.0.1 to 7.5.0.6.

The upgrade process was far from being a breeze even wtih technical support webex set up.

Just a summary of the issue while upgrading.

  1. Start upgrading my OpCenter to 7.5 which is installed on 1 of my media servers - This hung at the db upgrade stage for over 2-3 hrs, tech support told me to end task and log a case with the opscenter team.
  2. Proceeded to upgrade my master servers - The setup.exe crashed at 73%. However, tech support went and started the services and launched admin console. All seems to be working. I suggested if we could do a repair install to make sure all is well but tech support said to create a test backup job and if that works, it should be fine. I'm still skeptical about this but just went ahead as recommended as it was almost time for my backup jobs to start running.  After upgrading my active node, we proceeded to upgrade the passive node. The setup.exe again crashed at 73%. Tech support said to just proceed.
  3. Patched both active and passive nodes to 7.5.0.6 with no issues.
  4. Phase 1 and Phase 2 seems to have completed without errors. I have abt 30,000 images.

Now, my main issue here. I was browsing my backup images and realized that i can only see backups done last night. All my previous backups could not be found.

I have the catalog backup of before the upgrade but would like to know if there is anything else we can do without setting up a 7.1 from scratch and recovering the catalog.

I dont suppose there's an option of restore just the image db from my 7.1 catalog backup to the current 7.5.0.6 setup?

Overall, the upgrade has been a nightmare for me. Was on the line with tech support for almost 6 hours straight..

7.5
Backup and Recovery
NetBackup
Troubleshooting
Upgrade
Windows Server (2003-2008)
  • spying's avatar
    spying
    12 years ago

    SLP_DSSU_MIGRATION_STATE="1"
    LIST_FS_IMAGE_HEADERS="0"

     

    Anyway, i have since uninstalled, reinstalled 7.1, restored catalog, and performed the upgrade.

    Took a full 8 hrs, luckily this time it managed to import the old backup images correctly.

    I think there's a technote with regards to the setup.exe crashing in 2008 R2 and the resolution was to reboot the server, then perform the install again.

    So lessons to be learnt.

    1. Reboot your windows server prior to any major upgrade to be safe
    2. Always have a latest catalog backup, i had mine backed up to disk so the restore was relatively smooth.

    Thanks for the help guys.

17 Replies

Sort By
  • Nicolai's avatar
    Nicolai
    Moderator
    12 years ago

    Before panic - The GUI is exposed to the names you are using for browising backup. srv1 - srv1.acme.com or SRV1 are not the same servers. Maybe the upgrade changed default name.

    Check prvious backup with this command utility:

    bpimagelist -d 06/01/2013  -U (or -L or -l) 

    You might find this T/N usefull : 

    http://www.symantec.com/docs/TECH91133

    http://www.symantec.com/docs/HOWTO43661

    Update:

    Try this from the master server

     bpclimagelist -client {CLINET_NAME_FROM_POLICY}  -d 01/01/2013

    should output somthing like:

    Backed Up         Expires       Files      KB     C Sched Type   Policy
    ----------------  ---------- -------- ----------- - ------------ ------------
    07/08/2013 16:10  07/15/2013   100000   102450016 N Full Backup  GEN_DATA
    07/07/2013 16:10  07/14/2013   100000   102450016 N Full Backup  GEN_DATA
    07/06/2013 16:10  07/13/2013   100000   102450016 N Full Backup  GEN_DATA
    07/04/2013 16:10  07/11/2013   100000   102450016 N Full Backup  GEN_DATA
    07/03/2013 16:10  07/10/2013   100000   102450016 N Full Backup  GEN_DATA

     

  • spying's avatar
    spying
    Level 3
    12 years ago

    I did a bpimagelist and only images from yesterday are listed..

    Same with bpclimagelist

     

  • Gautier_Leblanc's avatar
    Gautier_Leblanc
    Level 5
    12 years ago

    Are you sure that your backups metadata have been migrated into relational database (NBEMM) ?

    Have you an "image cleaning" job after 1st NetBackup 7.5 start ? What is his status ? Logs say that images are migrated into NBEMM ?

    You can also look in the <NBU Install dir>\db\images if you have only .lck and .f files..

  • spying's avatar
    spying
    Level 3
    12 years ago
    Yeah, here's the last bits of the cleanup log. 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) [000:05:35] Overall progress: 37370 images imported, 0 skipped, 0 corrupt. Import rate = 111 images/sec 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) [000:05:35] Initiating import for client: scene-main 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) [000:05:35] Finished importing images for client: scene-main with 13 imported, 0 skipped, 0 corrupt. 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) [000:05:35] Overall progress: 37383 images imported, 0 skipped, 0 corrupt. Import rate = 111 images/sec 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) Finished importing all images into the database. (Count = 37383) 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) Cleaning up tables in the relational database 7/8/2013 5:52:04 PM - Warning bpdbm(pid=4208) Hot catalog backup is not configured for 'nbu.xxx.grp', catalog cleanup will return partial success until hot catalog backup is configured. 7/8/2013 5:52:04 PM - Info bpdbm(pid=4208) deleting images which expire before Mon Jul 08 17:46:28 2013 (1373276788) 7/8/2013 5:52:47 PM - Info bpdbm(pid=4208) deleted 496 expired records, compressed 0, tir removed 0, deleted 467 expired copies the requested operation was partially successful(1) The job was successfully completed, but some files may have been busy or unaccessible. See the problems report or the client's logs for more details.
  • Gautier_Leblanc's avatar
    Gautier_Leblanc
    Level 5
    12 years ago

     

    Idiot question (but...) : Are you sure that your NetBackup Master service (or IP) is on the node that have the storage owning database files ?

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    The OpsCenter upgrade doesnt surprise me - i have seen the database upgrade part run for several hours, especially if you use analytics and have a lot of data in OpsCenter - they should have left that to run!

    Do all of your cluster resources appear in Cluster manager?

    The upgrade tends to delete and recreate them all - just wondering if during the hang something was missed - like a drive that held some of your catalogs - and on that thought .... a cluster would use a shared drive for the catalogs - say Y drive ... if the installation hung then maybe it is writing the catalogs to C:\Program files\veritas now and you just have a configuration / registry entry that is incorrect

    Take a look to see if last nights backups are in the wrong place - that may be easily fixable!

  • Gautier_Leblanc's avatar
    Gautier_Leblanc
    Level 5
    12 years ago

    I agree with Mark. That's why I spoke of storage on wrong node... It sounds like a loss of the storage that contains database.

  • spying's avatar
    spying
    Level 3
    12 years ago
    Storage owning the database files meaning the folder Netbackup\db\ ? Then yes. There's a dependency created by netbackup installer during cluster setup so the services will follow the shared disk and virtual name.
  • spying's avatar
    spying
    Level 3
    12 years ago

    I dont use analytics.

    Yes, my cluster resources appear in the cluster manager. This was also one of the initial issues and the tech support had to run bpclusterutil -c to recreate them.

    In my case, the cluster disk is H: and last nights backups do seem to be on the H: drive.

  • spying's avatar
    spying
    Level 3
    12 years ago

    Here's a pic of my current cluster manager.