Testing 7.6 upgrade - errors running bpimage -cleanup -allclients

[SteveLaw]

Hi,

I've cloned my nbu 7.5.0.6 master server (win 2008 R2) in an isolated network on ESX (bit of a saga this) and have followed instructions to upgrade to 7.6.1. The binaries upgraded ok. Then I tried to run phase 2 of the image metadata migration process - as I understand it this is a conversion of the 7.5.0.6 catalog to the 7.6.1 format, right? I wanted to test the process as it is disk-intensive and I want some idea of how long it will take when I do it live.
I ran bpimage - cleanup -allclients as per instructions in NetBackup7.6.1_Upgrade_Guide.pdf. 
It only ran for 3 mins and gave errors. Here's a pic of the results (can't cut and paste from the vm): 

I was expecting this to run for hours. It seems to trying to connect to our Data Domain storage @aaaab and is complaining that the disk volume is down. My understanding was that this process is addressing the catalog stored locally on the master server, and won't need to connect to other storage devices. As the server is an isolated clone it can't conect to anything. 
Can I not test the upgrade and image migration process this way? What is actually happening when I run bpimage - cleanup -allclients ?

Thanks! 

[Marianne]

The command will kick of Image Cleanup similar to the 12-hourly job that you see in Activity Monitor.

For tape, it will just cleanup expired catalog entries and deassign tapes.

For disk, the master will connect to the media server to cleanup/remove disk images.

Because you are simulating the environment in a lab, all the components will not be available and you can ignore these messages.

[Steve_Law1]

Hi Marianne,
Thanks for your reply! 
I was expecting this to run for hours. I have upgraded from NBU 7.5.0.6 to 7.6.1. Pages 33-34 of the NetBackup7.6.1_Upgrade_Guide.pdf have a frightening graph estimating the amount of time for the image metadata migration- based on my 1.7million images it estimates the simple migration process to take anywhere between 4 & 8 hours. But the process I just ran (bpimages -cleanup -allclients, which is phase 2 of the catalog migration) took 3 minutes...! Why is that?

Why does the upgrade require running the bpimages cleanup? Have I now completed the upgrade on my test master?

To check it says to run nbemmcmd -listsettings -brief -machinename masterservername then check that LIST_FS_IMAGE_HEADERS = "0", which it does. 

Thank you!!!

[Steve_Law1]

I mean, an image cleanup is Netbackup identifying obsolete images on the storage and marking them for deletion - right? E.g. if a backup image retention is 6 weeks and the image is more than 6 weeks then it is marked for deletion. So why does the upgrade require an image cleanup? And why does the upgrade doc call it an "image metadata migration"? 

[watsons]

The "bpimage -cleanup -allclients" is a command to check for expired image.

It is designed to run, because you won't want a image format conversion (which happens during this upgrade) to go through expired images, that is pointless and may end up taking much more time. So this command will take care of expired images, and the conversion process can skip them.

The "image metadata migration" is actually referring to the entire process, that would include this check and the subsequent image format conversion. 

[Marianne · Accepted solution]

" the upgrade doc call it an "image metadata migration"? "

I think you misunderstood the upgrade guide. Image cleanup is not the same as metadata migration. This process happens during the actual upgrade when you upgrade from pre-7.5 version where image headers files are migrated to EMM database.

Since you are already on version 7.5, there is no metadata conversion.

[Marianne]

Oh! Why are you upgrading to 7.6.x when all versions up to 7.6.x will reach EOSL in about 4 months?
Why not latest 7.7.x?

[SteveLaw]

So that's it - the upgrade of the test master is complete? 

[SteveLaw]

We still have 50-60 Windows 2003 servers here - our infrastructure is like an IT museum :) 

There's a project underway to get rid of or virtualise them by xmas, once that's done we'll upgrade again.  

[SteveLaw]

Hi Marianne, 
I will be very relieved if the catalog conversion is not an issue. But the NetBackup7.6.1_Upgrade_Guide.pdf does not say it only relates to pre-7.5 versions. Page 27: 

Catalog size (pre-NetBackup 7.6.x upgrades only)
The first factor to consider is the size of the NetBackup catalog. The catalog size
is a factor because a catalog migration may be a part of the NetBackup 7.6.1
upgrade.
Depending on the size of the catalog, the catalog migration can take a long time.
If you upgrade from NetBackup 7.5.x.x, the catalog size is calculated based only
on the number of Oracle, SQL Server, and DB2 backups. For pre-7.5 environments,
the total number of images present in the catalog determines the catalog size. For
smaller catalogs, the Simple migration plan is preferred. If you have a larger catalog,
more time for the migration is required, and the Guided migration plan is
recommended. More information about calculating the time that is required for
catalog migration is available.
See “How to determine your plan for the image metadata migration” on page 32.

We have lots and lots of SQL and Oracle images. Page 32 section "How to determine your plan for the image metadata migration" then instructs how to identify how many backup images there are on your system and has on p34 the graph guesstimating how long migration will take. 

It doesn't clearly say anywhere that image metadata migration doesn't apply to NBU 7.5 and later. I'll take your word for it that it doesn't apply, but I've been worrying about the upgrade windows and the unknown amount of time it will take for weeks!