volume pool numbers

[sshagent]

I inherited a system with a large number of volume pools, which i've been fighting against ever since to reduce the tape wastage.  Is there an actual impact on the NetBackup catalog/db for having a few thousand volume pools?  Commands involving media seem sluggish to me.  Perhaps im just putting blame on something that just doesn't seem right.

Seemingly they now want the ability to expand on the volume pools even more for selling tapes back to clients *sigh*  

Opinions most welcome?

Oh its probably worth mentioning im aware of tape wasteage and how it impacts on tape usage/sharing/allocation.  Its more if anyone has similar environments or knows if pools cause problems with big numbers.  Think we're currently at 1500, but this is likely to get a lot more

[Nicolai]

I think nobody in Symantec ever tested a installation with 1500 volume pools 

But  I could imaging NBRB (recourse broker)  would use quite some time to do evaluation for each mount request.

What memory use have you set for the emm database (small, medium,large). Changing from small to large will speed up the inner working of Netbackup. You change the EMM memory setting with the dbadm utility. 

[sshagent]

I almost fell off my chair when i first saw the amount of them.  Crazy!

Anyway I haven't messed with the memory settings, nor any settings under dbadm, but we're currently set to small.  Seemingly this needs a restart to set, i will increase it to large later in the week.  Thanks for the suggestion.  

[Mark_Solutions]

I am with both of you - I fell of my chair when I saw that figure and could not imagine anyone having that amount of volume pools!

If you need something in black and white here the only actual quote i could find:

From the tuning guide:

Do not create more pools than you need. In most cases, you need only 6 to 8
pools. The pools include a global scratch pool, catalog backup pool, and the
default pools that are created by the installation. The existence of too many
pools causes the library capacity to become fragmented across the pools.
Consequently, the library becomes filled with many tapes that are partially
full.

[sshagent]

Yeah, it was text i was mainly hoping for.  Although that text warns against it, i'd much rather have had something more "threatening" rather than just tape and robot fragmentation/capacity issues.  I have googled like crazy, and gone through the manuals to no avail.  

There is an expected growth coming in, with the opportunity for clients to keep their data longer...and even take the tapes home ( despite it being practically useless to them ).  So pools seems the way forward.  I guess some scripting for freezing tapes used by "project X" after its used them could be a possibility but that sounds daft still.  

[mph999]

When I first saw the amount I fell off my chair ...

I managed to crawl round and put the question to a backline engineer ...

We haven't seen such an installation (I haven't done any searches, this is just my colleague and I ) and the things that spring to mind are ....

Increased load on EMM as mentioned by NIcolai.

Management nightmare - how do you kep track of that amount

There probably is a limit in NBU but I guess you haven't hit it yet - I would need a call logged to justify asking someone to look in the code to see what it is.

Regards,

Martin

[Amaan]

No comments, amazing :) i cannot understand why somebody needs so many pools.

[sshagent]

Thanks for the comments guys.  

mph999 i haven't raised a case for years, probably was "Veritas" when i last did one.  I will have to look into how i do it.

I can kinda see the logic when they set the system up ( they didn't have a NetBackup knowledgeable person ), and there are various jobs/projects which are fairly well self contained.  It must have been decided they wanted tapes dedicated to each project and thats how they went.  Its probably worth mentioning this is more of an archive type use as opposed to regular backups, so whilst there is 1500+ pools only 3 to 10 might be used per week, and other than the regular backup pool most pools aren't written to again once their respective jobs/projects are complete.  Long retention periods + lots of pools, its a bit of a nightmare.

The "new suggested plan" is to make sure all client data is separated when it hits tapes so it can be targeted for expirations or even handing over to the client.  I have experienced this sort of request in the past and in previous places but always managed to fight it off.  

[mph999]

Hi,

I also can see the logic, and if it was a small envirnoment (low number of vol pools required) etc ... it would be fine.  Sometimes though the scale of the issue means that what would be a excellent idea on a small scale, becomes not such a good idea on a large scale.

Hopefully for this 'issue' you will not have any 'technical' problems (eg. hit a limit).

The problem is, there is no manual on 'this is what you do for backups, this is what you don't do" , and of course, every knows best and is an expert ...

Martin

[Marianne]

Mark_Solutions is unable to login today (due to SymAccount being offline) and has asked me to post the following on his behalf:

I have found a cached page on Google for a tech note that no longer exists! 

This reads as follows:

http://www.symantec.com/docs/TECH19876

Problem

Is there a limit to the number of volume pools that can be used by VERITAS NetBackup?

Solution

It should be noted that there is a "practical limit" to the number of volume pools to be used. A configuration with more than a few dozen volume pools is unusual and probably undesirable, as it can lead to very inefficient tape usage.

As for a hard-coded software limit, a test configuration was created with 5000 volume pools named "test1" through "test5000". Aside from performance degradation in some screens of the GUI (waiting for the list of pools to be constructed), there was no problem performing backups and restores using tapes in the "test5000" volume pool.

[sshagent]

Thanks Marianne / Mark