EV 10 index drive capacity planning for upgrade from v9

[John_Santana]

Hi All,

Can anyone here please share the method to predict or approximate the disk space sizing for the EV v10 64 bit index ?

as at the moment, in my EV 9 environment, I got let say 1 TB of index and I'd like to upgrade to EV 10 with 64 bit indexing capability, how much do I need to allocate ?

[GertjanA]

Hello John,

I've been through the same excercise. What is more important is how much free space you have on your VaultStore disks. Upgrading to 64-bit indexes means a re-index of archived items. This means indexing has to go throught each and every item, meaning an extraction of cab-files (if you use these), and a load on the server.

How I approached it:

When upgraded to EV10, rightclick your EV-servers in the console, properties, tab advanced, set value "Maximum concurrent indexing capacity" to 10. (Default = 30). If you don't, and start upgrading indexes, you choke the server (at least, I did, and my servers are 16 to 32 core, with 32GB ram at least).

Upgrade indexes controlled. I initially started with 'indexes starting with A...', but then decided to start with the smallest. In the Upgrade Wizard, you can sort on several columns. Sorting on items will give you that choice. YOu can also do per Vault Store etc. YOu need to play with this to figure out your best approach.

I did 1200 archives with less than 1000 items in 90 minutes. The longest upgrade was a journal archive having litlle more than 55.000.000 items in it, that took 12 days. (of which about 10 hours were used to run, rest was backupmode etc). I choose to do user-archives first, then the Journal ones.

What happens during the upgrade?

If a 32 index is selected for upgrading, it is closed. A new 64 bit index is opened. The 64 index is being created. As soon as it is finished, EV checks to see if it is ok. If ok, the 32-bit index is being deleted.

You will see that after your upgrade, all archives will start getting 2 index-locations. EV starts using the 64 bit indexing for newly archived items. Users will not notice...

I found that the index disks grew about a mere 3% (we already had full indexing), which was easily possible. Also remember that you can close a 64 index location, and that that is really closed. (no new data added). I had little to no failures, from the approximately 22000 archives I upgraded, only 3 really failed (due to corrupted store location, or missing files). As these were archives for 'leavers', I left them as is.

Hope this helps.

[John_Santana]

Yes, I have recently enabled the Collection on 11 closed partitions with each 1 TB on my EV9. Does my index partitions or the Database grows even bigger as well ?

[John_Santana]

Do I need to provision the same disk letter and disk size (same 1 TB or do I need more ?) on the new EV 10 server for the index LUN drive ?

[AndrewB]

you can create new index locations or you can use the existing location. space management works the way gertjan described but yes, generally speaking, the 64bit indexes use 4% more space than the old 32bit version.

[AndrewB]

no, i haven't seen database growth as a result of rebuilding indexes. 

if you reuse the same index location then yes, you need more space for the two versions to coexist while they're being rebuilt. gertjan outlined the process based on his experience quite well.

[John_Santana]

Andrew and GertJan,

Is there any scenario where the Index size can be decreased ? or is it just going to get bigger overtime instead of decreasing ?

[TonySterling · Accepted solution]

It is unlikely index size will ever decrease.  The only way I see that happening is you stop archiving new items and start running Storage Expiry.
 

[John_Santana]

ok, but if I do run storage Expiry would that reduce the size of the EV index partition once the purge finished ?

or do I have to do some SQL command to resize the EV index size ?

[GertjanA · Accepted solution]

If you run Storage Expiry (and have never run that before), it depends on your amount that is going to be expired.

In theory, if you archived for 5 years, and expire items now older than 3 years, you *should* get space back on the index location, BUT, as you keep archiving, data also will be added. Eventually, provided you archive the same amount of data, your indexes should be in a more or less consistent state. In EV10, you can close index locations (no new data being added to indexes anymore), but you need to provide a new location...

[AndrewB]

just to be clear, if you expire data then the indexes get removed for that data by the system. there's no additional steps needed in sql or anything like that. 

"would that reduce the size of the EV index partition" - not sure what you mean exactly. do you mean free space? do you mean shrink the partition (disk) itself?

[John_Santana]

Thanks Andrew.

No what I mean is the index files usage itself not the drive partition for the index :)

[AndrewB]

that's what it sounded like... just wanted to make sure. do you have everything you need for this one?