Import many user PSTs into a single user archive

[Sortid]

Hi,

We are using EV10SP4.  We have a journal and user archives.  We are in the process of migrating PST files into the user archives.  We have about 10TB of archives which either don't belong to active users or are not in use by current users and have chosen to import these into a single user account set up for this purpose (there will be some over lap but with single instancing i'm not too concerned with this).  In this user archive, the import folder will include the users name, but where we don't know the user we will use 'unknown' or something similar.  So this archive will have thousands of folders.  Can EV handle this many or are there limits to user archive folders? 

Thanks.

[GertjanA]

EV should handle just fine.

I have archives having over 10.000 folders, no issues.

[AndrewB]

are you creating mailboxes for these or using shared archives?

[Sortid]

Thanks Gertjan.  Good to know.

Andrew, it will be a single mailbox to be used for DA searches, so not shared. Each user has their own archive, there are several shared archives, but this big one will be just for the DA team.

[AndrewB]

ok, either way, i wouldnt put that 10TB of data in a single archive or on a single EV server for that matter because you'll blow up the indexes and your vault store database. if you can split it up across several, you'll be in better shape for eDiscovery, maintenance, backups, index mgmt, etc.

[GertjanA]

Ah yes. I now see what you mean. (I sort of overlooked the 10TB :) )

Sortid. Keep in mind that if you full index, you will get a huge archive (anywhere between 5 and 8 TB), with an index of about 1.5TB.

That (indeed) will impact EV. Depending on your environment (do you have multiple servers, how's diskspace, how's SQL diskspace etc.) it is best to use multiple archives to spread the load. In addition, you might want to create 2 or maybe 3 seperate Vault stores (perhaps even in a seperate Vault Store Group) to spread the database load.

[Sortid]

Hi guys, good comments.  Our current environment is failry small, one journal server (running for 2 years) with maybe 2TB data, and one mail server with 1.5TB (running for about a year), in the same vault store group (sharing finger print database).  We recently imported 2TB data in to the mail server for users, and while the partitions didn't grow that much (most recent data already journaled) the vault store database went from 10GB to 30GB.  If we import this straight into the mail vault store, I expect that to hit 60GB or more.  Also, the index partition is 500GB.  If I need to accomodate 1.5TB more then that requires some thought.

If we added this to the same group, what kind of impact would this have on the finger print database?  Do they grow much?  If I was to create a new vault store but in the same group, can you reasonably guess the growth there?  It's currently 8GB.

Thanks guys.

[GertjanA]

I believe you can sort of do an estimation based on figures in the Performance guide for your specific EV version.

http://www.symantec.com/docs/DOC4553

[AndrewB]

fwiw, you say "didnt grow much" but essentially you trippled the size of your vault store db.

makes sense to add it to the same group so you can single instance across the entire data set. just remember, in case of a disaster, you have to restore the entire thing. if it's segmented out, you have the possibility of only having to restore the part up the the single instancing boundary.

as far as guestimating sizing for your data, i would use the figures from your current environment and extrapolate across the additional data set. you cant really know how much is going to SIS and compress without a deep analysis of the data itself but you can get close.

[Sortid]

I don't want to let this data impact the current servers either by bloating the indexes or the database of the mail vault store.  I would most likely need a new EV server with indexes and partitions, and a new vault store database.  I may need to also extend this to the fingerprint database to keep it separate, so if I have a vault store in a group but set to share within vault store only, it would still share the fingerprint database, right?  So would need a new group as well.  (We haven't decided on a solution, just getting all possibilities together before making a decision).

[AndrewB]

sounds like a pretty good summary of the considerations i brought up

[Sortid]

Yep, thanks Andrew, pretty much summing what you said.  Kinda leaning toward a separate vault store but might keep the same index so I don't have to have add another server.  I can always move the index folders for the user onto a new dedicated location and then close it.

I miss calculated the amount in the vault store, it had a heap of white space, so gre from 11GB to 24GB.  I didn't record how many items that represented, but now have 36mil in it.  It's interesting to note that the journal has 30mil and the journal VS is only 11GB.  Is that fairly typical?