Where is bottleneck when EV is archiving?

[Gogeta]

Hi,

We are planning to do a major archiving of our Exchange mailboxes.  The estimate is about five thousand mailboxes, and maybe 6 to 9TB of data.  Obviously we would like to finish this archiving as quickly as possible.  So we are wondering how can we beef up over system in order to speed up the process.  Is it network bandwidth, CPU power, memory, or storage I/O?  Which one is the bottleneck that needs to be beefed up?  We have one site, four virtual EV servers, each has 4 CPUs and 4GB RAM.  We are going to increase to 8GB RAM.  Would EV be benefited by the increase of RAM?  Should the SQL server be beefed up too, since there will be a lot of indexing during the ingestion?  Any suggestion will be greatly appreciated!

Thank you.

[JesusWept3]

bottlenecks are generally a mix of all them
if you are doing PST Exports you will see storageonlineopns take the most CPU , same with orphaned shortcut deletion
if you are doing update shortcut location (and retention category) you will see SQL start blocking excessively in some circumstances
if you are rebuilding indexes or get behind on your indexing, you will see a mix of IndexServer and StorageOnlineOpns take the most cpu time
If you are doing OSIS/Vault Store Sharing, you will see more network bandwidth being taken up

If your exchange isn't optimized disk wise, you will see long disk queues on the exchange server and possibly get warnings on end users machines from outlook saying that it is attempting to retrieve data.

If your SQL server isn't optimized disk wise, you will see long disk queues, and you will get excessive blocks as others are waiting on an IO Latch (i.e waiting for the disk to become free)

but if everything appears to be ok then you can do the following to help get things moving

Up your MAPI threads to more than 5 if you have few tasks
put storage server archiving processes to 10
make sure that update shortcut locations is turned off
make sure that orphaned shortcut deletion is turned off

put your site schedule to 24x7 with a gap of 15 minutes every 4-6 hours so that its not waiting with nothing to do
ensure that AntiVirus isn't scanning your temp directories, ev vault stores and your indexing location, MSMQ locations etc
make sure you have Outlook 2003 and not 2007 on your ev servers, as 2007 has decreased performance overall

Turn off monitoring for EV so you don't run in to costly SQL stored procedures like archivingperformancebyhours

for shortcuts, try and keep the characters as small as possible, don't put something crazy like 2 million as it could possibly hang the archiving process
Add things like EXE to the registry so as not to convert it

Make sure all the best practice registry keys are in place
up the MSMQ storage to 8GB from 1GB as if it breaches that limit, you wont get any warning from msmq or EV, things will just stop
Set the Profile Expiry in the registry to be 1 day instead of 3 as to stop any mapi mutex time outs

try and keep searches and pst exports to a minimum

[GertjanA]

sql and storage.

Make sure that if the archiving takes a while, you reserve time to do a backup in between of all. Exchange, SQL, and EV-servers + storage+indexes+msmq

If you are archiving using an existing EV-install, follow JW's recommendations. If you are archiving using a new install, follow recommendations also.

processors and RAM should be fine as yo describe.

Could  you inform us later what you saw as result (just curious)

Thanks, and good luck.

[Gogeta]

Thank you for all the advices, they are very useful.  Just want to be sure about one thing.  So EV can take advantage of 8GB RAM, but is there any setting that we have to modify to make it aware of  the memory increase?  If our concern is just ingesting new emails into EV, should we also boost the power of the SQL server?  We are doing full indexing.  The SQL server has 4 CPUs and 16GB RAM.

Another question is, where is the bottleneck for deletion?  If we have a lot of users deleting and recalling a lot of emails from EV at the same time, is there any way to improve the system so that the users won't have to wait a long time for it to finish?

Thanks.

[JesusWept3]

EV can only take advantage of 4GB, it is after all a 32bit app, a machine with 16GB will give you 0% performance increase
deletion itself is pretty quick though

[MichelZ]

Not 100% true, I think.
EV has lots of processes, and it's 4 GB on a per-process basis, isn't it?

/Michel

[JesusWept3 · Accepted solution]

i can almost guarantee you that even with EV pelting it full blast it won't go over 3GB
seen too many customers waste horrendous amounts of money on 32GB machines to see none of it being used