Poor? archiving performance, EV9

OK so first of all

1. Run a report mode against the users, see how much is genuinely there to archive in the report.
I know this sounds redundant, but people look at the age of items or the size of the mailbox and think that EV Should be archiving more, but there can be subtle things in the policy such as don't archive unread items, age limits, and stuff like that.

Also if you don't lock down your policies, users could be setting folders to do not archive, or they could be suspending their mailbox altogether.

Another thing is EVPM if you decided to customise things such as don't archive sent items, or changing different policies there.

2. Check the Event Logs
Although sounding obvious, its a critical thing to check, if you experience errors, especially against something like Exchange 2010, you maybe seeing MAPI errors where the task goes to sleep for 10 minutes, and if you get 6 of those for the same task in a row, well, you have just lost an hours worth of archiving. On the EV Server look for 3310, 3305 and 3301 errors.

Also check the Event Logs on the exchange server too, if you are running Exchange 2010, there are different mechanisms such as the Throttling Policy which dictates how much and for how long Enterprise Vault will spend in a CAS server, in AD, how many connections etc.  If EV breaches that policy then the Exchange CAS Servers will forcefully close the connection.

3. Make sure you are running the correct version of Outlook
If you are running Exchange 2003 and 2007 *only* then you only need run Outlook 2003 as it is somewhat more performant than Outlook 2007.    If you are running Exchange 2010 then you need Exchange 2007 with the performance hotfix released just over a year ago.

If you do not have the performance hotfix you will suffer performance issues and disconnects purely from a MAPI standpoint, make sure you do not have Outlook 2010 installed on the server as you will see a big degredation in performance as well as it just not being supported on the server.


4. Don't overload your servers
Typically in a large environment, you should not want more than 5 archiving tasks on a server at once.
MAPI itself typically starts to see MAPI Mutex errors and memory leaks when it goes over 32 connections, by default a single Archiving task will use 5 simultaneous MAPI connections, so if you have lots of tasks, you may breach 32 connections and cause slow downs and errors there.


5. Know what your environment is doing
Make sure you know how your environment is properly set up, especially with things such as provisioning and synchronization sechedules, if you have a lot of users and a lot of tasks, and the schedules all run in to each other, you can cause a lot of delays and in large enterprise environments a provisioning task can eat in to the schedule of the archiving task.

Also look at other things that you may be running such as Moved Shortcut Updates, Retention Category Updates, Delete Empty Folders or anything else that may be known to cause slow downs.

For instance with Shortcut updates, they will run before anything is archived from a mailbox, and depending on how many shortcuts there , it can take a long time


6. Examine the MSMQ's
In the Computer Management, check your MSMQ's to see how much is in there, especially on the A5 queue, because when an a task starts up and starts archiving an exchange server, all eligible users will be posted in to the A5 queue.

If there are items still in there from overnight then it would seem to suggest it may not have been able to process all the users in a single night, but before jumping to that conclusion check the date and time of the oldest item, if you see items that were posted at the very beginning of the archiving schedule then this indicates a problem.

However mailboxes will typically be posted multiple times to an MSMQ because of the Items Archived Per Pass setting in the Exchange Mailbox task.

So by default EV will archive 1000 items per pass, meaning it will connect to the mailbox, scan each message to see if its eligible for archiving, sort by size and then date of the email, and then archive 1000 messages. If it knows there are more than 1000 messages to archive it will simply Re-Queue the item.

So if you have a schedule between 12:00AM and 3AM, and you have a user in the MSMQ posted at 2:59AM, then thats typically OK, it does mean there was more to archive, it just means that it was posted 1 minute before the schedule stopped.


As well as the above there are other things you should probably be checking
 - Disk Queue Lengths on the Exchange Server, just like anything else, if the disk is slow, then accessing items will be slow
 
 - Network connectivity between the EV server and the Exchange Server, sometimes there can simply be a bad routing, a bad cable, a bad NIC Binding (such as the Backup NIC's being the primary connection) that can make things slower than need be

 - Check the SQL Server processes looking for any blocking or locking that maybe causing queries to run slower, and the disk queues on there as well.