Adding new EV server(s) to offload existing

[biosphere]

Hi, I have an EV server that is way overworked (not able to archive all enabled mailboxes in the available archiving window, basically). What are the steps for adding additional EV servers to offload an existing one and distribute the load? Cheers

[Nick_White · Accepted solution]

If the problem lies with the archiving agent then there isn't a lot you can do in terms of configuration as the Exchange agents are currently targetted as one per server, so without having user mailboxes spread across multiple Exchange servers you cannot load balance the archiving tasks meaning that a hardware upgrade would probably be your best bet. However, you can balance the storage and indexing load if this is where you are seeing a processing backlog

This is a little bit of thinking-out-loud, but the following is roughly what you would need to do.

Adding a new EV server is pretty straight forward; Install EV and in the configuration wizard point to the existing EV directory server and join the same site. Once that part is done things get a little more involved because you have already enabled the mailboxes. What you would need to do is define provisioning groups that split up the users into more manageable chunks, with each provisioing group using a different storage and indexing service located on the new EV server. The second storage service would have its own Vault Stores to manage and so you would then need to move existing archives over to the new Vault Store if you wanted them to be handled by the second EV server

[MichelZ]

Or you may want to rethink your archiving window, and may be use the whole weekend for archiving, or add some hours during the week. To determine the best strategy, we need a lot more information, like current archivng window(s), server specs, user/mbx count, exchange server counts, the exact limitations you see (e.g. cpu on 100% during the whole archiving window), msmq counts (are there specific queues building up?), and of course your EV version cheers

[biosphere]

Thanks for your input, MichelZ, much appreciated,

I have no leeway on archiving windows - it's already running at all available hours between backup windows and on weekends.

Anyhow, here's the nfo you sought (sorry for not providing this initially):

* Archving runs between 9 am and 6 pm, backup runs between 6 pm and 9 am
* Server is a quadcore Opteron @ 2.20 ghz, 12.0 GB ram, running 2003 R2 Enterprise SP2
* At the moment a little over 3000 enabled mailboxes, residing on 2 different Exchange 2007 mbx clusters. This is however epected to grow in the near future.
* EV version is 8.0 SP4
* CPU usage constantly spikes at 100% during scheduled task runs
* MSMQ counts are high in A5 and A7 (for one of the two Exchange servers in particular), with the occasional huge burst in A1. Hasn't been able to chew through A5 in one pass as of yet.

[biosphere]

Hi Nick, thanks for your answer :)

There are currently two mailbox archiving tasks, one pr Exchange server target. So I guess what you're saying is basically move all the archives "belonging" to one of the tasks to a new EV server and its storage/indexing service? If so, why the need to reprovision?

[MichelZ]

OK, so adding an additional EV Server and having each Server target one exchange cluster would be an option here. You would have to setup a new server, configure it, and then move all users from one exchange server to the other EV server using move archive.

Is SQL running on the same machine? Offloading this to a dedicated box might help, too.
What are you backing up during those 15 hours backup window?
Are you doing incremental? Are you backing up closed partitions, too?

As another suggestion, we got a tool called "EV NearSync" which replicates your vault stores to another location / datacentre, which could free up your server from frequent backup (and thus increasing your archiving window), which could proove more cost effective than having another server. Have a look at http://www.evtools.net/products/ev-nearsync if this is interesting for you.

Cheers

[Nick_White]

I mentioned reprovisioning because the provisioning group defines the Vault Store and Indexing service that will be used by the members of the group. It may not be necessary but I wanted to make you aware of it in case it is a required step

[biosphere]

Yep, eventually decided to go down this route - added a new EV server and moved the Archiving task for one of the Exchange clusters to the new server.

I reprovisioned all users belonging to that Exchange cluster to use the storage and indexing service on the new server.

However, I ran into loads of trouble when attempting to move the archives. I first tested the Move Archive Task on a few archives, but this process was horrendously slow, so I decided to have a go at the PST Export/Import route as described in http://seer.entsupport.symantec.com/docs/273232.htm. This proved to be somewhat faster, but the process of disabling/re-enabling the mailboxes produces some strange results. Essentially, when a mailbox is disabled/re-enabled according to the procedure, the user ends up with duplicate archives (see screenshot - names and stuff have been blanked to protect the innocent, but it's the same user).



Furthermore, a query along the lines of http://seer.entsupport.symantec.com/docs/278548.htm reveals that the user has several entries:



Importing the PSTs produces conflicting results - in most cases the import completes, but at times nothing is imported and in all cases the actual is left in the export folder.

So, I'm really confused and frustrated - at the moment I have loads of PSTs sitting in a folder, not knowing where they'll end up, if they're getting imported, if duplicates are being created - you name it.

Any pointers on how to proceed with this and getting everything into a proper state would be hugely appreciated.

(Just to answer your other questions: SQL is on dedicated server, backups are running on open partitions and indexes in incremental mode.)