cancel
Showing results for 
Search instead for 
Did you mean: 

Journal Message De-Duplication - auto-forward the mail from a mailbox on a round-robin basis

Reza
Level 3

 

Hi,
Just a quick question to see if anyone has implemented any work-arounds to the duplication caused by having multiple journal mailboxes in Exchange.
An idea that has been passed around (from Symantec consultant) is to point Exchange to a single mailbox and auto-forward the mail from this mailbox on a round-robin basis to multiple journal mailboxes in turn.  This does mean that the active journal mailbox will be receiving more mail than EV can handle, but when the forwarding moves on to the next mailbox EV finish the backlog before it's next turn.
I've over 1,000,000 messages per hour to journal & must use 30 active EV servers to ingest from 200 journal mailboxes.

We have a high usage of DA and CA. almost 500 CA reviewers. Duplicates are killing our reviewers. In some legal cases 90 % of items discovered in DA / CA are duplicates.  Meaning almost if you export 1000 items, it is actually 100 items that shows like 1000.

We can’t shrink number of journal mailboxes because then EV servers can’t keep of with the rate of items coming in.

The solution is to journal only to 1 mailbox (I like to call it proxy mailbox), do not archive directly from proxy mailbox but instead auto forward round robin to other mailboxes (ex. every 20 minutes auto-forward to a new journal mailbox) then have EV to archive from these journal mailboxes. So in this scenario I will forward only 20 minutes to each journal mailbox but I have hours to ingest items from that mailbox.

I tested this in the lab, it eliminated CA duplicates, DA duplicates, Storage duplicates, shrunk SQL Vault Store database size and shrunk indexes size.

The question is that I don’t have a automatic process to auto-forward messages to different mailboxes. I can write a script but this will takes months. Any suggestions? Have anybody used any tool to do that?

1 ACCEPTED SOLUTION

Accepted Solutions

Tremaine
Level 6
Employee Certified

Have you considered increasing the number of threads per journal archive target/task as AndrewB advised? If so has it helped? Obviously this will impact your system so you need to make sure you have the resources to cope with it and that you wont run into any Mapi session limitations.

From a reviewer perspective CA/DA can remove duplicate message for reviewers providing the messages meet the minimum criteria for de-duplication purposes, in DA the reviewer has three option to choose from in order to reduce the messages displayed.

If you are absolutely certain that it is the department tagging in the Journal connector that is creating the delay then it would need some more investigation to try and determine the root of the performance degradation. In order to do that it would be necessary to see a JournalTask dtrace and assess where the biggest delta's are between the various functions.

View solution in original post

7 REPLIES 7

idaman22
Level 4

Can you not just set up a scheduled task to open 4 different email profiles, each one with a rule to forward emails to a specific mbx, then schedule a task to close them after 20 minutes?

AndrewB
Moderator
Moderator
Partner    VIP    Accredited

first of all, what version of EV are you on? surely you dont have 200 exchange servers so are you certain you need 200 journal mailboxes?and just to clarify, are you thinking that you have 1 exchange server that can handle the entire journaling load for your exchange environment? that might be hard to pull off.

in EV 10 they redesigned the indexing so what you might want to look into is have dedicated journal archiving servers and seperate the indexing service to its own group of servers. you should be able to get a higher archiving ingestion rate by doing that.

also, if you think that EV is the bottleneck and not exchange, make sure that your journaling taks are set to use more then the default (5) number of connections (max is 25). if you have a constant backlog of messages you can also play with the number of items per pass to get the optimal settings for your environment.

AndrewB
Moderator
Moderator
Partner    VIP    Accredited

reza, do you need any further information here?

Reza
Level 3

Would you please elaborate? I am an EV guy but not much exchange.

Do you think how long will it take to synchronize.

My original thinking of forwarding apparently is not the best idea. The issue with forwarding is that I am not sure if the integrity of the messages will be retained. We do envelope journaling, that is why the original message is in an envelope with meta data (like BCC information) rapped around it. For example If I have a MSG file with BCC information in it if I emailed it to someone else then the BCC information will get lost.

 

Thanks

Reza
Level 3

Good points, we have a lot of volume, even now it is hard to catch up with journal mailboxes. The issue is that we use Compliance Accelerator Journal Connector that tags each and every messages with their CA departments. This is very very slow process. but thanks for the info

Reza
Level 3

well if you know a third party that done this before please let me know

 

Thanks

Tremaine
Level 6
Employee Certified

Have you considered increasing the number of threads per journal archive target/task as AndrewB advised? If so has it helped? Obviously this will impact your system so you need to make sure you have the resources to cope with it and that you wont run into any Mapi session limitations.

From a reviewer perspective CA/DA can remove duplicate message for reviewers providing the messages meet the minimum criteria for de-duplication purposes, in DA the reviewer has three option to choose from in order to reduce the messages displayed.

If you are absolutely certain that it is the department tagging in the Journal connector that is creating the delay then it would need some more investigation to try and determine the root of the performance degradation. In order to do that it would be necessary to see a JournalTask dtrace and assess where the biggest delta's are between the various functions.