Knowledge Base Article

Selective Journaling with Enterprise Vault

Introduction

Message journaling is a crucial aspect of Enterprise Vault. It builds on top of Microsoft Exchange’s journaling features and effectively means that all items to/from your organisation end up in a journal archive for retention, and discovery.

But…

There is a but!

In Microsoft Exchange it’s journal everything, or journal nothing. What do you do if you only want to journal messages between specific people, or only VIP’s? Well in Exchange 2003 you didn’t have much of a choice from Microsoft, in Exchange 2007 and Exchange 2010 you can use Transport Rules. However a simple and flexible approach from the Enterprise Vault side of things is called Selective Journaling.

Overview of Requirements

There are five simple steps to follow in order to configure Selective Journaling :

1. Setup journal archiving

2. Create a filtering rules file

3. Add the selective journaling registry keys for the journaling task

4. Restart the journaling task

5. Test

In more detail here is what we need to do :

1. Setup journal archiving

Setting up journal archiving is described in detail in the Enterprise Vault documentation, at a high level you will need to :-

a/ Configure an account/mailbox to be your journal “user”

b/ Configure the mailbox databases in Exchange so that journaling is enabled to your journal “user”.

c/ Create an Outlook profile on your Enterprise Vault server, so that you can open the journal “user” mailbox. Open it, and check it’s empty. Send a simple test message between two users, and check that a journal copy lands in the journal “user” mailbox.

d/ Create a new journal archive. You can do this in an existing, or new vault store.

e/ Check the journaling policy, and consider whether any changes are needed (I didn’t make any during this test)

f/ Create a journal task, don’t start it at the end of the wizard.

g/ Add a journal target, pointing it to the journal archive you created just now.

At this point you’re all set from a NORMAL journaling point of view. You’d just need to start the journaling task, and items would get hoovered up out of the journal mailbox in to the journal archive.

To facilitate further testing you may, at this point, want to give one of your test users (or Vault Service Account) permissions on the journal archive. This way we’ll be able to properly test things at the end.

2. Create a filtering rules file

This exists on the EV Server, and, since it’s one file, and later one set of registry keys this filtering will happen to all of the journal tasks on this EV server. The file needs to be :-

named SelectiveJournal_config.dat
Placed in the Enterprise Vault program folder
Be saved as a Unicode file

There are all sorts of parameters and options that you can put in the file, I’ll describe a few of them in the sections below.

3. Add the selective journaling registry keys for the journaling task

The key is as follows :

HKEY_LOCAL_MACHINE
\Software
\KVS
\Enterprise Vault
\External Filtering
\Journaling

Create a new STRING value with the name “1” (without the quotes) and set the value to be :

SelectiveJournal.SJFilter

4. Restart the journaling task

At this point you should check that the task doesn’t go in to a failed state in the VAC (wait a few minutes). You can also check the Enterprise Vault event log, for the following :

Event Type:    Information
Event Source:    Enterprise Vault
Event Category:    Journal Task
Event ID:    45329
Date:        5/9/2011
Time:        6:58:51 AM
User:        N/A
Computer:    EVAULT1
Description:
External Filter 'SelectiveJournal.SJFilter' initialising...

5. Test

Testing this configuration is best achieved by building up the tests from simple to more complex.

I set a very simple selective journaling rule which is :-

starts:alberto

This means that only mails to/from SMTP addresses alberto* will get touched by the filter. My test user is alberto@ev.local, and he’s sending and receiving mails to vaultadmin@ev.local.

So first of all, we do as above, and check that the task doesn’t go in to a failed state.

Next I’d suggest testing that the item NOT matching your selective journaling rule doesn’t get archived. What happens at this point is that the item should go “pending” in the journal mailbox, and then it should be moved (by default) to the deleted items folder in the journal mailbox.

You can override this, and hard delete the items, by having the following registry key in place :

HKEY_LOCAL_MACHINE
\Software
\KVS
\Enterprise Vault
\Agents
\SelectiveJournal

Adding a DWORD called HardDeleteItems and setting it to 1 (and then restarting the journal task if need be).

For now though, if I send an email from vaultadmin@ev.local to vaultadmin@ev.local, the mail will get to the journal mailbox, and when the journal task picks it up, it’ll be moved to the deleted items folder.

Last test is to check that the rule works. So, you can send a mail from alberto@ev.local, to vaultadmin@ev.local, or vice versa. The item shouldn’t end up in the deleted items folder of the journal mailbox. You should also be able to search the journal archive using browser search to locate the item.

Rules ?

There are quite a few options available when it comes to building rules. These are described in the “Setting Up Exchange Archiving Guide”, but to give you an idea :

Conclusion

Selective Journaling can be quite powerful, and be used to control what ends up in your journal archive. I can see several uses for this, such as only journal archiving specific people, or for journaling only mail into and out-of the organisation (not all the internal stuff)

Published 14 years ago

Version 1.0

Agents

Configuring

Enterprise Vault

Information Governance

Tip-How to

Was this article helpful?

14 Comments

SGF
Level 3
14 years ago
Hello Rob,

I would like to know the differences between the "Custom Filter Rules" and this "Selective Journaling". We are currently using Custom Filters for Journal Archiving where in we put xml files with filter rules in the "Custom Filter Rules" folder and a registry entry to point to the rules.

Now I'm confused with this Selective Journaling, which after reading through your article, does almost the same thing. Can you please list down the differences/advantages between these two approaches.
Rob_Wilcox1
Level 6
14 years ago
Good question.

I will put down my ideas in the next few days .. stay tuned :)
Rob_Wilcox1
Level 6
14 years ago

Selective Journaling, group journaling, and custom filtering are all related. They are ways to filter in/out data for the archiving/journaling tasks.

The best way to describe the differences is to reference the information in the "Setting up Exchange Server Archiving" PDF file :

Enterprise Vault provides the following filtering features:

*** Selective journaling.

This feature provides simple filtering of Exchange Server journaled messages. You set up a filter for the Exchange Journaling task that selects, by address, the messages to archive. Other messages are deleted.

*** Group journaling.

This feature enables the Exchange Journaling task to mark selected messages, in order to reduce the scope of subsequent searches. This can be particularly useful where there is a high volume of journaled email and you want to be able to identify messages sent between particular groups of users.

*** Custom filtering.

This feature provides sophisticated filtering. You create rules that select messages by matching one or more attributes, such as email addresses, subject text, message direction or the value of certain message properties.

The rules also include instructions on how selected messages are to be processed. This can include assigning a particular retention category, storing in a specified archive, deleting attachments of a specified type or size, or deleting or marking the message.

*** Custom properties. This feature is an extension of custom filtering. It enables you to configure Enterprise Vault to index additional properties on messages selected by the custom filters. These properties may be standard properties that a default Enterprise Vault system does not index or they may be properties added to messages by a proprietary, third party application.

Custom properties also introduces the concept of "content categories" for grouping the settings that are to be applied to messages that match a rule. These settings can include the retention category to assign, the archive to use and the additional properties to index.

Hope that helps.
ck-admin
Level 2
14 years ago
EV for Exchange Version: 8.0 SP4

Exchange Version: Standard 2003 SP2

Selective Journal Rules File (SlectiveJournal_config.dat) content: ends:yahoo.com

Test:

from user@gmail.com

to user@company.com

bcc user@yahoo.com

subject Test in BC

Expected Result: Store email item

Actual Result: Delete email item
Rob_Wilcox1
Level 6
14 years ago
Hmm that's interesting - I will take a look and see what I can find out.
Rob_Wilcox1
Level 6
14 years ago
Okay so I had a look at this, just to clear things up.

My rule in SelectiveJournal_config.dat is :

ends:gmail.com

I sent a mail BCC'ing a randomly made up GMAIL account, and in my journal mailbox I see :

P1

Sender: "mrg" <smtp:mrg@EV.Local>

Message-ID: <86CBFEC126FB16478F37D6B09490B28D9B3C@exch1.EV.Local>

Recipients:

"mrg" <smtp:mrg@EV.Local>,

"robw@gmail.com" <smtp:robw@gmail.com>

P2

From: mrg

Sent: 15 June 2011 04:31

To: mrg

Subject: test123hello3

The P2 is of course visible by opening the attachment in the P1, the P1 being the message in the journal mailbox.

When I start the journal task, the item DOES get archived.

Just to prove it is working, I sent another test mail, this time without cc'ing or bcc'ing any gmail.com address, and that gets correctly deleted by the selective journal filter :

So the questions then which come to mind are :-

* Which version of Exchange are you using?

* What does your journal mail look like in the journal mailbox

* Are you using Envelope Journaling? (If not, then bcc rules won't work)
ck-admin
Level 2
14 years ago
Your test case works because the sender is internal to the company; lets see if I can articulate my test case more clearly:

     Sender/Recipients:

          A = Internal to company (company.com)

          B = Outside party email (yahoo.com)

          C = Outside 2nd party email (gmail.com)

     Rule:

          ends:yahoo.com (Intent is to capture all correspondence between A and B)

      Test Case:

          C sends to A and bcc to B

First, answers to your questions:

* Which version of Exchange are you using?

     EV 8.0 SP4

* What does your journal mail look like in the journal mailbox:

     Not sure what you mean; journaling is working and currently everything is being archived

* Are you using Envelope Journaling? (If not, then bcc rules won't work)

     Envelope journaling is ebabled

     All other test cases pass except the test case I submitted in my earlier post; refer to the following

          A sends to B

          A sends to C and bcc to B

          A sends to C and cc to B

          C sends to A

          C sends to A and cc to B
Rob_Wilcox1
Level 6
14 years ago
Okay I will have another go with your scenario tomorrow.

You answered : What version of Exchange? With 8 SP 4.. whcih I assume is your EV version. What version of Exchange is it?

With regards to the journal .. what I'm asking is what does your journal message look like in the journal mailbox, BEFORE, the Journal Task picks it up, for processing.

What I did with my testing is to stop the journal task, send the test message, check the journal mailbox (opened it from a client machine, or you can open it from the Enterprise Vault Server), then, start the journal task and watch what happens to it.
ckadmin
14 years ago
Test case will never store email to the vault because the BCC header is stripped before sending email from party C to A with bcc to B

Sometimes you just need another person to help figure this out.

Thanks for responding
Rob_Wilcox1
Level 6
14 years ago
Yep I think you're right.

The proof would be in looking at the P1 in the journal mailbox.

from	user@gmail.com
to		user@company.com
bcc		user@yahoo.com
subject		Test in BC