Indexing additional email header info

[SHI-CRO]

I have a customer who has the need to search on information in email headers, specifically IP addresses and names of sending email servers.  I don't think EV indexes this by default, but I think there may be a way to make it happen.

I found a technote on defining custom properties, but it was for EV v6.0.  I don't know if it would still be applicable or if there is a newer technote.

Has anyone done this before or have any advice on how to do it?

[AndrewB]

i dont have the specific answer for you but i think it might be found in the custom properties feature: http://www.symantec.com/docs/HOWTO37039

if you can't find exactly what you need the article states, "If you have special filtering requirements for your archiving system, Symantec Corporation can supply the appropriate custom filters."

[SHI-CRO]

That's a good technote and it may be enough to get me through this.  I'm still apprehensive though.  I've done Selective Journaling filters a few times, but have never done Custom Filtering or Custom Properties.

Basically this customer gets asked for emails that came from a specific IP address or they need to do a search on the 'Return-Path' MAPI attribute and things like that.

I'm still not entirely clear on how to build the 'Custom Properties.xml'  file, for example, I'm looking at an email header that has 3 different 'Received' sections.  I don't know if I just need to use that once or put it in there multiple times.

[SHI-CRO]

I'm still trying to figure this out with no luck.  I'm hoping someone here might be able to tell me where I'm wrong.

The custom properties filter is pretty simple:

<?xml version="1.0" encoding="UTF-8"?>
<CUSTOMPROPERTYMETADATA xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="customproperties.xsd">

    <!-- 1. DEFINITION OF CONTENT CATEGORIES AVAILABLE -->
    <CONTENTCATEGORIES DEFAULT="HeaderData">
        <CONTENTCATEGORY NAME="HeaderData">
            <INDEXEDPROPERTIES RETRIEVE="Y">
                <PROPERTY TAG="Header"/>                
            </INDEXEDPROPERTIES>
        </CONTENTCATEGORY>        
    </CONTENTCATEGORIES>
    
    <!-- 2. DEFINITION OF CUSTOM PROPERTIES AVAILABLE -->
    <CUSTOMPROPERTIES>
        <NAMESPACE TYPE="MAPI">
            <PROPERTY NAME="0x007D" TAG="Header"/>
        </NAMESPACE>
    </CUSTOMPROPERTIES>
    
    <!-- 3. DEFINITION OF PRESENTATION PROPERTIES AVAILABLE -->
    <PRESENTATION>
        <APPLICATION NAME="search.asp" LOCALE="1033">
            <FIELDGROUPS>
                <FIELDGROUP LABEL="HeaderData">
                    <FIELD TAG="Header" LABEL="Header" CATEGORY="HeaderData"/>
                </FIELDGROUP>
            </FIELDGROUPS>
            <AVAILABLECATEGORIES>
                <AVAILABLECATEGORY CONTENTCATEGORY="HeaderData" LABEL="Header"/>
            </AVAILABLECATEGORIES>
        </APPLICATION>
    </PRESENTATION>
    
</CUSTOMPROPERTYMETADATA>

But the journal task fails after a short time.  This is in the dtrace:

2982    18:29:01.396     [5660]    (JournalTask)    <2816>    EV-M    {CustomPropertiesDefinition} Loading Custom Properties from file:\C:\Program Files (x86)\Enterprise Vault\Custom Filter Rules\Custom Properties.xml
2983    18:29:01.403     [5660]    (JournalTask)    <2816>    EV-H    {CustomPropertiesDefinition} Exception: '.', hexadecimal value 0x00, is an invalid character. Line 2, position 1. Info: Diag: Type:System.Xml.XmlException ST:   at System.Xml.XmlTextReaderImpl.Throw(Exception e)|   at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)|   at System.Xml.XmlTextReaderImpl.Throw(Int32 pos, String res, String[] args)|   at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Int32 pos, Char invChar)|   at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()|   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()|   at System.Xml.XmlTextReaderImpl.Read()|   at System.Xml.XmlTextReader.Read()|   at System.Xml.XmlValidatingReaderImpl.Read()|   at System.Xml.XmlValidatingReader.Read()|   at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)|   at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)|   at System.Xml.XmlDocument.Load(XmlReader reader)|   at KVS.EnterpriseVault.CustomProperties.CustomPropertyMetadata.CustomPropertiesDefinition.LoadCustomProperties(String ConfigFile) Inner:None
2984    18:29:01.404     [5660]    (JournalTask)    <2816>    EV:H    {CFilter::FinalConstruct} HRXEX fn trace : Error [0x80131940], [.\Filter.cpp, lines {119,131,135,142}, built Jul 10 17:51:28 2013].
2985    18:29:01.404     [5660]    (JournalTask)    <2816>    EV~E    Event ID: 45316 Could not start the custom filter. |Error: Error [0x80131940] |Internal References: |<0x80131940> |{CFilter::FinalConstruct} [.\Filter.cpp, lines {119,131,135,142}, built Jul 10 17:51:28 2013] |
2986    18:29:01.404     [5660]    (JournalTask)    <2816>    EV~E    Event ID: 3147 An error has occurred initializing the external filter 'EnterpriseVault.CustomFilter'. |Error: <0x80131940> |
2987    18:29:01.404     [5660]    (JournalTask)    <2816>    EV:H    {CEVFilterController::CreateFilterObject()} (Exit) Status: [<0x80131940>]
2988    18:29:01.405     [5660]    (JournalTask)    <2816>    EV:H    {CEVFilterController::InitializeFiltersFromRegistry()} (Exit) Status: [<0x80131940>]
2989    18:29:01.405     [5660]    (JournalTask)    <2816>    EV~E    Event ID: 3144 Failed whilst initializing the Filter Controller.  The agent will now shut down as it cannot reliably continue. |Error: <0x80131940> |
2990    18:29:01.405     [5660]    (JournalTask)    <2816>    EV:H    {CEVFilterController::Initialize()} (Exit) Status: [Failed whilst initializing the Filter Controller.  The agent will now shut down as it cannot reliably continue.     Error: %1      (0xc0040c48)]
2991    18:29:01.405     [5660]    (JournalTask)    <2816>    EV:L    {AgentMessageDispenser::ThreadDeInitialise} (Entry)
2992    18:29:01.405     [5660]    (JournalTask)    <6352>    EV:L    {CServiceSynchronization::Abandon} (Entry)
2993    18:29:01.405     [5660]    (JournalTask)    <2816>    EV:L    {CEVFilterController::DeInitialize()} (Entry)

I can't tell from this what EV doesn't like.  Anyone see what I'm doing wrong?

[Rob_Wilcox1]

Haven't tried it in a long time, but...

a. Check your MAPI prop is correct.

b. Have a read of my article on custom filters... Sorry can't link it from my ipad.

[SHI-CRO]

I've actually been using your article extensively.  I really appreciate you putting that together; without it I wouldn't be this far along.

I've done several searches for the MAPI header properties, and I think I have it correct:

http://msdn.microsoft.com/en-us/library/office/cc815628.aspx

The trace says it doesn't like the hex value 0x00, but I don't know where it's reading that from.

[Rob_Wilcox1]

Thanks :)

Well the only place that you have a hex value is:

 <PROPERTY NAME="0x007D" TAG="Header"/>

Without setting it back up again, I can't really offer any other opinion.

One thing to check though is that you should be able to access this same property using something MFCMAPI or MDBVU32 or even Outlook Spy.  It's worth checking.  Might also be worth implementing the whole of what is in the article, just in case there is something else amiss... then modifying things to work with the 'header' attribute.

[Nathan_Clark_2 · Accepted solution]

save your XML as ANSI

[SHI-CRO]

Saving it as an ANSI file did the trick.  Thanks so much Nathan!