Duplicate shortcuts and missing archived item attachments

[AWChris]

Hi All.

To give some background I'm running Exchange Server Enterprise 2007 SP2 UR4 and EV 8.0 SP2 with the typical user client is MS outlook 2003 SP2.

I've come up against an issue with a few EV clients with Exchange mailbox Archives, they have items in their vault that are missing their attachments as well as duplications of items in the vault both in plain text email and HTML email form. There may be a few different issues going on here so please bear with me. The vast majority of user mail is archived and displayed correctly, an example user has almost six years of email in the system however the problems seem to start at an arbitrary date and then continue from there.

The most problematic user's issues start at 10/12/2007 in DD/MM/YYYY before which, and right back to the begining of their mail archive in 2005, almost all of their shortcuts are duplicated both in plain text email and HTML email form. The plain text copies of the archived items have no attachments but have a text representation for the attachment while the HTML archived items have their attachments. There are also small instances of individual emails that are only in plain text and have no attachments and just the text.

I've run a EVSVR verify complete on the users mailbox as well as an indexcheck missingdocs and missingcontent on the users EV index and I get some errors as a result:

2010-08-23 17:09:47  Log file 'D:\Logs\EVSVR\MB_23082010.log' created/opened
2010-08-23 17:09:47  
2010-08-23 17:09:47  EVSVR Version (8.0.4.1991)
2010-08-23 17:09:47  
2010-08-23 17:09:47  
2010-08-23 17:09:47  Operation: Verify
2010-08-23 17:09:47  Option:    Verify Complete - Level Not Applicable
2010-08-23 17:09:47  Data Silo:
2010-08-23 17:09:47    Site:              VLine Vault Site
2010-08-23 17:09:47                        1D3289A142EDB5246B00597D36EAC350F1d10000EVSite
2010-08-23 17:09:47    Vault Store Group: Default Upgrade Group
2010-08-23 17:09:47                        1916B1D0E1623FB47B1D49C6FCFD2A5171013300EVSite
2010-08-23 17:09:47    Vault Store:       MAILBOX VAULT STORE 1
2010-08-23 17:09:47                        1D3FF21614FC9C04682F82A1422BFAD941210000EVSite
2010-08-23 17:09:47    Partition:         MAILBOX VAULT STORE 1 Ptn1
2010-08-23 17:09:47                        17A3B7AE0CB3AF4468B66396144060A5D1q10000EVSite
2010-08-23 17:09:47  Date range: Start: <None>, End: <None>
2010-08-23 17:09:47  Archive: Boccabella, Margaret
2010-08-23 17:09:47            184983E836D07094D95B135C3714280831110000EVSite
2010-08-23 17:09:47  Threads: 2, Priority: Normal
2010-08-23 17:09:47  Log file: D:\Logs\EVSVR\MB_23082010.log
2010-08-23 17:09:47  
2010-08-23 17:09:47  Performing initial database and partition checks
2010-08-23 17:09:47  
2010-08-23 17:09:47  
2010-08-23 17:09:47  All checks were completed successfully
2010-08-23 17:09:47  
2010-08-23 17:09:47  
2010-08-23 17:09:47  Operation started
2010-08-23 17:09:47  
2010-08-23 17:09:47  Site: VLine Vault Site
2010-08-23 17:09:47  
2010-08-23 17:09:47    Vault Store Group: Default Upgrade Group
2010-08-23 17:09:47  
2010-08-23 17:09:47      Vault Store: MAILBOX VAULT STORE 1
2010-08-23 17:09:47  
2010-08-23 17:09:47        Partition: MAILBOX VAULT STORE 1 Ptn1, DeviceType: CIFS, Collection Enabled
2010-08-23 17:09:47  
2010-08-23 17:09:47          Verify that Saveset SISPart entries exist in the Fingerprint Database
2010-08-23 17:09:47          ---------------------------------------------------------------------
2010-08-23 17:09:47  
2010-08-23 17:09:47            Saveset records: 33848
2010-08-23 17:12:29  
2010-08-23 17:12:29            Savesets processed:              33848
2010-08-23 17:12:29            Savesets with missing SIS Parts: 0
2010-08-23 17:12:29  
2010-08-23 17:12:29            Savesets without SIS Parts:      33848 (100.0%)
2010-08-23 17:12:29            Savesets with SIS Parts:         0 (0.0%)
2010-08-23 17:12:29  
2010-08-23 17:12:29          Verify that Fingerprint Database and Vault Store SIS Part reference counts match
2010-08-23 17:12:29          --------------------------------------------------------------------------------
2010-08-23 17:12:29  
2010-08-23 17:12:29            SISPart records: 0
2010-08-23 17:12:29  
2010-08-23 17:12:29            SISPart records:
2010-08-23 17:12:29              Processed:            0
2010-08-23 17:12:29              Unreferenced:         0 (0.0%)
2010-08-23 17:12:29              Unshared:             0 (0.0%)
2010-08-23 17:12:29              Shared:               0 (0.0%)
2010-08-23 17:12:29  
2010-08-23 17:12:29              Mismatched RefCounts: 0
2010-08-23 17:12:29  
2010-08-23 17:12:29          Verify that Collection reference counts are correct
2010-08-23 17:12:29          ---------------------------------------------------
2010-08-23 17:12:29  
2010-08-23 17:12:29            Collection records: 53475
2010-08-23 17:12:29  
2010-08-23 17:14:05  
2010-08-23 17:14:05            Collections processed:                 53475
2010-08-23 17:14:05            Collections with mismatched RefCounts: 0
2010-08-23 17:14:05            Collections with invalid record:       0
2010-08-23 17:14:05  
2010-08-23 17:14:05        Saveset records: 33848
2010-08-23 17:14:05  
2010-08-23 17:34:04        SavesetId: 900000000000000~200804012316530000~1~02E5A97ACB98477DAF01B7A9FA088F5, Archived Date: 2008-05-02 22:03:19Z
2010-08-23 17:34:04          ArchiveEntryId: 1698878B45C1C0E40B9AB22A18A2CEE731110000EVSite
2010-08-23 17:34:04          File not found [0x80070002]
2010-08-23 17:34:04  
2010-08-23 17:43:26        SavesetId: 201003270000000~201002250757000000~Z~50C1EEFD37ACEBEBF0B56F7F5D39B3D1, Archived Date: 2010-03-27 08:26:00Z
2010-08-23 17:43:26          ArchiveEntryId: 19B08768299F63C41B13C584C2355CEC51110000EVSite
2010-08-23 17:43:26          Collection Identity: 50521, DVS RelativeFileName: 2010\03-27\5\Collection50521.CAB
2010-08-23 17:43:26          Error: Failed to get custom properties.      [0xc004710f]
2010-08-23 17:43:26  
2010-08-23 17:43:26        SavesetId: 201003270000000~201002250757480000~Z~50BFCC1F433F56638AC70081B4E69271, Archived Date: 2010-03-27 08:26:00Z
2010-08-23 17:43:26          ArchiveEntryId: 1698878B45C1C0E40B9AB22A18A2CEE731110000EVSite
2010-08-23 17:43:26          Collection Identity: 50521, DVS RelativeFileName: 2010\03-27\5\Collection50521.CAB
2010-08-23 17:43:26          Error: Failed to get custom properties.      [0xc004710f]
2010-08-23 17:43:26  
2010-08-23 17:45:17  
2010-08-23 17:45:17        Verify failed count: 3
2010-08-23 17:45:17        Verify succeeded count: 33845
2010-08-23 17:45:17  
2010-08-23 17:45:17        Items Processed in Partition 'MAILBOX VAULT STORE 1 Ptn1': 121171
2010-08-23 17:45:17  
2010-08-23 17:45:17      Items Processed in Vault Store 'MAILBOX VAULT STORE 1': 121171
2010-08-23 17:45:17  
2010-08-23 17:45:17    Items Processed in Vault Store Group 'Default Upgrade Group': 121171
2010-08-23 17:45:17  
2010-08-23 17:45:17  Items Processed in Site 'VLine Vault Site': 121171
2010-08-23 17:45:17  
2010-08-23 17:45:17  Vault Store Groups: 1
2010-08-23 17:45:17  Vault Stores:       1
2010-08-23 17:45:17  Partitions:         1
2010-08-23 17:45:17  
2010-08-23 17:45:17  Processing rate: 204796 Items/hour
2010-08-23 17:45:17  Elapsed time:    0 days, 0 hours, 35 minutes, 30 seconds
2010-08-23 17:45:17  
2010-08-23 17:45:17  Operation completed
2010-08-23 17:45:17  
2010-08-23 17:45:17  Log file closed
2010-08-23 17:45:17  

Missing docs:
184983E836D07094D95B135C3714280831110000EVSite/Volume:53
18098,900000000000000~200804012316530000~1~02E5A97ACB98477DAF01B7A9FA088F5,-1

Missing content:
184983E836D07094D95B135C3714280831110000EVSite/Volume:53
8019,122700000000000~200603092148560000~0~74834FF7F9A54B7BAE5092BCD37FC61,12
12809,176200000000000~200608272338410000~1~A6216AFEAF9A42BF8D139D2F7AE202F,9
15334,216000000000000~200612220429520000~1~B9A62CE6D7B943328C9D403CB1078CF,12
15373,710000000000000~200708040251570000~1~1409FC057B22406BB25FAF90FC80E93,12
18895,926000000000000~200805152121120000~0~25D0675E25DA488C97F8F3924EFCD7E,9
21372,649000000000000~200810150307580000~0~8D41B40A583247B09E22A7776611F0C,14
23520,925000000000000~200902240451120000~0~5E8E71612A6D4BC9B9446E6C2A80DAF,14;14;14
33439,200909060000000~200907140440440000~Z~B1021E8339F2F5B11F3A415CE2596151,9
34487,200910070000000~200909070247350000~Z~70C27584A6F4D814C21DCF6C8D6EF121,9
35374,200911130000000~200910130749000000~Z~70F800FEB52DC8E5F66FBD0A69ABD791,5
37445,201002220000000~201001120320040000~Z~9160A2A9D802FE1EC664A00A878AD851,9
49667,201003240000000~201002210846460000~Z~50AC97B13692BD0EF8019142B6BEAD91,14
49887,201004020000000~201003022311250000~Z~50B1ED8994A019F2197FAC5D459FA961,14
51715,201006186854024~201005170005240000~Z~60985501735F6F47CAEC54B66F878991,9
53035,201007300481087~201006300044490000~Z~602AA5B11DAF25B2D40256D4514E3671,9
53058,201007300481108~201006300750000000~Z~602BCE797379D752CFEA5BA0772AD461,9

But I don't have an auditing database up and running, not my installation and I have no auditing on the CIFS drive itself. I don intend to install and configure both in the near future. I'm continuing to track down the collection files and checking to see if the DVS items are located in them but this will take quite some time.

What I'm wondering is

1) If there is anything else I can run that will give me some indication of where these items went to or what happened to them.

and

2) Is there any whay I could get duplicate shortcuts in a mailbox other than by using the Arhive item export tool and reimporting the items into the mailbox?

Any thoughts would be great. :)

     Chris.

[AWChris]

Oh something I just saw.

All the successfully archived items in the users mailbox with the attachments still referenced are IPM.Note class while the ones not referencing their attachments are IPM.Note.EAS.

No idea what this means if anything.

[Nick_White]

IPM.Note.EAS is the message class for an item that was archived by another archiving product. The attachments were most likely stripped by the old system and were not in the mailbox when we archived the items. You would need to restore the items from the old archiving product to have them fully archived by EV

[AWChris]

Hi Nick, thanks for the reply.

I've come to the same conclusion and I'm going to ask some questions along these lines of the people who were looking after this infrastructure at that time. Any ideas as to the product that would be using this message class? I have a few suspects with the primary one being HPs Email Archiving Software but I don't know for sure as I have not seen it.

     Chris.

[Nick_White]

EAS is most likely Autonomy's solution

[JesusWept3]

you should ost this as an article me thinks!

[AWChris]

Thanks for your time nick, I've found solutions for both issues one business and the other technical:

The IPM.Note.EAS situation was indeed the result of an export from a previous archival system. The mail import is no longer availible and the business is happy with that and doesn't want to go down the road of retrieving those items; yay me. :)


I have no idea how the duplicate items in the mailbox turned up, none whatsoever so I wrote a script to delete all the duplicate IPM.Note.EAS EV shortcuts if they had a matching IPM.note EV shortcut of the same subject and received date. The script is below:

- Begin script

' This script parses a users mailbox looking for IPM.Note.EAS EV shortcuts
' checks to see if they have a duplicate IPM.Note version then deletes the
' PM.Note.EAS email EV shortcut
'
'  Warning: Use at own risk, seriously, don't use this unless you know exactly what you're doing.
'
' Important note. On Exchange 2007 servers CDO is 32bit only and must be run from
'  C:\WINDOWS\SysWOW64\cscript.exe otherwise the MAPI object will fail
'
' Useful references for MAPI:
' olItemType reference - http://msdn.microsoft.com/en-us/library/bb208104%28office.12%29.aspx
' logon session - ' http://msdn.microsoft.com/en-us/library/ms526377%28EXCHG.10%29.aspx
' olDefaultFolders reference - http://msdn.microsoft.com/en-us/library/bb208072%28v=office.12%29.aspx
' Core CDO reference - http://msdn.microsoft.com/en-us/library/ms526914%28v=EXCHG.10%29.aspx
'
'  Created on 30/08/2010

Option Explicit

Dim strServer, strMailbox, strProfileInfo, objCdoSession
Dim objInfoStore, objFolderListEAS, objMessageEAS, objFieldEAS
Dim FileSystem, Logfile, Errorcode, objFolder, objFolderListIPM
Dim objMessageIPM, objFieldIPM, objMessageFiltEAS, objMessageFiltIPM
Dim colSize, i

'MAPI Property Tags
Const PR_MESSAGE_CLASS = &H001A001E
Const PR_ORIGINAL_MESSAGE_CLASS = &H8455001E
Const PR_CONVERSATION_TOPIC = &H0070001E
Const PR_MESSAGE_DELIVERY_TIME = &H0E060040
Const olMailItem = 0

'MAPI Properties to search for
Const cstrEVShortcut = "IPM.Note.EnterpriseVault.Shortcut"
Const cstrEASMessage = "IPM.Note.EAS"
Const cstrIPMMessage = "IPM.Note"

' Create CDO Session variables
Set FileSystem = WScript.CreateObject("Scripting.FileSystemObject")
Set Logfile = FileSystem.createtextfile("Delete_EAS.log",True)
Set objCdoSession = CreateObject("MAPI.Session")

'Logon session details
strServer="mbxservername.yournetwork.local"
strMailbox="Mail Alias or username"
strProfileInfo = strServer & vbLf & strMailbox


'Logon to Exchange Server - objSession.Logon( [profileName] [, profilePassword] [, showDialog] [, newSession] [, parentWindow] [, NoMail] [, ProfileInfo] )
objCdoSession.Logon , , False, False, , True, strProfileInfo

'Get the second infromation store for the session which is the users mailbox
Set objInfoStore = objCdoSession.InfoStores.Item(1)

'Get the IPM_Subtree folder for the information store
Set objFolder = objInfoStore.RootFolder

'initialise the logfile
logfile.writeline "Starting parse of mailbox " & strMailbox & " at: " & now

'Start parsing of the mailbox
ProcessFolder(objFolder)

'Close the MAPI session
objCdoSession.Logoff

'Clean Up
logfile.writeline "Finished at " & now
Set Logfile = nothing
Set objCdoSession = nothing
WScript.Echo "Finished"
wscript.quit(0)


'Parses the root of the mailbox looking for folders with mail items as their default
Sub ProcessFolder(RootFolder)

    Dim objSubFolder
    
    'set debugging
    On Error Resume Next
    
    'Check every subfolder of the root folder
    For Each objSubFolder in RootFolder.Folders
    
        'If the default items of that folder are mail items then
        If objSubFolder.DefaultItemType = olMailItem Then
           
            'Scan for EAS mail items and delete them
            DeleteMails(objSubFolder)
        End If
       
        'Recurse subfolders of the mailbox
        ProcessFolder(objSubFolder)    
    Next

End Sub


Sub DeleteMails(ScanFolder)

    'Set debugging
    On Error Resume Next

    If Not ScanFolder Is Nothing Then
     
        'Create a folder colleciton from the scan folder for EAS EV shortcuts
        set objFolderListEAS = ScanFolder.Messages
       
        'set the folderlist filter for EV shortcuts of EAS messages
        Set objMessageFiltEAS = objFolderListEAS.Filter
        objMessageFiltEAS.Fields(PR_MESSAGE_CLASS) = cstrEVShortcut
        objMessageFiltEAS.Fields(PR_ORIGINAL_MESSAGE_CLASS) = cstrEASMessage
       
        'Create a folder colleciton from the scan folder for IPM EV shortcuts
        set objFolderListIPM = ScanFolder.Messages
       
        'set the folderlist filter for EV shortcuts of IPM messages
        Set objMessageFiltIPM = objFolderListIPM.Filter
        objMessageFiltIPM.Fields(PR_MESSAGE_CLASS) = cstrEVShortcut
        objMessageFiltIPM.Fields(PR_ORIGINAL_MESSAGE_CLASS) = cstrIPMMessage
       
        'itterate backwards through the collection due to deletion of objects see:
        ' http://msdn.microsoft.com/en-us/library/ms527203%28v=EXCHG.10%29.aspx
        colSize = objFolderListEAS.count
     
        'For each message in the EAS EV shortcut list
        For i = 1 to colSize
       
            Set objMessageEAS = objFolderListEAS.item(colSize - i + 1)
       
            Set objFieldEAS = objMessageEAS.Fields    
                   
            'search through the IPM EV shortcut list
            For Each objMessageIPM In objFolderListIPM
                           
                Set objFieldIPM = objMessageIPM.Fields
                           
                'Check to see if the subject matches
                If objFieldEAS.Item(PR_CONVERSATION_TOPIC).value = _
                objFieldIPM.Item(PR_CONVERSATION_TOPIC).value Then
                                   
                    'Check to see if the submitted time is the same
                    If objFieldEAS.Item(PR_MESSAGE_DELIVERY_TIME).value = _
                    objFieldIPM.Item(PR_MESSAGE_DELIVERY_TIME).value Then
                                       
                        'Log the deletion
                        logfile.writeline "Deleting message '" & objFieldEAS.Item(PR_CONVERSATION_TOPIC).value & _
                        "' from the " & ScanFolder.Name & " folder, received at " & _
                        objFieldEAS.Item(PR_MESSAGE_DELIVERY_TIME).value
                   
                        'Delete the EAS item; moving to the deleted items folder
                        objMessageEAS.delete(TRUE)
                                   
                        Exit For
                                   
                    End If
                End If
            Next
        Next
    End If
     
End Sub

- End Script

[AWChris]

What is this article thing you speak of?