11-09-2017 12:45 PM
Hey Guys,
I am trying to figure out why EV sometimes pulls messags out of hte journal mailbox correctly and other times just sits there and does nothing.
So far, Mailbox Sync's without issue, services, tasks started. The number of items does update when the sync process is run.
I enabled DTRACE and am seeing 100's of these errors here:
http://entced.symantec.com/entt?product=ev&language=english&version=11.0.1.0&build=11.0.1.3706&error...
http://entced.symantec.com/entt?product=ev&language=english&version=11.0.1.0&build=11.0.1.3706&error...
http://entced.symantec.com/entt?product=ev&language=english&version=11.0.1.0&build=11.0.1.3706&error...
I looked but wasnt able to find much on those. Any assitance would be most appreciated.
Thanks,
Robert
11-09-2017 10:55 PM
Hi Robert,
You did not really past eventid's I'm afraid, but links to the Symantec KB's. Old stuff :)
Which EV version?
I (probably) have the same. Most of the time, it is a corrupted item in Journal Mailbox sitting in ' pending archiving' for a loooooong time. I found the only way to workround/fix is to go through the Journal Mailbox(es), and to move items that are old (I tend to look at older than 14 days) to a NEW subfolder underneath the Enterprise Vault Journaling Service folder. When placed in a new folder, the item will not be returned to the inbox if you restart the task, but will remain searchable with a Discovery search in Exchange.
If the Journal item count goes up (use powersheel to count items in Journal Mailboxes, like every 2/3 hours, to monitor archiving from the other end), we change journal rules to 'free' the journalmailbox having issues. Eventually the item count goes down.
I do know that the advise is to 'investigate the corrupt item'. I try to drag the item to another Journal Mailbox, or out on the desktop, and then open the message. If that works, usually the item then archives. I have a series of known mails which seem to arrive corrupt.
11-10-2017 09:06 AM
have you considered switching to SMTP Journaling?
11-10-2017 11:28 AM - edited 11-10-2017 11:29 AM
This is the hated EV 11. We are working to get upgraded to EV 12 so veritas will be cool with us again, that upgrade is supposed to happen next weekend.
I went through and deleted any messages that failed to journal (looked for errors in outlook).
I also went through and dumped the items from the j2 folder (MSMQ)
I am now resetting the messages that are in a pending archvie state to ipm.note.
Hopefully the above will fix this issue.
Robert
11-10-2017 11:31 AM
Andrew. SMTP Jouranling is something to consider for the future. However what i need now is to get these 400K messages into Vault.
Robert
11-12-2017 12:47 AM - edited 11-12-2017 12:50 AM
We have this issue often where the Journaling Task just sits idle without archiving any new emails from the Journal Mailbox. We noticed that there are duplicate entries in the J3 queues in MSMQ and the solution provided by Support was to clear these entries manually without any root cause analysis.
So we ended up clearing the J3 queues manually (am now doing it via Powershell) and restarting the tasks every few hours so that Journal Archiving works without idling. Maybe this will help you.
#stop service
Stop-Service "EnterpriseVaultTaskControllerService"
#clear J3 queue
Get-MsmqQueue | where {$_.queuename -like "*private$* j3*"} | Clear-MsmqQueue
#clear MAPI profiles
Get-ChildItem -Path "HKCU:\Software\Microsoft\Office\15.0\Outlook\Profiles" | Where-Object {$_.PSChildName -like "*EV*"} | Remove-Item -Force -Recurse
#start service
Start-Service "EnterpriseVaultTaskControllerService"
11-13-2017 11:55 AM
Thanks for the tip. I will run that when i get a second. If i am not mistaken while the cmdlet is running users wont be able to search the server correct?
Robert
11-13-2017 08:31 PM
@rbollinger1212 Note that since you are on EV 11, I believe you have Outlook 2010 installed on the journal archiving servers. In that case the registry key for Outlook profiles is "HKCU:\Software\Microsoft\Office\12.0\Outlook\Profiles"
Searches are run using the indexes and for that the Enterprise Vault Indexing Service should be up and running. However, item retrieval may be a problem when the Task Controller service is stopped. Nevertheless, the commands take less than a minute to run and will not be noticeable by end-users anyway. Are your users running searches on the Journal archives?
11-20-2017 10:15 AM
Thanks for all your help on this. Unfortuantely neither of our 2 journaling mailboxes are being cleared out by the ev journaling task. (2 separate mailboxes, 2 separate databases, 2 separate exchange servers).
I am just going to export these 2 mailboxes to PSTs. and chalk this issue up to yet another EV problem with no real answer.
11-20-2017 11:16 AM
Hi,
Do any email gets archived or none at all? This could be an issue with bulk emailers that send out messages with the same ID. EV attempts to validate whether the messages are a part of the same group by comparing to all of the other messages in the group. This becomes an issue when there are 1000's of messages. The first message is compared to all of the remaining and then each subsequesnt ones are checked one by one.
This can be tested by turing JournalDelay to 0 which stops the grouping checking.
Dtrace of JournalTask would show the following and a number of total message that share the same ID-
(JournalTask) <5896> EV:M {CJournalPart::ProcessMessageGroup:#545} Number of messages that share the same internet-msg-id:[1255]
Regards,
Patrick
11-20-2017 11:50 AM
I can check for that in some of our logs, but in our case we use rule level journaling that captures all messages in and out. We're a sales company with about 4500 Employees. So i doubt this is a message id issue. And to diretly answer your question its doing say 1000 messages every 2 days.
Robert
11-21-2017 06:28 AM
Hi robert,
Exactly which EV and Exchange version do you use?
We have had a hotfix for EV11.0.1 CHF3 to archive faster from Exchange 2013 Journal Mailboxes. If you're using that CHF, or something earlier, upgrade to CHF5 (which has the fix incorporated). It has something to do with additional searchfolders being used in the Journal Mailbox.(etrack 3820305)
As for the archiving from the existing ones, can you create new Journal Mailboxes, new tasks, and new archives? configure Ev, then change your Journal rules in Exchange to use the new mailboxes. That way, the old ones become static, which most likely will allow the old tasks to archive the content. Worth a try.
11-21-2017 07:09 AM
Agree with GertjanA on being on CHF5 for EV if that is not already installed.
The issue with the bulk emailers is that sending out the same message to 4500 can cause the issues if the same msg-id is used. EV has to wait until all of the emails with the message come in and then compare them all one at a time to the rest. This is extremely time consuming. This would be evident in the Dtrace log.
Patrick
11-21-2017 08:27 AM
Thanks for the response. Yes we are running CHF5. And yes today i am going to just create a new journaling mailbox then add a new task etc.
Robert
11-22-2017 05:50 PM
Ok so after all that work.... Still no luck.
1) Created a new Journaling Database, Mailbox
2) Created New Journaling Tasks
3) All tasks start, mailbox sync's, no messages are removed from the mailbox.
here are the 2 (new) errors we are getting:
Failed to find configuration information for the Journaling Agent in the Directory Service on computer EVSERVERNAME01
Failed to inform the Journaling Service of successful storage of a Saveset in a Vault Store.
Reason: Failed to find configuration information for the Agent in the Directory Service on computer This could be because the Task was deleted before completing all its outstanding work. Task entry id: For more information, see Help and Support Center at http://entced.symantec.com/entt?product=ev&language=english&version=11.0.1.0&build=11.0.1.3706&error... (0xc0040b28)
For more information, see Help and Support Center at http://entced.symantec.com/entt?product=ev&language=english&version=11.0.1.0&build=11.0.1.3706&error...
Failed to find configuration information for the Journaling Agent in the Directory Service on computer VDCEEV01
This could be because the Task was deleted before completing all its outstanding work.
Task entry id: 108FC2295E7A34C4099A3DCFBF56470701011y00EVSERVERNAME
From my searches its looks like i need to go and make SQL changes. That is not something i will do on my own. Any ideas how to fix this without making sql changes?
Robert
11-22-2017 11:41 PM
What is Safety Copy setting on the Vault Store to where the mails are being archived from this Journaling Server? Complete the following steps to identify the Safety Copy setting.
Can you open the mailbox in Outlook and view the Message Class of all the items? If the emails are in the Pending state, you would need to change the Message Class of all such emails and archive the mailbox again. Complete the following steps to view the Message Class of emails.
Stop the Journaling Task associated to the mailbox. Complete the steps given here to change the Message Class of all emails to IPM.Note and once done, start the Journaling Task to archive the mailbox.
You may also need to repair the Storage Queues, but the above steps should help you clear the mailbox which seems to be a priority at the moment.