EV stopped archiving

[Attila_Musluogl]

Hi all,
one of our customers has got Enterprise Vault archiving problem. Until 2-3 weeks ago everything was fine about archiving files and mailbozes with PRC over HTTP and OWA. I could see number of archvable items (can find below) when I run in report mode, but If i tried to run archiving and shortcut processing mode, nothing happens, there is no action, any movement. I can check in store reporting. There is no important logs on event viewer. After that, I deleted all Exchange mailbox policies and desktop policies and I left only Defult Exchange Mailbox and Desktop Policies and resetted all settings. Enterprise Vault 8.0 cluster on Windows 2003 Sp2 Cluster Servers and working on SQL 2005 cluster on same servers. You can find Mailbox archiving Dtrace logs attaced. Do have any idea? If necessary I could send logs of windows also.. Thank you so much.

[MichelZ]

Yes, please post the Windows Eventlog (EnterpriseVault) to your original message.

[Attila_Musluogl]

Hello,
thank you for your answer. I've added Windows 2003 evt files in a zip file. Tasks are running between 18 PM to morning 6 AM. Thank you.

[MichelZ]

Hi there The logs do not seem to correlate. From the eventlogs, you have started an archiving job on 14.04.2009 - 15:11, but your archiving dtrace is from 15:28, where the task probably was already finished. Could you try the following: 1. Start DTRACE the "Archiving issues" 2. Start an archiving Job (do a RUN NOW in normal mode) 3. Stop DTRACE and attach the logs Are you sure that nothing happed? According to the Report, you don't have many items eligible for archiving, so those can easily be missed spotting? Cheers Michel

[Attila_Musluogl]

Hi Michael,
updated attachments, you can change extension of SEV170409.txt to .evt. So, I can not missed anything in storege reporting cause the mailbox that i tried to archive has 0 archived item, I tried different users for it, I mean newly enabled or archived before mailboxes.
And I realized that this happend after Symantec Endpoind Protection AV software installation. I read something about AV exclusion list from http://seer.entsupport.symantec.com/docs/284807.htm but I was so late. I think my msmq corupted now, I know because when I start an archiving tast or another tasks, nothing happened in "private queues" or  "outgoing queues" under Services in Message Queue. I tried to re-install MSMQ but nothing changed because my MSMQ service is running under Vault  Cluster. There is 53 records under "V:\msmq\storage\lqs" MSMQ installation folder, when I tried to delete all of them, I couldn't bring online my cluster MSMQ service, which one is admin queue, I don't know.. Is there anyway to recover MSMQ cluster servise? Have you any experience about it? Really appreciate for your help, regards.

[Wayne_Humphrey]

Purge the queues that have items stuck, once purged remember to retsrat the MSMQ service, that will clear you v:\msmq. remember once the ites are off the queue it will remain in the MSMQ DB untill MSMQ housekeeping has run.  MSMQ HouseKeeping happens every 6 hours or on the restart of the service.

[Attila_Musluogl]

Strange thing is there is no Enterprise Vault Queue for purging. I think I need tot remove queues manually but which for Vault and which for admin, I 'm not sure..

[Wayne_Humphrey]

Well if there are no Queues i think there is somethinge else wrong.  can you past a screenshot of MSMQ? as your last statment makes no sence.

--wayne

[Attila_Musluogl]

when I click on "private queues" at the righ side it's saying "there are no items to show..". I have to recover MSMQ I think, but how? You can find attached scrrenshot named MSMQ. Thanks for your responds.

[Wayne_Humphrey]

Your best bet is to stop all EV Services, delete all MSMQ's or reinstall MSMQ start up EV and it will recreate the queues.

[Attila_Musluogl]

Hello again,
MSMQ service running under cluster dpendent with EV services. Maybe I could remove from cluster, then could remove from both servers and re-install again. After that I coud make dependant with EV servers. But first I want take an online image from both servers by System Recovery. I 've got 14.000 files and 800 Mb mailboxes archived before. I will inform you after do this at the customer site. How does it sound? Have you any advice? Cheers for your help..

[Attila_Musluogl]

Hello all,
EV has begun to archive again. I don't know how but stores set to backup mode. When I set "clear backup mode" for all stores, now servers are archiving. I think Vault chooses this option when backing up stores, maybe stucked in backup mode after backing up stores. Could this happen automatically? And why can we choose this option manually? I'm little bit confused. Thanks for your helps, best regards..

[Wayne_Humphrey]

well thats strange as the trace you provided did not report that it was in backup mode.