Header synchronization

[Edyxx]

Hello,

We are using Virtual Vault and I have a user with a lot of emails (1,2 millions). We had to perform a full reset and resync now the Vault. Normally when a new vault is set up EV synchronize first the header and then the content. But in this case header synchronization failed, but contentsynchronisation coninues to run. For better understanding I attach a screenshot.

From time to time the "Synchronize Now" button is active and I can click on it and the status for Header synchronization changes to "synchronize", but it fails again. 

Is it just a matter of time until the content synchronization is finished and afterwards Vault will finish the Header synchronization too? Or does it mean if I have this state there is something wrong with Header synchronization?

Thank you,

Edy

[Rob_Wilcox1]

You'd have to start by looking at a client trace and seeing WHY the header synch fails.

It might fail straight away, or after some time of building ...

[Edyxx]

Thanks - what is the different between a full reset and a partial reset?

[David_Jensen]

A partial reset, which ensures that the next Header synchronization is a full synchronization.

A full reset, which forces an initial synchronization of both the Header information and the Content Cache. The effect is the same as if the user has just been enabled for Vault Cache.

More information can be found here: http://www.symantec.com/docs/HOWTO84900

[JesusWept3]

Basically when a synchronization happens after it has been completed the first time, the next sync at the last item sync'd and then gets items archived after that and syncs changes from the Journal Update/Stub/Delete/Archive tables.

However there may be some circumstances where it may be missing items that were updated but you hadn't sync'd in a while, so a partial reset would start with item #1 and go through and make sure whats in the VV matches whats in the archive, that is the partial reset.

It does not delete any files either

But a full reset deletes all the XML/INI/DB/MDC Files and starts off from scratch, rebuilding the MDC and then building all the DB files on the servers and downloading them via BITS

As for Content Synchronization (Vault Cache and DB Files) and Header Synchronization (Virtual Vault and MDC File) -- these are two independent functions, so you can synch the header synchronization while the content synchronization might be failing -- and you might have your MDC fail to build but your content cache is building just fine.

One thing you could do is browse to http://yourEVServer/EnterpriseVault/ClientDiagnostics.aspx

This will show you all the failed synchs and give you some suggestions, it could be that the MDC is corrupt, or something is locking the MDC, Antivirus can intefere with it and all sorts.

For ClientDiagnostics you will have to have the EV10 SP3 Client and server
Also its worth looking at different cumulative hotfixes and service packs for your version of EV to determine if there are important VV/VC fixes that could be applied

[Edyxx]

Hello,

Thanks so far for your replies. This is helpful to understand it better. I do not have access to the EV server. I'm not sure which version it is. But the EV client is 10.0.3.1090. Is it SP3?

The biggest problem is that the Virtual Vault in the Outlook navigation pane only shows the Search Folders. Header synchronization keeps failing, content synchronization works fine. In the KVS folder I see a lot of db files and a mdc file. Are the db files created for the content and the mdc file for the header? The mdc file is over 5 GB big. I have turned off our Antivirus program which is Trend.

In the client maximum trace I can spot this

HDR:SYNC:ARC: Synchronization failed (COM error):: 0x80040813

The most thing I'm worried about is that VV doesn't show all the folders I see in the Archive Explorer.

Thanks,

Edy

[JesusWept3 · Accepted solution]

So the DB files are the full contents of your Archive split in to quarters of the year based on sent/received date
So you have like

2014_01_03_001.db -- this contains all email for January to March 2014
2014_04_06_002.db -- this contains all email for April to June 2014
2014_07_09_003.db -- this contains all email for July to September 2014
2014_10_12_004.db -- this contains all email for October to December 2014

So if you've had emails dating back to 2004 you'd theoretically have 40 DB files.

The MDC is the MetaDataCache, which is used for the Header Synchronization.
This contains the full folder structure of your archive and brief snippets of each email.
The data is things like Subject, Sent Date, To, From, Size etc

This MDC is whats mounted in Outlook and displays as your Virtual Vault.
When you click an item, it then looks in an XML file to determine which of the DB files the item exists in and pulls the item from the DB file.

You can have MDC without the DB files, which is contentless cache.
When this happens and you click an item, it gives you a yellow banner and the first 200 chars or so of the email so you can hopefully determine if its the right email, to open the full email you have to double click it and it will retrieve the item from the server

There used to be a policy in old versions of EV9 and below where you could have it retrieve an item from the server to show in the preview pane, but its a performance killer, so if you have someone who is using the down arrow to scroll through email, you could potentially be making 100's of calls to download.asp which isn't particularly good if you have lots of userrs doing it.

You can also have the DB files without the MDC, which is just regular Vault Cache.
Meaning that if you have shortcuts in your mailbox to archived items, when you double click it, it will retrieve the item locally and not from the server, and if you're offline (or depending on the setting) it will use "Offline Archive Explorer" -- so you can browse your vault cache and open email from there if you don't have shortcuts.

Typically the MDC can be anywhere between 8% - 15% of the archive.
So if you have a 5GB MDC, then it could be safe to assume that the entire archive size is somewhere around 50GB in size, though you can control the size of the vault cache through the policy where it will only keep the newer items on disk and remove the older items.

MDC files cannot be controlled in terms of size though, it will grow as big as it needs to.
And when you drag and drop items from your mailbox to virtual vault, the full items are placed in to the MDC, then when you synchronize, its placed in one of the DB files and made in to the snippet like all the other items.

The MDC does not shrink though, so if you have say a 120MB MDC, and then you drag and drop in 1GB of data, you then end up with a 1.1GB MDC, you sync and if though it may now make the MDC 180MB, the file is not compressed so will still consume 5GB

There are certain plugins that can intefere with how Vault Cache operates. such as the Voltage 6.x client, Windows Desktop Search and Antivirus, especially with on demand scanning.

As for 0x800408013 -- that's a difficult one, typically its an error you sometimes see with Outlook, but generic troubleshooting advice is given but no real concrete explanation for what it is.

I did however see some saying that the PST was corrupt.
So what you could do is close Outlook, go to your Office15 directory and run ScanPST against the MDC file and try again, because your folders probably aren't showing because it can't create them.

One other thing you could do is just rename the MDC file and resync and it will recreate the MDC from scratch, but if you truly have that large of an archive, could take some time to rebuld

[Edyxx]

Thank you for your detailed explanation. This is very helpful and highly appreciated. I guess archived contacts and notes items will also be build in the MDC file. Our policy is that every item older than 30 days will be archived automatically. Unfortunately contacts, notes and calendar were also included in the policy by mistake.

I need to move them back to the Outlook mailbox. I see all items fine in Archive Explorer and have to wait until the synchronization to Vault cache will be completed. It's a very big archive which many years worth of e-mails. I only see a  chance to move items back to the Outlook folder once the MDC file is ready.