EV - how to confirm all clips have replicated between prod and replica Centera

[goatboy]

Hi,

EV 9.0.2

This is a follow up from:

EV - migrating to new Centera

https://www-secure.symantec.com/connect/forums/ev-migrating-new-centera

As part of the migration, I want to ensure that all clips have replicated between the old prod and replica Centeras, before we turn on archiving to the new Centeras.

I current have 5 vault stores, 5 partions and 5 Centera pools with the following safety copy settings:

1. Mailbox archiving - After backup (immediate for Journaling)

2. Mailbox archiving - After backup (immediate for Journaling)
 

3. FSA - Immediately after archive

4. Journaling - Immediately after archive

5. Journaling - Immediately after archive

Once the EV tasks are stopped, I need to ensure all clips have replicated between the old prod and replica Centeras.

This is the plan:

1. Check the usage page and confirm "Awaiting Backup" = 0 for all vault stores

2. Run the following SQL query across all vault stores:

SELECT COUNT(*) FROM JournalArchive WHERE Indexcommited = '0'
SELECT COUNT(*) FROM JournalArchive WHERE BackupComplete = '0'

3. Check MSMQ queues: Check the 'A' Queues and 'J' Queues to confirm if there are any items left to be processed

Anything else I am missing? I understand that as we have various safety copy settings, just one check may not be enough to confirm there is no outstanding replication between Centeras.

[JesusWept3]

well regardless of After Backup or Immediately After Archive, in a centera with replication, you should still see BackupComplete = 0 when items haven't been replicated

But your best best is to look at the WatchFile table.
This will list all of the ClipID's that its looking for and continuously pinging against the replicated Centera.

If you were to dtrace StorageFileWatch you would see EV doing its checks against the replica each time
also the IndexComplete doesn't need to be queried against as that would not stop the replication.
In a centera collections scenario, you would typically see the items being indexed from the temp centera holding location before it even makes it to the primary centera, and if the primary centera is online indexing will get the item from the primary not the replica

[goatboy]

thanks. So, if the WatchFile table is empty, it's a good indication that everything has replicated?

(btw, we have centera with collections enabled).

[JesusWept3]

if ite empty it means that there are no files its checking for

For every item that is BackupComplete = 0, you should have a corresponding WatchFile entry
Because EV uses the WatchFile to go out to each item individually and check the archive bit, or in this case to see if its replicated, as opposed to scanning every single file that exists.

 

[goatboy]

ok so if the replication has finished, then we should have no items with BackupComplete = 0, and therefore an empty WatchFile table. So that will be my check. many thanks.

[JesusWept3]

unless you are archiving items to that vault store, then i would fully expect to see some items in there

[goatboy]

mmm, now I am getting confused. From: http://www.symantec.com/business/support/index?page=content&id=TECH66202

- If the JournalArchive and WatchFile Tables are empty, it would be safe to observe that there is nothing waiting for backup or indexing. Check the Storage Archive Queue for backlogged messages.

The way I read this, is that if I have stopped archiving and everything has been successfully backed up (or replicated between Centeras in this case), then the WatchFile table should be empty. Not correct?

The goal of this is to ensure that just before we cutover to new partitions on new Centeras, we don't have any non-replicated, orphaned clips at that point, which I assume EV will not be happy about. Thanks for your advice.
 

[JesusWept3 · Accepted solution]

OK so lets say in this scenario that there are some items awaiting replication, you are keeping the same ClipID's right? otherwise there would need to be a process to change all the clipID's in the collections table
So in the WatchFile table, you would have a reference to a ClipID, the StorageFileWatch will poll the replica for that ClipID

So lets say you have a backlog of clip's on your old primary centera to go to your old replica centera...if the new centera has those clipID's on the primary and it replicates correctly to the new replica, EV would query the new replica centera for that clipID when it switches over.

The technote you referenced is regarding items in mailboxes being in a pending state
so if you can confirm that theres nothing pending backup, then check the storage queues as its getting stuck before it even gets to storage.

If nothing if archiving anymore to that vault store, then nothing new should go in to the JournalArchive and WatchFile table