EV - Vault Partitions Scan for backup success, has stopped

see screenshots attached

Are you using the EV agent for BE? 
Is BE setting Partitions in Backup Mode when you are backing up?

Hi Tony,
no EV agent for BE, just 'standard' Backup2Disk-backup via Remote Agent for Windows Systems

how can i check whether BE is setting the partitions to backup mode, during backup? are there logs or do I need to manually log in to the EV Admin Console during time of BE backup, and check the backup mode of the partition then?

thanks
Ben

thanks very much Tony.  I've incorporated those scripts into my nightly SQL backup, to put the vault in backup mode and then take it out of backup mode- post successful vault backup.

Hi Tony,

wondering if you can assist with what seems to be an associated problem with backups...

i have archiving policy set so that the savesets are "removed after backup"

i notice that there are a number of items in different user mailboxes which remain in the 'This item is being archived by Enterprise Vault..." -state

i've read through some discussions, where you've also contributed eg:
http://aka-community.symantec.com/connect/forums/pending-archive-two-weeks
and associated KB's: http://seer.entsupport.symantec.com/docs/316761.htm &
http://seer.entsupport.symantec.com/docs/286728.htm

and as described, i ran the SQL queries
SELECT COUNT(*) FROM JournalArchive WHERE Indexcommited = '0'
SELECT COUNT(*) FROM JournalArchive WHERE BackupComplete = '0'

the result for Indexcommited = 0
the result for BackupComplete = 1340

[EDIT- i read from http://www.symantec.com/connect/articles/changes-backup-procedures-enterprise-vault-8

When the item is set to after backup, it is watched in two locations in the Vault Store Database.
One in the JournalArchive table with a BackupComplete flag set to 1, and one in a table called Watchfile, which contains a full path to the Archived item.

Every item listed in the watchfile table will be scanned upon Storage Service start or every 12 hours.
The StorageFileWatch process will go through each item listed in the WatchFile table and determine whether the item has been backed up or not. It determines this by looking at whether the archive bit exists on the file or not. If the archive bit has been removed, then Enterprise Vault removes that particular entry from the watchfile table, and then in the journalArchive table, it changes the “BackupComplete” column from a 0 to a 1]

does the changing of the BackupMode of the Vault Stores (as you pointed out in your previous post, and now i've implented) NOT change the BackupComplete flag from 0 back to 1?

what process would change this ?

[EDIT 2 - i manually restarted the Vault Storage Service, which then ran the archive process and created full shortcuts for about 900 of the 1340 items. [i presume the remainder of items were due to archiving running -after- the last backup

Question, when is the 12hr cycle started?  "...The Storage service performs the check when it starts and then every 12 hours."
ie do i read this as 12hrs of the last time the Storage Service was restarted?

so imlpementing a backup strategy, with this in mind, could be as follows:
01:00am - SQL Backup
02:00am - EV server backup via remote BUEXEC file-to-disk
03:00am - (providing backup @2:00am finished) - have Storage Service run.  this cuold be done by restarting the Storage Service at 3pm the day before??
03:30am - schedule EV archive of mailboxes to run

thanks
Ben

you should backup AFTER your archive run.  Doing it the way you suggest means you will have pending items until the next back up run the following night.

so the flow is this:

Archive Maiboxes
Put EV in backup mode
Backup EV and SQL
remove EV from backup mode

thanks for the clarification Tony.
i've put that in place.  will check it out over the next couple of days

can you confirm whether, when the storage crawler service runs, does it complete processing of any archived items that were hanging in the "This item is being archived by Enterprise Vault..." -state, into EV shotcuts

ie is the trigger for completing archive of items, the storage crawler service (at every 12 hr interval)
or, is the trigger to archive immediately when the vault store/partition is brought out of backup mode

eg in this scenario...

23:00 - Archive Maiboxes
02:00 - Put EV in backup mode
03:00 - Backup EV and SQL
04:00 - remove EV from backup mode
05:00 - Storage Service 12hr interval runs
...
08:00 -[users manually runs EV archive on items in Outlook]
17:00 - storage crawler 12hr interval ticks over and runs the service
> At this point, are the 08:00am archived files processed from 'pending' state to 'archived'. 
> Or are they only archived at 04:00am the next morning when the vault store is brought out of backup mode?

They will only be switched from pending to shortcuts when they vault store partition is backed up at night.

Hi Tony,

i checked this morning after yesterday's changes and it appears the archiving isnt completing the shortcut creation

as shown in the log table,
>the archiving job ran against both my Exchange servers (as shown in the table below at times 22:00:25 and 23:00:15)
>the backup mode is then set at 02:00:05,
>then various databases are backed up between 02:00:05 and 02:00:18 (we only have a handful of trial users at the moment, hence the small backup times). 
>the backup mode is then cleared at 02:00:20.

this indicates that the correct running order completed successfully overnight.

So I run this SQL query, with the following results:
SELECT COUNT(*) FROM JournalArchive WHERE Indexcommited = '0'     (result=0)
SELECT COUNT(*) FROM JournalArchive WHERE BackupComplete = '0'    (result=1189)

to me, this indicates that the 'new' msgs that were vaulted during last night's run have not been marked as being backed up.  and in practice, i checked one of the user's mailboxes that we had manually archived items yesterday and it's still in the 'this item is being archived byu Enterprise Vault..." pre-processing state

can you help explain?

08/26/2010 02:00:20	Enterprise Vault	Successfully cleared Backup Mode from all the Vault Stores that are managed by the Storage Service on this computer.
08/26/2010 02:00:20	Enterprise Vault	StorageDelete threads restarted successfully.
08/26/2010 02:00:20	Enterprise Vault	Request thread pool (InstanceId = [1]) stopped. Total threads stopped = [8].
08/26/2010 02:00:18	Backup	Database backed up. Database: EnterpriseVaultDirectory
08/26/2010 02:00:18	Backup	Database backed up. Database: EnterpriseVaultFSAReporting
08/26/2010 02:00:17	Backup	Database backed up. Database: EVVSGExpressVaultStoreGroup_1_1
08/26/2010 02:00:14	Backup	Database backed up. Database: EnterpriseVaultMonitoring
08/26/2010 02:00:13	Backup	Database backed up. Database: EVVSExpressVaultStore_1
08/26/2010 02:00:05	Backup	Database backed up. Database: EVVSITFilesVaultStore_2
08/26/2010 02:00:05	Enterprise Vault	Request thread pool (InstanceId = [1]) started with maximum thread count = [8].
08/26/2010 02:00:05	Enterprise Vault	Successfully set Backup Mode on all the Vault Stores that are managed by the Storage Service on this computer.
08/26/2010 02:00:05	Enterprise Vault	StorageDelete threads restarted successfully.
08/26/2010 02:00:05	Enterprise Vault	Request thread pool (InstanceId = [1]) stopped. Total threads stopped = [8].
08/26/2010 00:00:13	Enterprise Vault	The Exchange mailbox provisioning task has completed.
08/26/2010 00:00:11	Enterprise Vault	The Exchange mailbox provisioning task has started.
08/26/2010 00:00:11	Enterprise Vault	Completed a synchronization run for task: File System Archiving Task_Synchronization.
08/26/2010 00:00:11	Enterprise Vault	Starting a synchronization run for task: File System Archiving Task_Synchronization.
08/26/2010 00:00:05	Enterprise Vault	Starting synchronization run of all mailboxes on Exchange server MELEXC01
08/26/2010 00:00:04	Enterprise Vault	Starting synchronization run of all mailboxes on Exchange server SYDEXC01
08/25/2010 23:00:15	Enterprise Vault	Start to process the Exchange system MELEXC01.
08/25/2010 22:00:25	Enterprise Vault	Start to process the Exchange system SYDEXC01.