Index Locations still in Backup Mode in spite of Events suggesting that the Backup Mode was cleared

[The_Dark_Knight]

Hi,

Working on EV 10.0.4 in a Clustered Environment. The Cluster is an N+1 MSCS Environment.

Have been noticing an odd behaviour in EV for quite some time now. For the moment, I do the most logical thing to do but then it is not supposed to be like that. The issue is related to Index Locations being in Backup Mode.

Every day there is an Incremental Backup of the Index Locations that happens early in the morning 5:30 AM IST. In spite of it when I check the EV Server System Status log (after a re-run), it shows that the Index Locations are in Backup Mode.

When I try to clear the Backup Mode via the EV Site, it say that ifailed to contact one of the Servers (everytime the same EV Server 02) with a "General Exception" Error.

On EV Server 02, I do see events suggesting that the Backup Mode was cleared on the same morning within minutes when the backup mode was set.

Please check the screenshots attached to this post.

Need help in troubleshooting this issue in the right manner.

[JimmyNeutron]

So the indexes enter backup mode successfully but when its time for the system to take them out of backup mode it does not happen?

[Rob_Wilcox1]

It works fine with Node 1?  It might be worth rebuilding the second node...

[JimmyNeutron]

Dark Knight,

Can you make sure node two is allowed to run scripts:

Before you run the script, you must set the PowerShell script execution policy, to allow only signed scripts to run. You only have to do this once.

To set the PowerShell script execution policy

1. Start the Enterprise Vault Management Shell.

2. Run the following command:

   Set-ExecutionPolicy -executionPolicy AllSigned

From: https://www.veritas.com/support/en_US/article.HOWTO108641

[GertjanA]

How are you setting backup mode? If you use a bat file to set backupmode on the SITE, try setting it on the servers instead.

Please see this technote: http://www.veritas.com/docs/000018747

That might be the issue for you.

[The_Dark_Knight]

Hi,

Apologies for the late reply. I found something else while troubleshooting another issue but is related to this scenario.

http://www.veritas.com/docs/000018954 - Event 41397 logged while setting and clearing Backup Mode on Index Locations.

The scenario mentioned here in is exactly what I see in my environment.

The solution provided is easy provided it is a stand alone EV Server. But my environment is a clustered environment with 2 EV Servers running on 2N +1.

The solution is articulated in http://www.veritas.com/docs/000064598 - What is the purpose and functionality of the repository.xml file and index metadata utilized by the 64-bit indexing engine introduced in Enterprise Vault (EV) 10.0?

To be more precise;

In the event of repository.xml corruption, the following disaster recover (DR) steps should be carried out by an Enterprise Vault administrator.

      Steps:

      1- Stop the EV Indexing service.

      2- If repository.xml is corrupt, delete the corrupted repository.xml file.

      3- Start the EV Indexing service.

      4- On start up the EV Indexing service will re synchronize repository.xml and other missing metadata.  

Note: This re synchronization process can take an extended amount of time depending on the number of index volumes (collections) managed by the indexing service. Once the metadata synchronization process is completed then indexing service will become fully functional.