Forum Discussion

Per_Holmgren's avatar
Per_Holmgren
Level 4
6 years ago

New Exchange 2019 features and new headaches

FYI:

New 8.2 environment backing up Ip-Less DAG on new Exchange 2019 running on Windows 2019 Core Servers.
Backup is working ok, restores not so much. We suspect that a new feature in Exchange 2019 causes the restore to fail at the commit/mount database stage. In Exchange 2019 they have implemented a SSD cache database, for quicker searches etc, that are referenced by all mail databases. When you restore an entire database, eg EXDB01, to a recovery database, eg RDB01 , the recovery database have no reference to the cache and mounting the database fails.

Progress log filename : N:\Program Files\Veritas\NetBackup\Logs\user_ops\nbadmin\logs\jbp9FAC.tmp.log Restore Job Id=145846
Restore started 08/14/2019 10:33:14

10:33:14 (145846.xxx) Found (4 608) files in (1) images for Restore Job ID 145846.xxx
10:33:25 (145847.xxx) Found (4 608) files in (1) images for Restore Job ID 145847.xxx
10:33:33 (145847.xxx) Searched ( 10) files of (4 608) files for Restore Job ID 145847.xxx
10:33:33 (145847.001) Restoring from copy 1 of image created 2019-08-14 10:27:26 from policy Silver_Exchange2019_DAG
10:33:53 (145847.001) INF - Beginning restore from server xstobup01.ref.xxx.xx to client XSTOEX41.REF.XXX.XX (client_hostname XSTOEX41.REF.XXX.XX).
10:33:55 (145847.001) TAR - Microsoft Exchange Database Availability Groups:\
10:33:55 (145847.001) TAR - Microsoft Exchange Database Availability Groups:\xstoexdag41\
10:33:55 (145847.001) TAR - Microsoft Exchange Database Availability Groups:\xstoexdag41\Microsoft Information Store\
10:33:55 (145847.001) TAR - Microsoft Exchange Database Availability Groups:\xstoexdag41\Microsoft Information Store\EXDB01\
10:33:55 (145847.001) MNR - The file was renamed to the following:
10:33:55 (145847.001) UTF - Microsoft Information Store:\RDB01\
10:34:22 (145847.001) TAR - Microsoft Exchange Database Availability Groups:\xstoexdag41\Microsoft Information Store\EXDB01\Database
10:34:23 (145847.001) MNR - The file was renamed to the following:
10:34:23 (145847.001) UTF - Microsoft Information Store:\RDB01\Database
10:34:26 (145847.001) TAR - Microsoft Exchange Database Availability Groups:\xstoexdag41\Microsoft Information Store\EXDB01\Logs_1565771088
10:34:27 (145847.001) MNR - The file was renamed to the following:
10:34:27 (145847.001) UTF - Microsoft Information Store:\RDB01\Logs_1565771088
10:34:36 (145847.001) INF - There are no jobs remaining for this restore request. Exchange database recovery will be performed.
10:34:37 (145847.001) INF - After restoring a Microsoft Exchange LCR, CCR, or DAG database, you must use the Exchange Management Shell to resume replication.
10:35:01 (145847.001) WRN - MountDatabase failed with error(0x0) for machine xstoexdag41, storage group RDB01, database RDB01
10:35:01 (145847.001) WRN - Exchange database recovery may have failed. Please check the event log for further details.
10:35:01 (145847.001) ERR - An Exchange restore error was detected. You may need to manually mount the Database stores to successfully finish the restore.
10:35:01 (145847.001) INF - TAR EXITING WITH STATUS = 5
10:35:01 (145847.001) INF - TAR RESTORED 6 OF 6 FILES SUCCESSFULLY
10:35:01 (145847.001) INF - TAR KEPT 0 EXISTING FILES
10:35:01 (145847.001) INF - TAR PARTIALLY RESTORED 0 FILES

10:35:01 (145847.001) Status of restore from copy 1 of image created 2019-08-14 10:27:26 = tar did not find all the files to be restored

10:35:01 INF - Server status = 5
10:35:02 (145846.xxx) INF - Status = the restore failed to recover the requested files.

10:35:02 INF - Server status = 2810
10:35:02 (145847.xxx) INF - Status = MS-Exchange-Server policy restore error.

FROM EXCHANGE SERVERS ERROR LOG:
msexchangerepl (5708,U,98,15.02.0330.008) RDB01\XSTOEX41: Database recovery failed with error -1216
because it encountered references to a database, C:\ExchangeMetaCacheDbs\EXDB01\EXDB01.mcdb\EXDB01-mcdb.edb', which is no longer present.
The database was not brought to a Clean Shutdown state before it was removed (or possibly moved or renamed).
The database engine will not permit recovery to complete for this instance until the missing database is re-instated.

I have just now started a case with Veritas support.

Exchange 2019
Other
  • Marianne's avatar
    Marianne
    Level 6
    6 years ago

    Per_Holmgren 

    Can you confirm that the Exchange version is 2019 CU1?

    According to the 2019-07-25 version of the DB SCL, only this version is supported with NBU 8.2.

    I am sure Lowell_Palecek will be along soon with some 'inside info'. 

    • Per_Holmgren's avatar
      Per_Holmgren
      Level 4
      6 years ago

      Hi

      Yes, they are running 2019 CU1. Funny thing, we did some early tests with 8.1.2 software and were successfull with backup and restore. At that time there was only one node and SSD cache had not been enabled. Since it was not supported the Exchange 2019 project came to a halt until the 8.2 release. Now we have 8.2, a clustered DAG and SDD cache and it does not rock'n roll at all.

      • Lowell_Palecek's avatar
        Lowell_Palecek
        Level 6
        6 years ago

        I do not have any insight into this scenario, but thank you for the heads up. I will start a conversation with the development engineers while your support case works its way through the system.

        You can check your backup to confirm that your cache data files are not being backed up. With that confirmation, you can urge our support engineers to escalate it to engineering. Use my name.

        I expect that it will turn out that NetBackup hasn't implemented support for the database cache. I'll ask the developers.