Forum Discussion

Hamza_H's avatar
Hamza_H
Moderator
7 years ago
Solved

Exchange 2013 Backup fails code error 24

Hi Everyone,

We have a backup issue for a long time on exchange server 2013.

The configuration is like this, 

24 Databases to backup with one cluster (yes I know it's not recommended)

The problem is, the backup of some DBs don't work and it's not the same database, every time we have a new failing backup.

after the application logs on the cluster, I've found the Error id 401 and 403 on the same time of the error 24 appears on the backup log, and yes we know it's a Microsoft issue as this TN explains :

 https://www.veritas.com/support/en_US/article.TECH197008

 

however, what we can't understand is, why the backup of the weekend (full_backup) works just fine, no error 401 and 403 on the cluster application logs and no backup error,  but during the week (also full backups) there are issues.

Is this related to the Microsoft KB? because we have the KB3000850 installed, do we need to install the KB3000853?

Thanks in advance.

 

HHA.

 

 

8
windows server 2012
  • Lowell_Palecek's avatar
    Lowell_Palecek
    7 years ago

    Your DAG is a virtual cluster name. NetBackup resolves it to a mailbox server. I think from the bpbrm log that the snapshot was taken on a host named MAILBOX02.

    I also see that the first "from client" message after bpbrm passed the backup directives was the error. It occurred almost exactly 30 minutes after the start of the job, and 29 minutes after bpbrm started bptm.

    ?19:58:40.800 [180589.180589] <2> bpbrm mm_sig: received ready signal from media manager
    >20:27:12.543 [180589.180589] <32> non_mpx_backup_archive_verify_import: from client DAG: FTL - socket write failed

    The only two sockets bpbkar32 writes to are to bpbrm (we see that works), and bptm (the media manager).

    If you have a verbose bpbkar log on MAILBOX02, I think you will find that it failed while trying to write the .edb file to bptm. That suggests that the problem is not with reading the snapshot but rather with writing to the media manager. This takes it out of the scope of my expertise.

11 Replies

Sort By
  • Lowell_Palecek's avatar
    Lowell_Palecek
    Level 6
    7 years ago

    The consistency check is optional if you are backing up a DAG. You can turn it off in your policy, or you can specify that the backup continue if the consistency check fails.

    Having said that, here's a guess from other customer experience.

    Have you configured enough space for your VSS snapshots? If not, you will get exactly this result. NetBackup will make a snapshot and start running a consistency check while backing up the data files. If the snapshot runs out of space, the OS deletes it out from under bpbkar32. The first thing you see in the logs is that the consistency check fails. If you configure the policy to keep going, the next thing you will see is that the backup of the data file (usually the .edb file) backs up too few bytes. (It think it fails because it knows how much data to expect, but I'm not sure.)

    • Hamza_H's avatar
      Hamza_H
      Moderator
      7 years ago

      Hello Lowell_Palecek,

       

      Thank you for your reply and your tip,

      unless mistaken, to turn off the option " Perform consistency check before backup with VSS" is under the client's props > exchange and not policy.. ( snapshot attached).

      I have already configured the space for VSS to illimited but it didn't resolve the problem.

      I've just turned off the option, I will wait for today's execution to see if it's resolved...

      btw, we don't have a policy configured (ms-windows) to back up the data (.edb), so there is no harm in turning off the option of consistency, in this case, right?

       

      Thanks a lot.

       

      Hamza.

       

      • Lowell_Palecek's avatar
        Lowell_Palecek
        Level 6
        7 years ago

        You are correct. The consistency check options (yes, no, maybe) are in the client host properties.

        MS-Windows policies exclude Exchange data files in order to prevent backing up multiple times, so no, your MS-Windows policy is not backing up your .edb files.

        The consistency check is a Microsoft requirement in non-DAG environments for them to support VSS backup and restore. They waived the requirement for DAG backups.

        With the option that failing the consistency check fails the backup, is the backup job (bpbkar32) retried from the same snapshot job at least once? If the OS drops the snapshot, the second failure will differ from the first. The first bpbkar and Event log will report that the consistency check fails part way through. The retry will report that the check fails to read the header information.

        With the other two options, check to make sure you are getting the whole .edb file. If not, I think you get status 1, but I'm not sure. Check the bpbkar log to make sure.

        You can run out of VSS space even if you tell vssadmin it's unlimited, if you provision it on a volume that's too small. If snapshot space is your problem you may see the results vary depending on how busy the Exchange system is, and how long it takes from the snapshot creation to the end of the backup. VSS system provider snapshots are copy-on-write.

        The TechNote you found, https://www.veritas.com/support/en_US/article.TECH197008, does not say it's a Microsoft bug. In fact, it describes the first bpbkar failure before retry when the snapshot runs out of space. I'm disappointed that the TechNote doen't say more. It's dated late 2013, which may be before I figured out the cause. I will see about getting it updated.