DFSR backup takes a long time to start data transfer

[rafanto]

Hello,

Please help with this behavior. We have an environment with two DFSR fileservers between two sites, with compression and deduplication enabled on all partitions.

We have implemented backups through Shadow Copy Components (as indicated in the documentation), with the Accelerator feature.

One of the partitions has 18TB of user information, the FULL backup took about 60 hours and the incremental backups took about 22 hours.

After carrying out a thorough review, we have observed that when starting the backup, the VSS Snapshot is not created immediately but rather takes, in FULL, about 30 hours or much more (we have been able to verify this through the command vssadmin list shadows), after that time the backup just starts transferring information.

We have also noticed that during that time of 30 hours, there are many bpbkar32 processes reading/processing files, that is, there is activity.

Another observation is that during those 30 hours of processing, another backup of another partition on the same server cannot be executed, because after 35 minutes it gives an error with code 6.

Is this normal behavior on a file server with DFSR and with that amount of information? Is there any way at the NetBackup level to speed up the process?

Regards!

Rafael

[Nicolai]

DFS-R is my hate backup object. However designed this at Microsoft deserves a really big brown star.

From my past experiences, you cannot run more than 2 concurrent backup stream for a DFS-R volume. Another lesson I had was running additional Shadow Copy Components actions for other drives e..g c:\, had the chance to mess with the DFS-R backups.

[rafanto]

Hello Nicolai,

Thanks for your response and suggestions, it's true that supporting a DFSR environment is really hard.

And it is also true that you cannot run two concurrent backups of the same volume.

But what catches our attention is that you cannot run a simultaneous backup of another volume if the bpbkar32 process has not finished processing on "counting" the files, it only allows us to run another backup if it has finished processing and has already been created the shadow (VSS snap), it is also possible that since the information is deduplicated/compressed, it has to be rehydrated, hence the long time.

Our problem in the end is that this pre-processing lasts a long time!

The only thing we can recommend is to migrate to a computer with more features at the IO level of SSD or NVMe disks. We do NOT know if this will improve, because as of today, this equipment has mechanical disks (SAS)

Regards!

Rafael

[Nicolai]

Hi @rafanto 

I haven't seen or noticed the mentioned behaviour of bpbkar need to finish counting before being able to start new jobs. Normally bpbkar processes are dedicated to one backup job only. 

I suspect the behaviour you see is more related to VSS than Netbackup. Have you tried to create a manual VSS volume snap while "bpbkar counting files". If same behavior seen, it looks like VSS has some sort of locking mechanism implemented

VSS admin commands: 

https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/c...)

[rafanto]

Hello, OK I will try to troubleshoot this behaviour, in other way, we will open a support case.

Regards!

Rafael