Forum Discussion

DanielCapilla's avatar
DanielCapilla
Level 4
3 years ago

Backup-to-disk-to-tape slow

Hello,

I hope, I'll get some thoughts about recurring performance problems with backup-to-disk / duplicate-to-tape.

The servers we use as backup servers are usually:

- HP DL380
- 64-128 GB RAM
- P408 controller incl. cache/battery
  - Boot-RAID 1 with SSDs
  - Backup-to-Disk-Space (B2D, RAID 5). Mostly approx. 50 TB
  - 7.2K SAS 12G
- Second RAID controller: P408e incl. cache/battery
  - LTO 6/7/8
- Backup-Software: Backup-Exec 21.x
- OS: Server 2019


What works well (no issue):
a) Backing up HyperV VMs through 10GBit from a HV-Cluster or standalone HV to the
    backup server SAS 7.2k RAID
    - 20-30GB/min
    - 5TB data
b) Backup up HyperV VMs through 10GBit to a connected, external LTO 8 drive.
    - 17 GB/min
    - 5TB data


What works bad:
- Duplicating from backup server to the LTO 8
  - 5-6GB/min
  - 5TB data
- When duplicating from disk (7.2K SAS) to tape, there is an HDD queue of up to 50(!)

 

We have tested any P408 controller settings. Different cache settings etc. pp.
We see this behaviour with many installations, where 7.2k SAS HDD are used.
Network ist 10/20GBit.

10k/15k harddisks (HP) have a maximum of 1TB - not enough space - to many disks.
SSDs are to expensive at the moment as backup-to-disk space.
All RAID-Controllers are performance-controllers, no rubbish.

7.2k disks and P408 controllers are best-practice in this scenario from hewlett-packards side.
I made several calls with HPE, but they have no clue.

 

Summary:
- Backup to Disk = Fast
- Backup to Tape = Fast
- Backup to Disk to Tape = Slow
- Network speed is tested and fine, tape-drives are tested and fine

What do you think, how do you solve this?
I'd like to hear your experiences regarding Backup-to-disk-to-tape.

 

Thanks in advance,
Daniel

21
Windows Server 2019
  • DanielCapilla's avatar
    DanielCapilla
    3 years ago

    Thank you so far, for trying to help.

    I think, i've solved it on my own.

    https://www.veritas.com/support/en_US/article.100043264

    "There are currently no plans to address this issue by way of a patch or hotfix in the current or previous versions of the software at the present time."

    The more issues we see with BE, the more we are looking forward to switch customers to a different product.
    Backup-Exec itself is a good product, but has a lot of shortcommings.

  • PJ_Backup's avatar
    PJ_Backup
    Moderator
    3 years ago

    Not saying this will solve your problems, but for reference there is a wealth of useful info in this pdf - few years old now but still well worth reviewing :

    https://blog.pingus.de/wp-content/uploads/2016/11/Backup-Exec-Best-Practices-Guide-v.4.0.pdf 

    Are you using dedupe storage or disk storage? If so this comment applies (pg 19) :


    Backup to a deduplication storage is less performant than Backup-To-Disk
    • Changing from Backup-to-Disk-to-Tape to Backup-to-Dedup-to-Tape may in some case extend the backup window beyond what is practical for lesser performant Backup Exec servers
    • Especially the attempt to read from a deduplication storage (i.e. for copying data to tape or verify) at the same time,
    where backup jobs are running will dramatically decrease the performance of the storage device.

    • DanielCapilla's avatar
      DanielCapilla
      Level 4
      3 years ago

      We do not use dedup. We use regular B2D space.
      Interesting is also, that backup to disk is fast, but duplicatiing afterwards incredibly slow.

      I already know the (realy good) best-practice guide.

      The only bottleneck I see is the 7.2k Disk.
      The disk space in an actual example consists of 8 6TB SAS 7.2k HDDs (RAID 5).
      *No Boot SSD RAID*
      BE is installed on the same RAID. The catalogues are written to the same RAID - could
      this be a massive issue?

      Performance bad after the B2D process, when copying to tape.

       

      1 = B2D from HV-Cluster to backup-server
      2 = Duplicate/copy to LTO drive (slow)
      3 = B2T (appending) to LTO drive (fast)

       

  • DanielCapilla's avatar
    DanielCapilla
    Level 4
    3 years ago

    Thank you so far, for trying to help.

    I think, i've solved it on my own.

    https://www.veritas.com/support/en_US/article.100043264

    "There are currently no plans to address this issue by way of a patch or hotfix in the current or previous versions of the software at the present time."

    The more issues we see with BE, the more we are looking forward to switch customers to a different product.
    Backup-Exec itself is a good product, but has a lot of shortcommings.