cancel
Showing results for 
Search instead for 
Did you mean: 

Slow Duplication and Replication Issue

Hi All, 

This is to share with you the Long pending issue which we faced in our environment for 6 months . Our Duplication Jobs started running slow and we started seeing backlog of Duplication jobs building up on daily basis which went up to 500 TB.

Earlier Setup :-

Duplication Jobs Window - 24x7 

Replication Jobs Window - 24x7 

Storage Unit groups Used for Disk Pool Storage units and Tape Storage units.

Netbackup Appliance 5230 ( 2.6.0.2 )- 1 Master Appliance , 1 Media Appliance  in Primary Site , 1 in Remote site for DR.

Netbackup Version - 7.6.0.2.

 

We started seeing Backlog increase slowly which began to increase to huge size over the period of few months. Backups also were running slow and Replication also went slow. Overall Appliances Performance became very slow.

 

Case raised with Symantec and worked with Backline To resolve this Performance issue.

Did Data Collects , Logs collected and Sent to Symantec multiple Times.

 

Actions Taken:

 

1. Replaced predictive disk failures for 2 disks in Master appliance on two occasions.

2. Upgraded Netbackup patch to 7.6.0.4

3. Changed the NUMBER_DATA_BUFFERS and NUMBER_DATA_BUFFERS_DISK to 32 to avoid long delays.

4. Broke the Stu Group and Started using Netbackup Disk Storage Unit and Tape Storage Unit for Backup and Duplication respectively.

5. Added Alternate read server in SLP as same as the one used for backup to overcome Data travelling from appliance to other for duplication to tape.

6. /usr/openv/lib/ost-plugins/pd.conf  changed from PREFETCH_SIZE = 33554432  to PREFETCH_SIZE = 67108864

7. /disk/etc/puredisk/contentrouter.cfg  changed from PrefetchThreadNum=8   to PrefetchThreadNum=16

8. Changed the Max duplication to 5 TB and Min to 500 GB

9. Changed the Replication window and Duplication window such that they dont Overlap. 

10. Upgraded the Appliances BIOS and RAID Software

11. Upgraded the Netbackup software to 7.6.1.2 Version

 

Now After the Upgrade to 7.6.1.2 Version , The performance has improved Drastically and we are seeing Backlog decreasing rapidly with 50TB per day using 10 Tape drives.

Replication performance has also increased and Backups are completing well within time.

 

Cheers and thanks to Symantec Support and all others who helped us in this issue.

 

 

 

 

 

1 Reply

Points 9 and 11 are critical.

Points 9 and 11 are critical. Thanksf or sharing.