Hot-Catalog Backups-poor perfmance after MP5

[mlacount]

Hi all,

Hot-Catalog backups always worked great in our environment prior to MP5.

The Catalog is approx. 260GB. We do use Veritas compression but never seemed

to have issues backing up the catalog and really no negative impact on restores.

The main problem we are seing is that throughput has dropped to 6.2Mb/Sec from 13-14Mb/Sec.

So it now takes us almost 12 hours for a hot-catalog backup to complete.

Database is clean-ran consistency check. Removed all (0Kb) headers. Only change made was upgrading

to MP5. Reverting back to MP4 is not an option for us.

Any suggestions, tips would be greatly appreciated.

Thanks,

Mike

[sdo]

O/S version?  You mention MP4/MP5, but which of v5.0, v5.1 or v6.0?

[mlacount]

OS Version : Windows 2000 Server-SP4 . NB6 MP5

[sdo]

Did the performance drop suddenly, or over a period of time?

Is the volume hosting the catalog near capacity?

We had to setup regular defrag of our catalog, which resides on a SAN attached (replicated) cluster resource volume in Win2000 Adv Server SP4.  Performance tailed off terribly after about a year with no defrags.

I understand that Windows NTFS volume peformance starts to tail off when they go above 80%.

[mlacount]

The poor performance began after the install of NB6 MP5. To prove this, I reverted back to MP4 because of this and another issue we were experiencing where compression stopped working within our db/image directory.

Symantec escalated and sent us a revised bpdbm.exe which did fix the problem. MP4 Hot Catalog Backups always finished within 6hrs. Went back and re-installed MP5 (definately not best practice) - poor performance again. I'm now going to rely on Cumulative Online backups to get us migrated.

Last defrag was about 2 month ago. Do you recommend Windows Defrag utility or something else?

I don't want the defrag to interfere with backups/restores and daily replication. I may have to schedule some down time, that's never easy..

[Stumpr2]

---

Michael LaCount wrote:

Symantec escalated and sent us a revised bpdbm.exe which did fix the problem. MP4 Hot Catalog Backups always finished within 6hrs. Went back and re-installed MP5 (definately not best practice) - poor performance again.

---

Forgive me, but I am a little bit confused. Is there still poor performance after using the bpdbm symante provided to fix the MP5 bug?

[sdo]

I would recommend a defrag of teh catalog volume.

We defrag out catalog volume daily (Mon-Fri) between 14:00 and 16:00 - when backups are usually not running, and the catalog backup and it's image db cleanup have completed usually - however, sometimes, backups overrun, and/or the catalog backup overrruns and/or image db cleanup overrun, and overlap with the defrag - and we've never had any issues in several years of running Raxco Perfect Disk (we currently use v8) - all on a clustered resource SAN attached (HSG replicated over twin fabrics).  Also, we defrag the C: (system vol) and D: (NetBackup binaries) volumes weekly on Sunday's whilst backups are running.

Because it's Windows we occaisionally have to reboot during the day, and we never worry about whether defrag is runnign or not - for us Raxco Perfect Disk was a set and forget.

Catalog backup of a 150GB catalog (compressed using 7 day catalog compression from 150GB to 70GB sitting within a 100GB volume) is back down to an average duration of 3:30, but it did creep up to 6 hrs without defrags.

The daily defrag run takes about 45 to 60 minutes to pass over 150GB of data (70GB on disk) in a 100 GB NTFS volume.

Windows 2000 Adv Server SP4 MSCS (active/passive) cluster, running NBU ES v5.1 MP6, over twin fabrics.

[mlacount]

Yes, even after we received the revised bpdbm.exe, there was still the same poor performance during the online Catalog Backup. Throughput is still hovering between 6-7MB/Sec. when we saw twice that prior to MP5.

Whether or not MP5 is to blame, I have no proof. But that is the only change in our environment since the installation of MP4 back in Nov. 2006.

I'm going to go with Raxco Defrag to see if there are any improvements. Thanks for the suggestions.

Mike

[mlacount]

Just an FYI-

Successfully recovered our catalog onto a new Master Server using Full/Incr hot catalog backup...wasn't easy and process took most of a day. A full catalog backup now only takes a little over 3 hours. 

Being on a new server with no fragmentation is great but showed me that I need to plan a monthly analysis

of the local install drive and set up a process of defragging more often.

A fragmented drive can really torment a Master Server. Lesson learned.

Keep the local drive clean. Do an occasional Db consistency check, and Don't let your 

Master Server get too fragmented.  

-Mike