Catalog backup is really slow

[River1]

When backing up the catalog, is it supposed to take a really long time? it takes my setup 4.5 to 5 hours to backup the catalog.

My environment:

Master

Solaris 9 (A Domain of a 15k)

NetBackup 5.0 MP5

9940b tape drives in a L700

/usr/openv directory is only 170Gb in use.

5 media servers

Solaris 9

Also NetBackup 5.0 MP5

using SSO to attach to the same drives in the L700

each /usr/openv directory is ~2 GB or less in size.

 Any suggestions to get this to speed up?

[sdo]

The NetBackup v6.5 page:

http://seer.entsupport.symantec.com/docs/290282.htm

...curiously has a link to the v6.0 Backup Planning and Performance Tuning Guide:

http://seer.entsupport.symantec.com/docs/281842.htm

...whereas the v6.5 Backup Planning and Performance Tuning Guide, can actually be found here:

http://seer.entsupport.symantec.com/docs/307083.htm

...which was found in the catalog/listing of documents for NetBackup v6.5, which is here:

http://seer.entsupport.symantec.com/docs/NBUESVR_index.htm

Anyway, the first thing to do is to attempt to identify where the slowness lies.  Is it in the disk reader, memory management, or the tape writer?  Try the instructions at page 94 of the v6.5 Backup Planning and Performance Tuning Guide, re evaluating disk read performance independent of other NetBackup components.

Basically run the test, but read the "<install path>/NetBackup/db/images" path, as it is usually the images that are slow to back-up.  Or, is the EMM portion/phase/stream(s) of your catalog backup also slow?

If there's nothing that you can do re a slow disk sub-system, then maybe the alternative is to do daily incremental catalog backups?

[pkunk]

Try running this command to see the size of your catalog backup.

du -sk /usr/openv/netbackup/db 

Here are my specs:

V440 4 CPU 2.4 SUNW, Ultra SPARC

Solaris 9

RAM: 8 GB

------

L700e 

LT03 tape drive: native 80 Mb throughput

catalog size ~ 700 GB takes about 5 1/2 hr for the DB Backup.

writing to tape takes about 5-6 hrs.

[sdo]

I have the same problem. 

Can I ask both of you what the underlying disk storage sub-system/array is that your catalog images path resides upon?

I'm getting a very slow images path (10MB/s) and a quite fast (110MB/s) EMM part of our catalog backups.  It's almost as if the setup performs well with large files, but poorly with many small files.  I've run the bpbkar test (from the performance guide) and bpbkar reports reading about 38GB of images at 12.7 MB/s.  Not good.  Does anyone have tuning tips to get bpbkar to read faster?

Solaris 10, NBU v6.5.2A, storage NetApp FAS3070 over FC - 1 x 400 GB LUN synchronously replicated using OnTAP snapmirror - the 400GB LUN resides over 3 x 300 GB physical spindles (RAID 2D+1P - I think).

[Omar_Villa]

Check if you can see all your media servers and how is the connection speed with them, catalog grabs data from each media server you have and if for some reason cannot connect to any media server the disconnect time out can be an hour.

hope this help.

regards

[Wile_E__Coyote]

Is your catalog on SAN storage?

We had our catalog on SAN storage for a while, a very underutilized EMC Symmetrix DMX2000 to be specific. 72GB of data, striped across 9 spindles. It would take 3 hours 10 minutes to backup the catalog.

We moved it back to local storage, and the catalog time shrunk to less than 1 hour.

Nobody can tell us why it was slower on SAN. For all intents and purposes, the SAN should have been faster.

[sdo]

I've done three read tests on a very quiet master server.

Sun V445, 16GB, 2 x 147GB HDD.

Solaris 10, NBU ES v6.5.2A

Path                                            Size       Residing Upon                                    Speed

/usr/openv/netbackup/db/images      38 GB     SAN attached NetApp, snapmirror LUN   13 MB/s

/opt/openv/logs                              8 GB     SAN attached NetApp, plain LUN           41 MB/s

/var                                              4 GB     Local HDD                                         46 MB/s

(both of the NetBackup paths above use Veritas file file systems and symlinks to point to the SAN attached storage) 

The next thing I'm going to try is using tar to read the folder tree, and compare the time and speed.

[bcblake]

sdw303-

Did you happen to resolve this? Running into the same problem with db/images. Yep, it's a lot of files which I'm sure is the culprit. Back end storage is EMC Clariion. Incremental backups are ridiculously slow (over 6 hours). Catalog volume size is 500G, about 15-20G updates per day, about 30K files.

Thanks,

Brian