MSDP Environment - Configuration Considerations

[SYMAJ]

I am looking into a configuration on a site where the components of the NBU environment are as follows:

Master Server - W2K8R2, NBU7602

1 x MSDP Media Server - W2K8R2, NBU7602, Server 16 cores, 96GB RAM, Attached disk HP MSA P2000 G3 FC

2 x Media Server - W2K8R2, NBU7602, tape use only

The disk subsystem is dedicated to NBU and attached to the Media server over Fibre Channel.  MSA is dual controller - dual port, 24 x 4TB 7.5k rpm SAS disks.  Disks are configured into 2 x 12 drive RAID6 sets, one on CTL A and one on CTL B.  The site has configured the disk as follows (due to limitations on the MSA presenting LUNs larger than 40TB):

2 x VDISKS - 39.9TB each - chunksize 64KB

The VDISKS are presented to the W2K8R2 server over fibre channel where they have been configured into a single SPANNED dynamic disk of approx 80TB in size.

NBU MSDP pool has been configured with ALL components (database/logs/data) residing on this single volume.

On initial investigations I am seeing performance to local LTO5 tape drives significanlty better than performance of backups to the MSDP pool.  I believe the Media Server (16 core, 96GB RAM, 4 x FC interfaces to the storage) is adequate, but when I look at the MSA stats I am seeing CTLA being hammerred and CTLB basically Idle. 

I have a couple of concerns here, and am interested in comments relating to any experiences configuring MSDP in a similar environment.

1. Is a single 80TB MSDP pool viable on this configuration ?

2. Best practice states that the database/logs should be seperated fromt the data - i.e. different volume

3. Is it good practice to create a single spanned volume (80TB) in windows of the two VDISKS on the MSA ?

4. Is RAID6 slowing me down here ?  Would RAID5 be a preferable option ?  (Disks are 4TB 7.5k rpm - slow disk ??)

I am considering re-configuring the MSA to present 2 x 40TB RAID5 volumes, spread accross the controllers, and configuring 2 x MSDP pools to which I would spread the backup load.  Whilst we would not benefit from the same Global Dedup I believe this would improve performance.

Any experiences / comments appreciated.

AJ

[Nicolai]

My comments:

1: A MSDP can only be 64TB at time of writing. A advanced disk can be 80TB but I guess you have MSDP in mind. You can only have one storage server and MSDP pool per media server.

2: This recommendation is lifted in 7.6 because a new type of finger print database.

3: See 1

4: RAID levels is not interesting here. RAID6 is preffered becuase of the resilliance - both Symantec Appliances and Data Domain uses RAID6. Having enouch spindels to drive IO is more interesting usally.

[SYMAJ]

Nicolai,

Thanks for the input.  Just confirm for me that the MSDP pool will still show up in NBU as a PUREDISK server type ?

This pool is showing as 69.7706TB usable size (currently 39% used).  There is no 50xx appliances etc. and the storage is directly attached to the media server so it must have been configured as an MSDP Storage server surely ?  I was aware of the increase of the pool size to 80TB, but not that this was an ADVANCED pool only.  Will it allow you to create a pool larger than 64TB if this is not supported ? (I don't have access to my test system from here so can't check)

Concerning my point about presenting the two 40TB disks to the media server, then creating a single dynamic spanned disk - any comments here ? 

Thanks,

AJ

[Nicolai]

It does sound like a MSDP pool. MSDP pool are a fork of the PureDisk software.

While the NTFS file system can be 80TB (and thefor also a advance disk)  I am  sure you can't expand the MSDP pool beyound 64TB.

Did you by mistake make disk size = MSDP size ?

Update:

From the NBU 7.6 dedupe guide

About MSDP storage capacity
The maximum deduplication storage capacity is 64 TBs. NetBackup reserves 4
percent of the storage space for the deduplication database and transaction logs.
Therefore, a storage full condition is triggered at a 96 percent threshold.

http://www.symantec.com/docs/DOC6466 - Page 50

[SYMAJ]

Nicolai,

Not sure what you mean by the last comment "did you by mistake make disk size = MSDP size" ?

I am not sure as to the setup as it was done by someone else, but attached are the details of the Windows presentation, the storage server and the disk pool.

Thanks,

AJ

[Nicolai]

OK - Fair enough

The disk configuration look good. The attachment helped a lot.

You have 2 time 37GB - with one logical volume of 69.7706. However Netbackup MSDP software will not be able to use the space above 64TB. That is a limitation with-in Netbackup. We all hope to see that limited liftted.

Does this answer qustion 1 ?

[SYMAJ]

Thanks.

So NBU will only use the 64TB, but is it an issue having it created at 69TB ?  Will this cause problems ?  Also, what 'chunk' size is NBU writing to the disk at (16K, 64K, 256K) ?  Disk is currently set to 64K (default).

Also, I am still concerned as to the performance here, it is 3 x slower than the same backups to tape.  Is this an issue with the disk configuration, perhaps because they are using 4TB 7.5K rpm disks.  With the spanned configuration it appears to be hammerring one of the VDISKS (the one which is the first part of the spanned LUN) and the other is getting barely any I/O.  I don't need to go down the route of "how fast can we get data from the client" etc. as I am proving that performance is good when writing to tape.  When I point ot the de-dup pool however performance is unacceptable.

I have been implementing many 52x0 appliances recently, and now I see why the 'build your own' approach is fraught with issues......

What are my options to investigate / resolve the MSDP performance issue ?

AJ

[Bmitche]

As far as the 64TB limit....I have a MSDP pool with usable size=78.5TB. My used capacity is at 61.4TB. I don't know what if anything will happen when my used capacity goes over 64TB....but as far as being able to create and use a msdp pool over 64TB....it is doable.

[Nicolai]

No it is not a problem with larger volumes than Netbackup can use.

But I/O problem on MSDP is a serious issue. 

Take a look at these T/N (nbperfchk)

http://www.symantec.com/docs/TECH156743

http://www.symantec.com/docs/HOWTO72940

What value do you get ?

You should get >200MB/sec. Worst case scenario is a duplication of all MSDP data to a new media server and a new disk design with a lot of performance tuning before putting the "old" server into to production again.

What I can say - performance tuning is not a "next - next - finish" installation.RAID element size and NTFS cluster size need to be understood and that require time.

I also found this MSDP performance tech note:

http://www.symantec.com/docs/HOWTO61249