NB 7 / PureDisk architecture question

[Hostasaurus]

Hopefully someone can help me with this, we purchased NB 6.5 four months ago and then got side tracked on installing everything so I can't remember how it had all been planned out with the sales rep at the time.  I just got an upgrade to 7 notification so I figure might as well use that since we haven't even installed it yet.  So, what we had originally envisioned was running one beefy server connected to a large storage array and a large tape library, backups come from the clients to disk, de-dupe happens on the server, what's left is written to tape for archival purposes or for restores from dates beyond what is kept on disk.  We don't want cpu activity on the clients related to de-duplication as our servers are compute-bound.

We have a box with dual quad-core cpu's and 32 GB of memory, it's got 5 TB of disk and a tape library.  Can we do all in one or would that be asking for trouble?  Currently we have an older netbackup install that's doing straight to tape backups for about 350 servers with a concurrency of 12 going to two LTO3 drives and it's working fine, obviously its not doing backup to disk or de-dupe.

What we've got purchased is described as:
SYMC NETBACKUP DEDUPLICATION OPTION 7.0 XPLAT 1 FRONT END TBYTE STD LIC
VRTS NETBACKUP PUREDISK 6.5 XPLAT 1 TBYTE FRONT END CAPACITY STD LIC

I'm honestly not sure what the difference is between puredisk and "deduplication option" in the 7.0 license or if it's just the two merged together now.  First question is can the combination of what we've bought accomplish what we want irrespective of hardware; i.e. can we back all the clients up to disk, de-duplicate that data on disk, then write it to tape?  I don't care if we can only restore from tape but would prefer to restore from disk if it's present, otherwise hit the tape.

Assuming the question of whether we can do what we want is yes, can we do it all with one server?  We don't have a lot of data to back up, at least initially, and it will probably take a long time before we'd reach 1 TB to back up with this server.  If the answer is no, what do we need, and what is optimal, obviously both would be considered.

Thanks in advance,

David

[Kevin_Good]

Yes, you can make that all work, one a single server.

Now, for the longer answer, which requires more details

How Many Newtwork Interfaces do you have?
What Speed are they?
What about Network uplink speed?
Network Utilization?

All of those answers impact the amount of time your backups will take to write to disk.  If you want to use Client Side dedupe, you can reduce the impact, both to the network, and that the network has on the overall backup performance.  This of course comes at a cost of increasing your CPU utilization on the clients (which may or may not be a problem)

Back End Questions

What is your connection to the 5 TB of Disk?
What type of Disk is it?
What type of Tape drives are you using?
What is the connection to the tape drives?

All those answers impact your ability to spin the data off to tape, (if you can't archive the data as fast as it comes in, you'll overload the Disk Storage, no matter how large it is) (Okay, so there IS a point where you won't overload the disk storage, if you are able to hold ALL of the backup images until they expire, and your Expiration Rate + Archive Rate > Data Growth) but that's a different discussion altogether.

[Hostasaurus]

Thanks for the reply Kevin.  I've decided to change the configuration somewhat since the time of my post just based on things I've read since then, and enhance the storage as well, but ideally if I can I'd still like to use just the one media server.  I'm now going to install a seperate master server and media server to start; the media server where the dedupe will occur is a dual cpu 2.5 GHz quad-core Dell box with 32 GB of memory.  The 5 TB of internal storage is built around eight 15krpm drives.  The new storage is a seperate JBOD from Dell with a 512mb cache raid controller to talk to it; this one is going to be 20 TB of 7200 rpm on 11 spindles.   Ideally I'd want the active backups and dedupe to occur on the faster internal storage but I think the external will be pretty decent too because of the spindle count even though those drives are slower; not sure if NB7 can use one set of storage for active backups and move things off to the slower disks after dedupe.  The additional storage was to satisfy retention requirements without needing to hit tape if we can avoid it; customers regularly want restores from weeks past so we want to keep a few weeks on disk now instead of just a few days and rely on tape for the rest.  The server will still copy off the backup images to tape each day for redundancy or further retention.

Network-wise, the server can be GigE into one of our core routers, multiple gig or even ten gig if we buy a card for the server.  I hadn't thought that far ahead since I wasn't sure what the throughput capabilities of NB7 with dedupe would be.

[Kevin_Good]

You can do the dedupe on the client side (before it even hits the LAN).
You can have it happen in-stream at the media server, so it writes the active backups in a dedupe form straight to the disk
If you REALLY want to, you can create a Storage Lifecycle Policy that lands the data (fully intact) on one disk volume, and then runs it through a de-dupe spin cycle to a second disk volume, and then spins it a third time to tape (today this is a fully intact tape copy, so it is restorable without the PureDisk engine running, they will have a Deduped + metadata version later this year.  this would require a 2 stage recovery, and is intended for archive data, not DR)

Of course, your Storage Lifecycle Policies are not limited to the last option, you can dedupe at the client, and use SLP to move the data from one disk to another, and/or to tape.

[Hostasaurus]

Very cool, thanks for that info.  I'm not wanting to do client-side dedupe because we run compute-intensive tasks on the servers so I don't want to take cycles away from that if possible.  I guess I'll do some performance testing once it's all set up and see if I get a noticeable difference between the in-stream dedupe to the fast disks vs the external array and if not I can just set it all up as an advanced storage group and write it wherever there's space then deal with the dupe to tape afterwards; tape capacity isn't an issue so I'll write full images there rather than relying on puredisk.

[teiva-boy]

While there is a CPU hit on the clients when doing client-side dedupe, the backup duration is shorter than a traditional backup as now only the unique segments are being sent over the wire.

So the trade-off is higher CPU about up to 30% CPU hit, but for a shorter duration than a traditional backup.

If using Windows for the media servers, and NOT Win2k8, make sure to do all the little disk tricks to get more DIsk I/O out of your JBOD.  disk alignment, play with cluster sizes, and NTFS formatting.  diskalignment is key though.

 For linux, it depends on the OS, but it's still an issue too.  As seen here:
http://thias.marmotte.net/archives/2008/01/05/Dell-PERC5E-and-MD1000-performance-tweaks.html

[Hostasaurus]

Wow, huge difference with the blockdev adjustment; not much change with the sector alignment.  What I've got so far with disk benchmarking on a Dell MD1200 with their H800 controller doing raid 50 across (10) 7200 rpm drives, 128k stripe size and a 16 TB partition has been:

1) XFS with hardware read ahead: 455 MB/sec write, 675 MB/sec read, 97 MB/sec random rewrite, 397random seek/sec.

2) XFS with hardware adaptive read ahead: 218 MB/sec write, 290 MB/sec read, 40 MB/sec random rewrite, 431random seek/sec.

3) EXT3 with hardware read ahead: 510 MB/sec write, 633 MB/sec read, 187 MB/sec random rewrite, 796random seek/sec.

4) EXT3 with hardware adaptive read ahead: 507 MB/sec write, 632 MB/sec read, 205 MB/sec random rewrite, 887random seek/sec

Pretty respectable, so I went with ext3 and hardware adaptive read ahead.  Now I set the kernel read ahead to 8192 and re-ran all the tests; the ext3 with hardware adaptive read ahead gained the most, jumping up to 516 MB/sec write, 959 MB/sec read, 292 MB/sec random rewrite, 806 random seek/sec.

[teiva-boy]

I'm glad you got some benefit to the link I provided.