Best Practice for SAN Backup

[Techinfo-BPM]

Hi,
    I have a question about time of the backup and another question about the best practice to backup on a SAN.

    I explain what I have:

• 1 IBM DS3300 with 12 1TB Hard drive,  with 1 controller and 2 power suppply,
• 1 EXP3300 with 12 1TB Hard Drive, with 2 power suppy
• 1 IBM x335 With Backup Exec 12.5.

    The issue is when I do the backup on a USB Drive, the job rate is around 522.00 MB/min but when I do the job on the SAN the Job rate is around 259.00 MB/min.
    I want to know if that is normal because if the job rate saty a 259 the ob don't complete in the same day.

    The second question:
          I have 8 raid 5 of 3 hard drive on my SAN and I want to know what is the best practice to manage my backup. Because windows can't see more than 2TB and my SAN count 24 TB.

   If you need more explanation let me know.

Thank you.
 

[JoaoMatos]

Hi there,

Can you talk more about your hardware and BE configuration?
259 MB does not make sense on the SAN.
How do you have your tape device connect to the san?
Tell me more about your topology.

Regards,
JoaoMatos

[CraigV]

Hi there,

That speed doesn't sound right.
Thing to look out for would be:

1. Your zoning is correct, and follows best practice...ie. You have zoned your servers to see the drives presented to the SAN separately from a tape drive.
2. Do a copy to those disks from Windows and take note of the speed you are getting.

What else are you backing up? Are there other servers connected to the SAN you are backing up large amounts of data from? If you have a SAN-attached tape drive, it might be worth your while looking into the SAN SSO option which allows your other SAN-attached servers to back up to the disks/tapes.

Furthermore, we get around the 2TB LUN size by presenting multiple LUNs to Windows, and then creating a spanned drive. For example, our SAP R/3 DB is 4.4TB, hosted on a 5TB spanned volume.
however, depending on the amount of space you are backing up, you would also need to consider backing up using a different method. For example differential/incremental rather than FULL which would take a number of hours to do.

Laters!

[Techinfo-BPM]

Hi,
    My BE is configure to go on some server and PC ( around 15), some SQL,  one exchange and othe is for the DATA.
 

   When the backup start on the SAN is going very fast but after some minutes is falling to 259 MB

   I'm plug on the ethernet with my san (With ISCSI initiator) (My san is in the factory and the server is in the office) We have a fiber between the 2 switchs and the san and the server goes a GB,

   The usb drive is plug directly in the usb 2.0 port of the BE server. (IBM x335)


If you need more info let me know.

Thank you.

[teiva-boy]

Since you are using iSCSI, leverage MPIO.  You'll need good quality dual port server NICs (I prefer Intel server NIC's)

You'll also want to benchmark from your BE server to your SAN, your possible throughput.  Use something like HDTach for low level info as for max possible read/write speeds, and an FTP copy to also demonstrate some protocol overhead.

[Techinfo-BPM]

HI,
   I do the test on two différent drive on the SAN and this is what I get.
  See attach image (r.jpg and w.jpg)


Thank you

[teiva-boy]

For a SAN, that is pretty dismal.  I get more throughput from a single desktop SATA drive.  

Something is a miss in your environment and storage area network.

I'd start digging into your initiators, MPIO, Jumbo Frames, drive configurations, etc...    I dont know if it could be covered in a forum, I'd best suggest you get a professional consultant to do the testing and configuration.

[CraigV]

Speed of your disks?
Is your zoning correct in that you're not zoning other SAN hardware with your backup server?
Any hardware failures on the storage?

[Techinfo-BPM]

The speed of my disk is SATA 7200 RPM (data Rate 3 GBps)
I don't understand what you mean with correct zoning.
No I don't have hardware failure on the SAN storage.

Thank you

[CraigV]

OK...I see that you are using iSCSI.
What sort of traffic are you getting across that fiber connection?
The SATA 7200RPM disks can have something to do with a bottleneck. SATA is more useful as archival storage...
Have you always had this issue, or is it a new occurance?

[Techinfo-BPM]

RE.

We have around 20 users pass on RDP and with a ERP.
We have the SAN from 2 weeks from now.

Thank you.

[Khue]

I wrote out a long post for you about the setup of your DS3300 but I clicked an email link and it erased what I typed. =/

I have some experience on the DS products from IBM. I am interested in the design of the storage subsystem. I am not sure what you meant by saying "8 raid 5 of 3 hard drive." Can you please elaborate?

Here are some concerns:

• The more arrays on a subsystem, the less i/o available for each array
• The more LUNs on a subsystem, the less i/o available for each LUN
• The fewer controllers on a subsystem, the less i/o available to the entire system
• SATA, while the theoretical speed is high, I really haven't seen high i/o on any system other then the XIV storage system which is a completely different animal then traditional subsystems

Answer these questions if possible:

• Do you have 12, 1 terabyte drives available or do you have 12 terabyte drives with one allocated to a hotspare?
• How arrays do you have setup?
• How many LUNs do you have setup?
• How many LUNs do you have on each array?

[Techinfo-BPM]

Hi Khue,

How arrays do you have setup: I have 8 arrays

How many LUNs do you have setup:  8 Lun

How many LUNs do you have on each array: I have one LUN on each array.

Do you have 12, 1 terabyte drives available or do you have 12 terabyte drives with one allocated to a hotspare: I have 12 , 1 terabyte with no hotspare.

Thank you

[CraigV]

Hi Khue,

I think he MIGHT have 8 x RAID5 sets of 3 disks each.
This would also impact his throughput...more disks = more IOPS.

Anyways, look forward to seeing how you will help here. I have lots of experience with HP SANs...nothing with IBM!

Laters...

[Khue]

Thanks Tech. Few more questions for you.

• 8 arrays x 3 drives each array means you have 24 disks. Is this correct?
• If this is correct, the DS3300 only has 12 drive spaces, where are the other 12 drive spaces?

Those questions aside, I think we have found your issue! I think you need to rethink your array/LUN configuration a bit. I know this will be a pain especially if you have already cut your production system over to the SAN, but in the long run if you want to see a good return for your SAN investment it really needs to be done.

Craig keyed in on something and if throughput is a must "more disks = more IOPS" is a staple statement.

[Techinfo-BPM]

HI, Khue,

    Yes I have 24 disk
   The DS3300 have 12 disk and I have a EXP3300 with 12 more HD for the 24 TB of 1 TB hard drive.

What is the best solution to get a month of backup on the SAN?

Thank you.

[Khue]

Hmmm, that is a very long and in depth answer depending upon your environment and a month backup could mean a couple things. What is the volume of data that is held within a month?

[Techinfo-BPM]

Hi,
   We have around 600 GB of data backup per day.
  I want to keep one full backup per day during a month.
  I hope that help you understand the situation.

Thank you

[Khue]

Hmm, 600 gigs a day is a lot of data to keep on hand for an entire month without deduplication. Furthermore, with that volume of data it seems like you should have a bigger solution then a DS3300 with an EXP3000 and a single control module. By my count, 600 gigs a day at a rate of 1024mb/min would take roughly 10 hours a day. To be quite honest with you 1024 sustained mb/min is kind of hard to deliver if the data is regular file based. If all your data was in like, a large block format, then maybe it would be possible to see speeds of 1024mb/min and upward.

Moving on. So 600 * 31 is about 18 terabytes a month. A very basic RAID 5 is about 7 to 8 disks. In your environment you could create (4) 6 disk RAID 5 arrays. If you can span sub systems (which I am sure you can do, but I don't recommend it) then you could do (3) 8 disk RAID 5 arrays.

So now you will have 4, 4.7 terabyte RAID arrays. Your i/o should be higher per array. now. I would create a number of LUNs on each array between 2 and 4. Make them whatever size you want. I would then try to devise some backup schedule to write to only a SINGLE LUN a day and make that media non overwriteable but appendable for 30 days or so. I think your goals should be:

• Write to as many disks as possible
• Write to a single array at a time (since you only have one controller)
• Write to a single LUN at a time

That is a 30,000 foot view of course. Unfortunately I think you made the mistake of building out a SAN from a physical capacity standpoint only. You may also want to look at the amount of data you are backing up a night and see if you really need to store 600 gigs a night. If you are doing fulls you might want to switch to diffs or incrementals every night and do a full on the first or the last day of the month. 

Anybody else have suggestions? I am by no means an expert at any of this. I just know what I have dealt with in the past.

[teiva-boy]

I really think this is a case for disk reconfiguration, and lack of good planning on the creation of the disk arrays.

Not to mention if Windows 2003, you'll want to make sure you correctly aligned your partitions using DISKPART and NOT the disk manager.
You will have made sure to use use large block sizes on the array and formatting on the arrays used for the backup data.

Please take a look at this, I think this applies to you:
http://vmtoday.com/2009/06/ibm-ds3300-iscsi-write-performance-solved/

Because it's iSCSI, you'll want to use Jumbo Frames for max performance too.

You'll also want to create multiple simultaneous jobs to disk so you can leverage MPIO.  A singular job will ONLY use one link even if you were backing up 50 servers.


Ultimately, contact IBM, and ask them what the optimum disk/array/LUN configuration should be.  12-1TB SATA drives isn't it IMO, all that config would be good for is archiving data or flat office files off a NAS.