cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Veritas.com
Support

NDMP: multistreaming & parallel running policies

Jim-m
Level 5

‎09-10-2008 10:58 PM

Hi

 

NDMP backups to an external server will not multistream as per expections. 

But what happense when you run two polices at the same netapp filer at the same time?  

Does one of the job queue or both run at the same time?

 

Assume same netapp filer the only thing that is different between the policies is the storage objects/volumes

Message Edited by Jim-m on 09-10-2008 11:00 PM
0 Kudos
8 REPLIES 8

Andy_Welburn
Level 6

‎09-11-2008 12:50 AM

If there are tape drives available in the Storage Unit & the settings are correct on the policy, the jobs should multi-stream (but will not multiplex).

 

We backup 2 NetApp filers through 3 different policies with streams backing up snapshot areas of selected volumes/qtrees. One policy has 10 streams (separated by NEW_STREAM directive) limited to 2 jobs/policy another has 8 limited to 2 & the last 3 limited to 3 jobs/policy. All use the same Storage Unit consisting of 3 drives. They all start at approximately the same time on a Monday morning & continue until Wednesday. Three streams (from any combination of policy) utilise the 3 tape drives & the rest queue expectantly until a drive becomes available.

 

On top of that, at each month end there are a further 3 separate policies for one of these filers (one stream each) & they all run happily together.

0 Kudos

jim_dalton
Level 6

‎09-11-2008 07:18 AM

To repeat...

Multistream - yes ; multiplex streams to same media - no. Shame really as I could do with mpx to get things to go a little faster.

For multistream to work you need multiple paths to backup and you need multiple drives available in the storage unit, one per stream, and to enable multistream for the policy.

 

Anyone know if mpx for ndmp is ever likely to be a runner or if its a technical non-starter? Seems to me like it ought to be do-able.

I ask since Im trying to get NDMP to run faster and have so far failed (8 M/sec!).(Infrastructure is:Netapp NAS with SSO to LTO4 over 2Gbit fibre, network is 1gbit and the netb master is Sol10/T2000 with 16G mem and 32 virtual cores and can write at 250M/sec as fact).

TIA,Tim

0 Kudos

Andy_Welburn
Level 6

‎09-11-2008 07:42 AM

@Tim walton wrote:

Anyone know if mpx for ndmp is ever likely to be a runner or if its a technical non-starter? Seems to me like it ought to be do-able.


I was sure I read somewhere that it was coming - but that may just've been a caffeine overdose Smiley Wink

 

@Tim walton wrote:

I ask since Im trying to get NDMP to run faster and have so far failed (8 M/sec!).(Infrastructure is:Netapp NAS with SSO to LTO4 over 2Gbit fibre, network is 1gbit and the netb master is Sol10/T2000 with 16G mem and 32 virtual cores and can write at 250M/sec as fact).

TIA,Tim

Network limited? - NetApp/SSO/LTO3/Sol9 master (V240 - 6Gb mem & 2x 1280SparcIIIis)/4Gb F/C (between master/library/tapes/filers) running at +20M/sec, got upto +40M/sec once & that wasn't caffeine induced!)

 

 

0 Kudos

jim_dalton
Level 6

‎09-11-2008 08:58 AM

Andy,

Network?...well possibly (dont forget the data dont go over the network , only the metadata ). I think its something more related to lots of smallish files, but Im sort of inclined to think its the NAS thats throttling. Large files go faster. If I reduce the metadata (HIST=N) , it makes no difference to backup speed which suggests thats not the bottleneck.

 

I'm told the NAS is under no strain when backing up...perhaps it ought to be as the rest of the data chain are twiddling their thumbs waiting for the bytes to arrive!

 

Thanks,Tim

0 Kudos

Darren_Dunham
Level 6

‎09-11-2008 10:58 AM

@Andy Welburn wrote:
@Tim walton wrote:

Anyone know if mpx for ndmp is ever likely to be a runner or if its a technical non-starter? Seems to me like it ought to be do-able.


I was sure I read somewhere that it was coming - but that may just've been a caffeine overdose Smiley Wink


Multiplexing capability appears to have been a big requirement point in the NDMP 5 specifications documentation. But from the folks I've talked to, *no one* seems to be driving NDMP these days. Ideas and specs for 5 have been out for years. But I see no sign of action.

 

--

Darren

0 Kudos

Andy_Welburn
Level 6

‎09-12-2008 12:27 AM

@Tim walton wrote:

Andy,

Network?...well possibly (dont forget the data dont go over the network , only the metadata ).


Yeah wasn't thinking straight (holiday mode come early). I was just comparing like for like & actually looking at the F/C.    Smiley Happy

 

@Tim walton wrote:

I think its something more related to lots of smallish files


Quite possibly. Our throughput does vary a lot from stream to stream likely dependant on the data being backed up.

 

0 Kudos

Jim-m
Level 5

‎09-12-2008 06:20 AM

Thanks to you all  - it looks like NEW_STREAM is the way to go - testing this weekend
0 Kudos

Andy_Welburn
Level 6

‎09-12-2008 06:48 AM

Good luck, Jim-m!!

 

For info:

 

Originally on NB5.1MP6 our NDMP backups were multistreamed as follows:

/vol/vol1/directoryA

/vol/vol1/directoryB

/vol/vol1/directoryC

etc

& they did multi-stream if resources were available.

 

These were carried thru' when we upgraded to NB6.5GA. However, this new version introduced us to the parent (stream discovery) process (amongst other things). What we found out then was that, if we had to cancel one of the jobs for whatever reason ALL of the jobs currently running for that policy would also cancel - this didn't happen with 5.1.

 

Consequently, our NDMP backup selections had to be changed to:

/vol/vol1/directoryA

NEW_STREAM

/vol/vol1/directoryB

NEW_STREAM

/vol/vol1/directoryC

etc

which brought back this 'independence'.

 

0 Kudos