Solved: Max drives utilization in NetBackup backup policy

satish_munduru · ‎03-18-2012

Hi,

I am using windows2008 server as my master server and have 16 LTO5 drives. when i fire a backup then it is using only 3 drives only but there are 20 streams. when i fire another backup then it will wait until the previous backup is completed after the backup is completed then only the next backup is starts with another three drives only.

Why it is using only three drives if we are having 16 drives.

Thanks in advance,

satish_munduru · ‎03-31-2012

i just cleaned all drives and refired the backup and had 31 tapes in library.

then master server used all the drives.

Thanks supporting.

View solution in original post

Marianne · ‎03-19-2012

Hopefully you have NOT configured 16 drives on ONE server?

I have never seen any media server being able to handle/stream that amount of tape drives.

You should divide the tape drives among 4 - 5 media servers.

To understand why only 3 drives are being used, we need to see Storage Unit config as well as MPX settings in policies.

STU config:
<install-path>\veritas\netbackup\bin\admincmd\bpstulist -U

Policy config (do this for a couple of polcies that normally run together):
<install-path>\veritas\netbackup\bin\admincmd\bppllist <policy-name> -U

Also check Max Jobs per Client in Master Server Host Properties -> Global Attributes. The default is 1.

Handy NetBackup Links

satish_munduru · ‎03-19-2012

Thanks for reply

here what ypu need...

C:\Program Files\Veritas\NetBackup\bin\admincmd>bpstulist -U

Label: LTO-5

Storage Unit Type: Media Manager

Host Connection: ******************

Number of Drives: 15

On Demand Only: yes

Max MPX/drive: 1

Density: hcart2 - 1/2 Inch Cartridge

Robot Type/Number: TLD / 0

Max Fragment Size: 1048576 MB

Label: LTO-4

Storage Unit Type: Media Manager

Host Connection: ******************

Number of Drives: 2

On Demand Only: yes

Max MPX/drive: 1

Density: hcart - 1/2 Inch Cartridge

Robot Type/Number: TLD / 1

Max Fragment Size: 1048576 MB

Label: 3592

Storage Unit Type: Media Manager

Host Connection: ******************

Number of Drives: 1

On Demand Only: yes

Max MPX/drive: 1

Density: hcart2 - 1/2 Inch Cartridge

Robot Type/Number: TLD / 2

Max Fragment Size: 1048576 MB

Label: NDMP-enpfiler1

Storage Unit Type: NDMP

Host Connection: ******************

Number of Drives: 15

On Demand Only: yes

Max MPX/drive: 1

Density: hcart2 - 1/2 Inch Cartridge

Robot Type/Number: TLD / 0

Max Fragment Size: 1048576 MB

NDMP attach host: enpfiler1

Label: NDMP-enpfiler2

Storage Unit Type: NDMP

Host Connection: ******************

Number of Drives: 15

On Demand Only: no

Max MPX/drive: 1

Density: hcart2 - 1/2 Inch Cartridge

Robot Type/Number: TLD / 0

Max Fragment Size: 1048576 MB

NDMP attach host: enpfiler2

Label: catalog_Backup

Storage Unit Type: Disk

Storage Unit Subtype: Basic (1)

Host Connection: ******************

Concurrent Jobs: 1

On Demand Only: yes

Max MPX: 1

Path: "D:\Catalog_Backup"

Max Fragment Size: 524288 MB

Stage data: no

Block Sharing: no

High Water Mark: 98

Low Water Mark: 80

Ok On Root: yes

C:\Program Files\Veritas\NetBackup\bin\admincmd>bppllist ENPFILER1 -U

-----------------------------------------------------------

Policy Name: ENPFILER1

Policy Type: NDMP

Active: yes

Effective date: 08/03/2011 11:10:51

Mult. Data Streams: yes

Client Encrypt: no

Checkpoint: no

Policy Priority: 0

Max Jobs/Policy: Unlimited

Disaster Recovery: 0

Collect BMR info: no

Residence: NDMP-enpfiler1

Volume Pool: Full-LTO5

Server Group: *ANY*

Keyword: (none specified)

Data Classification: -

Residence is Storage Lifecycle Policy: no

Application Discovery: no

Discovery Lifetime: 28800 seconds

Granular Restore Info: no

Ignore Client Direct: no

HW/OS/Client: NDMP NDMP enpfiler1

Include: /vol/****

/vol/****

/vol/*****

/vol/****

/vol/*****

/vol/****

/vol/*************

/vol/*****************

/vol/*************************

/vol/**********

/vol/****************

/vol/***************

/vol/****

/vol/*************

/vol/**************

/vol/***********************

/vol/**************************

Schedule: Full

Type: Full Backup

Maximum MPX: 1

Synthetic: 0

PFI Recovery: 0

Retention Level: 5 (3 months)

Number Copies: 1

Fail on Error: 0

Residence: NDMP-enpfiler1

Volume Pool: Full-LTO5

Server Group: (same as specified for policy)

Calendar sched: Enabled

Saturday, Week 5

Residence is Storage Lifecycle Policy: 0

Daily Windows:

Saturday 23:00:00 --> Sunday 16:00:00

Schedule: Weekly

Type: Cumulative Incremental Backup

Maximum MPX: 1

Synthetic: 0

PFI Recovery: 0

Retention Level: 1 (2 weeks)

Number Copies: 1

Fail on Error: 0

Residence: NDMP-enpfiler1

Volume Pool: Weekly-LTO5

Server Group: (same as specified for policy)

Calendar sched: Enabled

Wednesday, Week 1

Wednesday, Week 2

Wednesday, Week 3

Wednesday, Week 4

Wednesday, Week 5

Residence is Storage Lifecycle Policy: 0

Daily Windows:

Wednesday 22:00:00 --> Thursday 08:00:00

Yasuhisa_Ishika · ‎03-19-2012

Check number of drives configured on enpfiler1.
I suppose that only 3 drives are configured on this filer. Please post output of "tpconfig -dl"

Check number of available tapes in volume pool.
I suppose that only 3 tapes are available in each pool.

Marianne · ‎03-19-2012

So, problem seems to be with STU NDMP-enpfiler1 backing up policy ENPFILER1, right?

Drives in robot shared between enpfiler1, enpfiler2 and master?
All 15(16) drives configured on all servers at OS level in NBU?

How many drives in use on enpfiler2 and master?

What is Max Jobs per Client set to?

As per Yasuhisa's post - how many tapes available in the pool?

Handy NetBackup Links

satish_munduru · ‎03-19-2012

Here is the drive configuration. we configured 15 drives to both filers.

tapes are 36 available.

C:\Program Files\Veritas\Volmgr\bin>tpconfig.exe -dl

Currently defined drives and robots are:

Drive Name IBM.ULT3580-TD5.000

Index 0

NonRewindDrivePath nrst8a

NDMP client enpfiler1

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=1

Serial Number 00078A1ADE

Drive Name IBM.ULT3580-TD5.000

Index 0

NonRewindDrivePath nrst10a

NDMP client enpfiler2

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=1

Serial Number 00078A1ADE

Drive Name IBM.ULT3580-TD5.000

Index 0

SCSI coordinates {2,0,0,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=1

Serial Number 00078A1ADE

Drive Name IBM.ULT3580-TD5.001

Index 1

NonRewindDrivePath nrst14a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=2

Serial Number 00078A1B08

Drive Name IBM.ULT3580-TD5.001

Index 1

NonRewindDrivePath nrst12a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=2

Serial Number 00078A1B08

Drive Name IBM.ULT3580-TD5.001

Index 1

SCSI coordinates {2,0,1,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=2

Serial Number 00078A1B08

Drive Name IBM.ULT3580-TD5.002

Index 2

NonRewindDrivePath nrst7a

NDMP client enpfiler1

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=11

Serial Number 00078A1B39

Drive Name IBM.ULT3580-TD5.002

Index 2

NonRewindDrivePath nrst3a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=11

Serial Number 00078A1B39

Drive Name IBM.ULT3580-TD5.002

Index 2

SCSI coordinates {2,0,10,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=11

Serial Number 00078A1B39

Drive Name IBM.ULT3580-TD5.003

Index 3

NonRewindDrivePath nrst3a

NDMP client enpfiler1

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=12

Serial Number 00078A1B37

Drive Name IBM.ULT3580-TD5.003

Index 3

NonRewindDrivePath nrst11a

NDMP client enpfiler2

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=12

Serial Number 00078A1B37

Drive Name IBM.ULT3580-TD5.003

Index 3

SCSI coordinates {2,0,11,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=12

Serial Number 00078A1B37

Drive Name IBM.ULT3580-TD5.004

Index 4

NonRewindDrivePath nrst2a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=14

Serial Number 00078A1AF1

Drive Name IBM.ULT3580-TD5.004

Index 4

NonRewindDrivePath nrst2a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=14

Serial Number 00078A1AF1

Drive Name IBM.ULT3580-TD5.004

Index 4

SCSI coordinates {2,0,12,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=14

Serial Number 00078A1AF1

Drive Name IBM.ULT3580-TD5.005

Index 5

NonRewindDrivePath nrst1a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=15

Serial Number 00078A1AF0

Drive Name IBM.ULT3580-TD5.005

Index 5

NonRewindDrivePath nrst0a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=15

Serial Number 00078A1AF0

Drive Name IBM.ULT3580-TD5.005

Index 5

SCSI coordinates {2,0,13,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=15

Serial Number 00078A1AF0

Drive Name IBM.ULT3580-TD5.006

Index 6

NonRewindDrivePath nrst4a

NDMP client enpfiler1

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=16

Serial Number 00078A1AEE

Drive Name IBM.ULT3580-TD5.006

Index 6

NonRewindDrivePath nrst1a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=16

Serial Number 00078A1AEE

Drive Name IBM.ULT3580-TD5.006

Index 6

SCSI coordinates {2,0,14,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=16

Serial Number 00078A1AEE

Drive Name IBM.ULT3580-TD5.007

Index 7

NonRewindDrivePath nrst10a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=4

Serial Number 00078A1ADC

Drive Name IBM.ULT3580-TD5.007

Index 7

NonRewindDrivePath nrst14a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=4

Serial Number 00078A1ADC

Drive Name IBM.ULT3580-TD5.007

Index 7

SCSI coordinates {2,0,2,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=4

Serial Number 00078A1ADC

Drive Name IBM.ULT3580-TD5.008

Index 8

NonRewindDrivePath nrst11a

NDMP client enpfiler1

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=3

Serial Number 00078A1AD6

Drive Name IBM.ULT3580-TD5.008

Index 8

NonRewindDrivePath nrst13a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=3

Serial Number 00078A1AD6

Drive Name IBM.ULT3580-TD5.008

Index 8

SCSI coordinates {2,0,3,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=3

Serial Number 00078A1AD6

Drive Name IBM.ULT3580-TD5.009

Index 9

NonRewindDrivePath nrst6a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=5

Serial Number 00078A1B09

Drive Name IBM.ULT3580-TD5.009

Index 9

NonRewindDrivePath nrst8a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=5

Serial Number 00078A1B09

Drive Name IBM.ULT3580-TD5.009

Index 9

SCSI coordinates {2,0,4,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=5

Serial Number 00078A1B09

Drive Name IBM.ULT3580-TD5.010

Index 10

NonRewindDrivePath nrst13a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=6

Serial Number 00078A1AD9

Drive Name IBM.ULT3580-TD5.010

Index 10

NonRewindDrivePath nrst9a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=6

Serial Number 00078A1AD9

Drive Name IBM.ULT3580-TD5.010

Index 10

SCSI coordinates {2,0,5,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=6

Serial Number 00078A1AD9

Drive Name IBM.ULT3580-TD5.011

Index 11

NonRewindDrivePath nrst12a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=7

Serial Number 00078A1B1C

Drive Name IBM.ULT3580-TD5.011

Index 11

NonRewindDrivePath nrst6a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=7

Serial Number 00078A1B1C

Drive Name IBM.ULT3580-TD5.011

Index 11

SCSI coordinates {2,0,6,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=7

Serial Number 00078A1B1C

Drive Name IBM.ULT3580-TD5.012

Index 12

NonRewindDrivePath nrst9a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=8

Serial Number 00078A1B07

Drive Name IBM.ULT3580-TD5.012

Index 12

NonRewindDrivePath nrst4a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=8

Serial Number 00078A1B07

Drive Name IBM.ULT3580-TD5.012

Index 12

SCSI coordinates {2,0,7,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=8

Serial Number 00078A1B07

Drive Name IBM.ULT3580-TD5.013

Index 13

NonRewindDrivePath nrst5a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=9

Serial Number 00078A1B0B

Drive Name IBM.ULT3580-TD5.013

Index 13

NonRewindDrivePath nrst7a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=9

Serial Number 00078A1B0B

Drive Name IBM.ULT3580-TD5.013

Index 13

SCSI coordinates {2,0,8,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=9

Serial Number 00078A1B0B

Drive Name IBM.ULT3580-TD5.014

Index 14

NonRewindDrivePath nrst0a

NDMP client enpfiler1

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=10

Serial Number 00078A1B0A

Drive Name IBM.ULT3580-TD5.014

Index 14

NonRewindDrivePath nrst5a

NDMP client enpfiler2

Type hcart2

Status DOWN

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=10

Serial Number 00078A1B0A

Drive Name IBM.ULT3580-TD5.014

Index 14

SCSI coordinates {2,0,9,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(0) Definition DRIVE=10

Serial Number 00078A1B0A

Drive Name IBM.ULT3580-TD4.000

Index 15

SCSI coordinates {3,0,0,0}

Type hcart

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(1) Definition DRIVE=1

Serial Number 00078C0C97

Drive Name IBM.ULT3580-TD4.001

Index 16

SCSI coordinates {3,0,1,0}

Type hcart

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(1) Definition DRIVE=2

Serial Number 00078C0C9A

Drive Name IBM.03592E06.000

Index 17

SCSI coordinates {3,0,2,0}

Type hcart2

Status UP

SCSI Protection SR (Global)

Shared Access No

TLD(2) Definition DRIVE=1

Serial Number 0000078A5C88

Currently defined robotics are:

TLD(0) SCSI coordinates = {2,0,0,1}

TLD(1) SCSI coordinates = {3,0,0,1}

TLD(2) SCSI coordinates = {3,0,2,1}

EMM Server = ***********************

satish_munduru · ‎03-19-2012

there are 15 drives are shared between filer1 and filer2.

Max Jobs per Client = 10 and there are 36 tapes available in the pool.

Marianne · ‎03-20-2012

Shared Access No

You have the same drives configured on the Master and 2 filers without SSO?

You also have quite a number of drives in DOWN state.

Please also post output of 'vmoprcmd -d' on the master.

This will list drive status on all servers in easier to read format and also show which drives are in use (tape mounted).

Handy NetBackup Links

satish_munduru · ‎03-20-2012

C:\Program Files\Veritas\Volmgr\bin>vmoprcmd.exe -d

PENDING REQUESTS

<NONE>

DRIVE STATUS

Drv Type Control User Label RecMID ExtMID Ready Wr.Enbl. Req

0 hcart2 TLD Yes 0245L5 0245L5 Yes Yes 0

1 hcart2 TLD - No - 0

1 hcart2 DOWN-TLD - No - 0

2 hcart2 TLD - No - 0

2 hcart2 DOWN-TLD - No - 0

2 hcart2 TLD - No - 0

3 hcart2 TLD Yes 0233L5 0233L5 Yes Yes 0

4 hcart2 TLD - No - 0

4 hcart2 DOWN-TLD - No - 0

5 hcart2 TLD - No - 0

5 hcart2 DOWN-TLD - No - 0

6 hcart2 TLD - No - 0

6 hcart2 DOWN-TLD - No - 0

6 hcart2 TLD - No - 0

7 hcart2 TLD - No - 0

7 hcart2 DOWN-TLD - No - 0

8 hcart2 TLD - No - 0

8 hcart2 DOWN-TLD - No - 0

8 hcart2 TLD - No - 0

9 hcart2 TLD - No - 0

9 hcart2 DOWN-TLD - No - 0

10 hcart2 TLD - No - 0

10 hcart2 DOWN-TLD - No - 0

11 hcart2 TLD - No - 0

11 hcart2 DOWN-TLD - No - 0

12 hcart2 TLD - No - 0

12 hcart2 DOWN-TLD - No - 0

13 hcart2 TLD - No - 0

13 hcart2 DOWN-TLD - No - 0

14 hcart2 TLD - No - 0

14 hcart2 DOWN-TLD - No - 0

15 hcart TLD - No - 0

16 hcart TLD - No - 0

17 hcart2 TLD - No - 0

ADDITIONAL DRIVE STATUS

Drv DriveName Shared Assigned Comment

0 IBM.ULT3580-TD5.000 No *******************.

1 IBM.ULT3580-TD5.001 No -

2 IBM.ULT3580-TD5.002 No -

3 IBM.ULT3580-TD5.003 No *******************.

4 IBM.ULT3580-TD5.004 No -

5 IBM.ULT3580-TD5.005 No -

6 IBM.ULT3580-TD5.006 No -

7 IBM.ULT3580-TD5.007 No -

8 IBM.ULT3580-TD5.008 No -

9 IBM.ULT3580-TD5.009 No -

10 IBM.ULT3580-TD5.010 No -

11 IBM.ULT3580-TD5.011 No -

12 IBM.ULT3580-TD5.012 No -

13 IBM.ULT3580-TD5.013 No -

14 IBM.ULT3580-TD5.014 No -

15 IBM.ULT3580-TD4.000 No -

16 IBM.ULT3580-TD4.001 No -

17 IBM.03592E06.000 No -

C:\Program Files\Veritas\Volmgr\bin>

Marianne · ‎03-20-2012

You need to deal with all the DOWN drives. You need to find WHY they are being DOWN'ed.
Check Event Viewer System and Application logs.

You would normally need VERBOSE entry in vm.conf on Windows server for additional Media Manager logging to Event Viewer. (Device Manager needs to be restarted after adding the entry.)

I cannot see how it all can work together without SSO....

Handy NetBackup Links

satish_munduru · ‎03-20-2012

Not DOWN.

when there are any backup is running those are in active and remaining all are in DOWN state.

Marianne · ‎03-20-2012

I see DOWN drives in tpconfig and vmoprcmd output. That will explain drives not being utilized.

Again - I cannot see how sharing drives can work correctly without SSO (Shared Storage Option).

Handy NetBackup Links

satish_munduru · ‎03-31-2012

i just cleaned all drives and refired the backup and had 31 tapes in library.

then master server used all the drives.

Thanks supporting.

VOX

Max drives utilization in NetBackup backup policy