Solved: Tape Drives not being used

Mike_3G · ‎01-05-2012

NBU 7.0.1, running on RH LInux with a Quantum i2000 robot - 12 drives (8 LTO3 and 4 LTO4).

I've searched Google but couldn't find the answer to my question. I've only got a few months experience with this setup. When I look at the robot, only 3 drives are being used and I have jobs queued up that say "drives are in use". The policies are set to "Use multiple streams" and the Storage Units are set up to allow 8 "Maximum concurrent write drives". So what have I missed?

Also there is 1 Master server and 2 Media servers.

Mike

Mark_Solutions · ‎01-06-2012

Well - 208 only has 4 drives in it and you say only three are being used ......

That is because the others are all DOWN

You need to UP the drives (and find out why there were down to get this sorted out)

View solution in original post

Marianne · ‎01-05-2012

You should have multiple Storage Units in this setup, right?

Please post output of the following:

bpstulist -U

How are Storage Units assigned to policies - specific STU's or STU groups?

Are jobs queueing for one specific STU or more than one STU?

Handy NetBackup Links

Mike_3G · ‎01-05-2012

yes, I have multiple storage units. I'm focused on the NDMP-based storage units especially. Specific STUs are assigned to policies - I am not using STU groups at the moment (willing to change, however). NDMP jobs are queued for these 2 specific STUs.

master-hcart3-robot-tld-0-208 and
master-hcart3-robot-tld-0-208_copy

 [root@master ~]# bpstulist -U


Label:                master-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      master
Number of Drives:     5
On Demand Only:       no
Max MPX/drive:        6
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB

Label:                media03-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media03
Number of Drives:     2
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media04-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media04
Number of Drives:     2
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media05-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media05
Number of Drives:     2
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media01-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media01
Number of Drives:     7
On Demand Only:       no
Max MPX/drive:        10
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB

Label:                media06-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media06
Number of Drives:     2
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB

Label:                media02-hcart3-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media02
Number of Drives:     1
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB

Label:                master-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      master
Number of Drives:     4
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media03-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media03
Number of Drives:     4
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media04-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media04
Number of Drives:     4
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media05-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media05
Number of Drives:     4
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media06-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media06
Number of Drives:     4
On Demand Only:       yes
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media01-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media01
Number of Drives:     4
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                media02-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      media02
Number of Drives:     4
On Demand Only:       no
Max MPX/drive:        4
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB

Label:                master-hcart3-robot-tld-0-208
Storage Unit Type:    NDMP
Host Connection:      master
Number of Drives:     8
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB
NDMP attach host:     192.168.123.75

Label:                master-hcart-robot-tld-0-208
Storage Unit Type:    NDMP
Host Connection:      master
Number of Drives:     2
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart - 1/2 Inch Cartridge
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048576 MB
NDMP attach host:     192.168.123.75

Label:                master-hcart3-robot-tld-0-208_copy
Storage Unit Type:    NDMP
Host Connection:      master
Number of Drives:     8
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB
NDMP attach host:     192.168.123.75

Label:                master-hcart3-robot-tld-0-208_copy2
Storage Unit Type:    NDMP
Host Connection:      master
Number of Drives:     8
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB
NDMP attach host:     192.168.123.75

Label:                media01-hcart3-robot-tld-0-208
Storage Unit Type:    NDMP
Host Connection:      media01
Number of Drives:     4
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart3 - 1/2 Inch Cartridge 3
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB
NDMP attach host:     192.168.123.75

Label:                backup_to_disk
Storage Unit Type:    Disk
Storage Unit Subtype: Basic (1)
Host Connection:      master
Concurrent Jobs:      15
On Demand Only:       yes
Max MPX:              1
Path:                 "/usr/openv/netbackup/backup2disk"
Max Fragment Size:    524287 MB
Stage data:           yes
Block Sharing:        no
High Water Mark:      85
Low Water Mark:       60
Ok On Root:           yes

J_H_Is_gone · ‎01-05-2012

in NB there are a number of things that can cause a job to queue up.

The easy way for us to help is to find out why they are queued up.

If you go to the gui and

either look at the column that says "State Details" of the job that is queue

or look at the details of the job that is queued

it will tell you why it is waiting -

With that info we can get closer to why are are not using all the drives.

Marianne · ‎01-05-2012

What is the reason for having 2 STU's for the same device parameters?

master-hcart3-robot-tld-0-208 and
master-hcart3-robot-tld-0-208_copy

The only difference is the number of drives. Maybe this is 'confusing' NBU?

To really see what is in use, assigned and allocated, compare output of all of the following on the master:

vmoprcmd -d

vmdareq

nbrbutil -dump (check the bottom section of output with allocation info).

Handy NetBackup Links

Mark_Solutions · ‎01-06-2012

In addition to the helps and requests above can i ask if all of your Media Servers have had the Shared Storage Option license registered on them?

Run /usr/openv/netbackup/bin/admincmd/get_license_key on each one to make sure it has been added.

Hope this helps

Mike_3G · ‎01-06-2012

@J.Hinchcliffe - Here is the last bit of the Detailed Status for one of the queued jobs:

Jan 5, 2012 1:34:17 PM - awaiting resource master-hcart3-robot-tld-0-208. Waiting for resources.

Reason: Drives are in use, Media server: master,

Robot Type(Number): TLD(0), Media ID: N/A, Drive Name: N/A,

Volume Pool: media02, Storage Unit: master-hcart3-robot-tld-0-208, Drive Scan Host: N/A,

Disk Pool: N/A, Disk Volume: N/A

@Marianne - those 2 STUs were present when I inherited the job. I didn't realize they could conflict. Here is output of your 1st suggestion. I'm reviewing the other data.

[root@master ~]# vmoprcmd -d

                                PENDING REQUESTS

<NONE>

DRIVE STATUS

Drv Type Control User Label RecMID ExtMID Ready Wr.Enbl. ReqId

0 hcart3 DOWN-TLD Yes 000201 000201 Yes Yes 0

1 hcart3 DOWN-TLD - No - 0

2 hcart3 TLD Yes 001266 001266 Yes Yes 0

3 hcart3 DOWN-TLD Yes 000284 000284 Yes Yes 0

4 hcart3 TLD Yes 000960 000960 Yes Yes 0

5 hcart3 DOWN-TLD - No - 0

6 hcart3 TLD - No - 0

7 hcart3 TLD - No - 0

8 hcart TLD Yes 004057 004057 Yes Yes 0

9 hcart TLD Yes 004034 004034 Yes Yes 0

10 hcart TLD - No - 0

11 hcart TLD - No - 0

ADDITIONAL DRIVE STATUS

Drv DriveName Shared Assigned Comment

0 IBM.ULTRIUM-TD3.005 Yes media01.natio

1 IBM.ULTRIUM-TD3.006 Yes -

2 IBM.ULTRIUM-TD3.007 Yes master.natio

3 IBM.ULTRIUM-TD3.004 Yes media01.natio

4 IBM.ULTRIUM-TD3.000 Yes media01.natio

5 IBM.ULTRIUM-TD3.001 Yes -

6 IBM.ULTRIUM-TD3.002 Yes -

7 IBM.ULTRIUM-TD3.003 Yes -

8 IBM.ULTRIUM-TD4.002 Yes media02.natio

9 IBM.ULTRIUM-TD4.003 Yes media02.natio

10 IBM.ULTRIUM-TD4.000 Yes -

11 IBM.ULTRIUM-TD4.001 Yes -

@Mark_Solutions - licenses for Shared Storage Option are present on master, media01, media02 and I'll check the others but I'm sure they're OK.

Mark_Solutions · ‎01-06-2012

Well - 208 only has 4 drives in it and you say only three are being used ......

That is because the others are all DOWN

You need to UP the drives (and find out why there were down to get this sorted out)

Mike_3G · ‎01-06-2012

@Mark_Solutions -- I've reset the drives to UP and enabled the necessary paths. I'm checking the Scalar i2000 to see what errors are on the drives - one drive at least needs cleaning, but at least all of the drives are being utilized.

I would still like to understand how to correct the config of STUs.

Mark_Solutions · ‎01-06-2012

Glad this has helped

In general each media server will have a single storage unit that relates to the number of drives of the same denisty in a library

So in your case, assuming all drives are shared, then each media server should have hcart3 storage unit with 8 drive and a hcart storage unit with 4 drives

You only tend to have additional ones with less drives if you want to restrict a particular media sevrer or policy from using all of the drive, in which case you can take a copy of it and reduce its drive count.

Do check all of your policies and schedules (and disk staging / SLPs if you use them) before deleting any to make sure that they are not specified somewhere which would revert the policy to fail or use "any available"

Hope this helps

Mike_3G · ‎01-06-2012

I think I have a problem setting up the STUs correctly, but I'll move that to another thread.

VOX

Tape Drives not being used