All drives are not getting utilized for backups

[imashu]

Hi,

My environment is Netbackup 6.5.6 and currently there were few changes done at the hardware library end as we were encountering few of the drives in AVR/down mode regularly. And all was well after that for 1-2 days.

But now we are seeing that only 3 of the total 8 drives are being used or backups. All the drives are perfectly fine and all are used but only any 3 drives at once.

We have checked the maximum concurrent drive setting and multilexing setting and have tried with all the values (high, low, same) but result the same.

Also, output from '/usr/openv/netbackup/bin/admincmd/bpstulist -U' is simliar to the GUI which means the changes we are doing at the storage unit from the gui are being replicated correctly.

I need immediate help please as my backups are not getting completed due to the drives not being utilized for their maximum capacity.

[Marianne · Accepted solution]

I assume that you have verified that all drives UP and TLD control?

If so, it is possible that device allocations got stuck/orphaned in NBU Resource Broker.

The quikest way to release/reset all allocations, is to release ALL allacations, but that would kill backups that are currently active:
nbrbutil -resetAll

To reset individual orpaned allocations, run 'nbrbutil -dump'.
compare MDS allocations at the bottom of the screen with actual device usage in Device Monitor or vmoprcmd output.
Release incorrect allocations with:
nbrbutil -releaseMDS <mdsAlocationKey>

PS: Are you aware of the fact that NBU 6.5 reached EOSL on 3 October?

[imashu]

Hi Marianne,

Many Thanks.

I have already tried removing the MDS allocations earlier by running the command on our master server which is the robotic controlled host.

Do i need to run this on on all the media servers?

I have re-ran the below commands;

1.) nbrbutil -resetAll

2.) nbrbutil -dump

Output :

nbrbutil -dump

Allocation Requests
(AllocationRequestSeq )

Allocations
(AllocationSeq )

MDS allocations in EMM:

Still the same issue. Do you think it should be related to hardware library side.

I have also posted in the one of the forum for NBU upgrade which we are planning to do asap.

[Mark_Solutions]

Are jobs actually getting queued - if so what does the detailed status of those jobs say?

I am thinking that if you have 8 drives and a MPX setting of say 6 then up to 18 jobs can run and use only 3 drives

Also, since the hardware changes are all the drives of the same denisty (HCART2 etc.) I am guessing they are as you say all drives do get used.

Give us a few outputs (storage unit lists etc) or screen shots so that we can see all of the details of everything to visualise it better

Thanks

[imashu]

Thanks all for your useful comments.

Marianne, Special Thanks to you.

After running the below commands again on the master server;

1.) nbrbutil -resetAll

2.) nbrbutil -dump

Restarted netbackup services on the master as well as on each media servers.

I ran the below command;

/usr/openv/netbackup/bin/admincmd/nbemmcmd -deletealldevices -allrecords

Rescanned the storage units in through netbackup GUI and ran the backups and they are running fine utilizing all the drives.

Marianne, Many Thanks again!!

[Marianne]

Thanks for the solution!

Seems you have solved your own problem by deleting all devices and re-running device config.

Maybe Mark was correct in his thinking that parameters have changed during 'the few changes'?