reservation conflicts

Abhishek_Sharma · ‎07-25-2012

Hi All,

We are getting a lot of reservation conflicts messages in our OS logs .We are running Netbackup 7.1.0.4 and have 6 media servers sharing tape drives b/w them using SSO .3 media servers are running Suse Linux 11 and 3 are running Windows 2008 R2 .But there are a lot of reservation conflicts seen on all media servers in OS logs .For eg: see below

Jul 23 17:27:26 PNIOYNBMDS01 kernel: [345273.747021] st 3:0:4:0: reservation conflict

Jul 23 17:27:27 PNIOYNBMDS01 kernel: [345274.747541] st 3:0:4:0: reservation conflict

Jul 23 17:27:30 PNIOYNBMDS01 kernel: [345278.040974] st 3:0:4:0: reservation conflict

Jul 23 17:27:50 PNIOYNBMDS01 avrd[22683]: Reservation Conflict status from SL3K--Drive001 (device 7)

Jul 23 17:27:50 PNIOYNBMDS01 kernel: [345297.832748] st 3:0:4:0: reservation conflict

Jul 23 17:28:05 PNIOYNBMDS01 kernel: [345313.038963] st 3:0:4:0: reservation conflict

Jul 23 17:28:20 PNIOYNBMDS01 kernel: [345328.215236] st 3:0:4:0: reservation conflict

Jul 23 17:28:35 PNIOYNBMDS01 kernel: [345343.389629] st 3:0:4:0: reservation conflict

Jul 23 17:28:50 PNIOYNBMDS01 kernel: [345358.390239] st 3:0:4:0: reservation conflict

Jul 20 14:34:45 PNIOYNBMDS02 avrd[7227]: Reservation Conflict status cleared from SL3K--Drive019 (device 4)

Jul 20 14:38:20 PNIOYNBMDS02 avrd[4156]: Reservation Conflict status cleared from SL3K--Drive017 (device 5)

Jul 20 16:06:17 PNIOYNBMDS02 avrd[4156]: Reservation Conflict status cleared from SL3K--Drive017 (device 5)

Jul 24 13:32:35 PNIOYNBMDS02 avrd[9808]: Reservation Conflict status cleared from SL3K--Drive003 (device 6)

Jul 23 16:45:02 PNIOYNBMDS01 kernel: [342730.175456] st 3:0:2:0: reservation conflict

Jul 23 16:45:14 PNIOYNBMDS01 kernel: [342742.625683] st 3:0:4:0: reservation conflict

Jul 23 16:45:14 PNIOYNBMDS01 kernel: [342742.626197] st 3:0:4:0: reservation conflict

Jul 23 16:45:14 PNIOYNBMDS01 kernel: [342742.626731] st 3:0:4:0: reservation conflict

Jul 23 16:45:32 PNIOYNBMDS01 avrd[22683]: Reservation Conflict status from SL3K--Drive002 (device 2)

Jul 23 16:45:32 PNIOYNBMDS01 kernel: [342760.176401] st 3:0:2:0: reservation conflict

This message will keep apperaing in logs until it is cleared by Netbackup .We are using SPC-2-SCSI reserver which is by default and are using a FC tape library .Persistent binding is already set on HBA level as per Unix team and we donnot see any missing paths after reboot ,

Please suggest how we can fix this

Abhishek

revarooo · ‎07-25-2012

Perhaps another host is accessing the drives?

Abhishek_Sharma · ‎07-25-2012

But ideally it should not show reservation conlficts and reservation conflicts getting cleared ....And if one host is using the drive another host should should see that it is already in reserved status and not poll for it

nibackup@PNIOYNBMDS01:~> vmdareq -M pnioynbmst01 -display |more

PG---hcart-000 - RESERVED on Wed Jul 25 12:15:03 2012

PNIPGNBMDS01 UP

pnipgnbvstg01 RESERVED SCAN_HOST UP

pnipgnbvstg02 UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-001 - RESERVED on Wed Jul 25 13:28:06 2012

PNIPGNBMDS01 RESERVED SCAN_HOST UP

pnipgnbvstg01 UP

pnipgnbvstg02 DOWN

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-002 - RESERVED on Wed Jul 25 10:53:02 2012

PNIPGNBMDS01 SCAN_HOST UP

pnipgnbvstg01 UP

pnipgnbvstg02 RESERVED UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-003 - AVAILABLE

PNIPGNBMDS01 UP

pnipgnbvstg02 SCAN_HOST UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-004 - RESERVED on Wed Jul 25 08:00:09 2012

PNIPGNBMDS01 SCAN_HOST UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 RESERVED UP

PG---hcart-005 - AVAILABLE

PNIPGNBMDS01 DOWN

pnipgnbvstg01 UP

pnipgnbvstg02 DOWN

pnipgdmzmds1 UP

pnipgnbmds02 SCAN_HOST UP

pnipgdmzmds2 UP

PG---hcart-006 - RESERVED on Wed Jul 25 13:00:05 2012

PNIPGNBMDS01 UP

pnipgnbvstg01 UP

pnipgnbvstg02 RESERVED SCAN_HOST UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-007 - AVAILABLE

PNIPGNBMDS01 UP

pnipgnbvstg01 SCAN_HOST UP

pnipgnbvstg02 UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-008 - RESERVED on Wed Jul 25 12:38:55 2012

PNIPGNBMDS01 SCAN_HOST UP

pnipgnbvstg01 UP

pnipgdmzmds1 UP

pnipgnbmds02 RESERVED UP

pnipgdmzmds2 UP

PG---hcart-009 - AVAILABLE

PNIPGNBMDS01 SCAN_HOST UP

pnipgnbvstg01 UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-010 - RESERVED on Wed Jul 25 13:35:09 2012

PNIPGNBMDS01 RESERVED DOWN

pnipgnbvstg01 UP

pnipgnbvstg02 DOWN

pnipgdmzmds1 SCAN_HOST UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-011 - RESERVED on Wed Jul 25 13:15:11 2012

PNIPGNBMDS01 SCAN_HOST UP

pnipgnbvstg01 UP

pnipgnbvstg02 UP

pnipgdmzmds1 RESERVED UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

PG---hcart-012 - RESERVED on Wed Jul 25 11:42:45 2012

PNIPGNBMDS01 UP

pnipgnbvstg01 RESERVED SCAN_HOST UP

pnipgnbvstg02 UP

pnipgdmzmds1 UP

pnipgnbmds02 UP

pnipgdmzmds2 UP

SL3K---Drive001---NDMPpniwaevs10 - AVAILABLE

PNIOYNBMDS01 SCAN_HOST UP

pnioynbvstg03 DOWN

SL3K---Drive002---NDMPpnievs5 - RESERVED on Wed Jul 25 12:55:39 2012

PNIOYNBMDS01 RESERVED SCAN_HOST UP

pnioynbvstg03 DOWN

SL3K---Drive004 - RESERVED on Wed Jul 25 13:32:09 2012

PNIOYNBMDS01 RESERVED UP

PNIOYNBMDS02 UP

pnioynbvstg01 UP

pnioydmzmds1 UP

pnioynbvstg02 DOWN

pnioydmzmds2 UP

pnioynbvstg03 SCAN_HOST UP

SL3K--Drive000 - RESERVED on Wed Jul 25 10:33:27 2012

PNIOYNBMDS01 UP

PNIOYNBMDS02 UP

pnioynbvstg01 SCAN_HOST UP

pnioydmzmds1 UP

pnioynbvstg02 RESERVED UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive001 - AVAILABLE

PNIOYNBMDS01 SCAN_HOST UP

PNIOYNBMDS02 UP

pnioynbvstg01 UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive002 - RESERVED on Wed Jul 25 09:45:08 2012

PNIOYNBMDS01 DOWN

PNIOYNBMDS02 RESERVED UP

pnioynbvstg01 SCAN_HOST UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive003 - RESERVED on Wed Jul 25 13:42:06 2012

PNIOYNBMDS01 SCAN_HOST UP

PNIOYNBMDS02 UP

pnioynbvstg01 RESERVED UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive005 - RESERVED on Wed Jul 25 00:45:04 2012

PNIOYNBMDS01 UP

PNIOYNBMDS02 UP

pnioynbvstg01 RESERVED SCAN_HOST UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive017 - AVAILABLE

PNIOYNBMDS01 SCAN_HOST UP

PNIOYNBMDS02 UP

pnioynbvstg01 UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive018 - RESERVED on Wed Jul 25 12:00:55 2012

PNIOYNBMDS01 UP

PNIOYNBMDS02 UP

pnioynbvstg01 UP

pnioydmzmds1 UP

pnioynbvstg02 RESERVED UP

pnioydmzmds2 UP

pnioynbvstg03 SCAN_HOST UP

SL3K--Drive019 - RESERVED on Wed Jul 25 11:22:33 2012

PNIOYNBMDS01 SCAN_HOST UP

PNIOYNBMDS02 UP

pnioynbvstg01 RESERVED UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

SL3K--Drive020 - RESERVED on Tue Jul 24 08:26:13 2012

PNIOYNBMDS01 RESERVED SCAN_HOST UP

PNIOYNBMDS02 UP

pnioynbvstg01 UP

pnioydmzmds1 UP

pnioynbvstg02 UP

pnioydmzmds2 UP

pnioynbvstg03 UP

Will_Restore · ‎07-25-2012

A properly configured NetBackup environment and properly configured tape devices should not receive a reservation conflict message from a tape drive

see this technote: Article URL http://www.symantec.com/docs/HOWTO32842

If the avrd process receives a reservation conflict message, it changes the status of the device to PEND. It also writes the following message in the system log:

Reservation Conflict status from DRIVENAME (device NUMBER)

When the conflict is resolved, the following message is written to the log:

Reservation Conflict status cleared from DRIVENAME (device NUMBER)

Will_Restore · ‎07-25-2012

shows some drives RESERVED for hours and some DOWN

You need to sort this out at host/OS level & rerun the NetBackup Device Configuration Wizard

Abhishek_Sharma · ‎07-26-2012

Hi

Thanks for the response . Can you tell me what we need to check/fix on OS level as persistent binding is already there on OS level .See below

We have got HP tape device drivers installed for HP tape drives on windows media servers Also i read a forum long time back on Symantec connect which suggested that HP tape drivers should be installed in non exclusive mode or Symantec tape drivers should be used

"Only one host can have access to one device on a SCSI bus. So sharing such devices requires specific mechanism provided by a special driver e.g. Veritas Netbackup tape device driver provide such functionality. Not sure, but Linux standard tape driver seems to reserve the device for himself, which only works for the first host connected to the device."

nibackup@PNIOYNBMDS01:/etc> cd /dev/tape/by-path/

nibackup@PNIOYNBMDS01:/dev/tape/by-path> ls -ltr

total 0

lrwxrwxrwx 1 root root 9 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfda4:0x0000000000000000-4:0:0:0st -> ../../st4

lrwxrwxrwx 1 root root 10 2012-07-19 17:33 pci-0000:06:00.1-fc-0x500104f000acfd9b:0x0000000000000000-3:0:3:0st-nst -> ../../nst2

lrwxrwxrwx 1 root root 9 2012-07-19 17:33 pci-0000:06:00.1-fc-0x500104f000acfd9b:0x0000000000000000-3:0:3:0st -> ../../st2

lrwxrwxrwx 1 root root 10 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfda4:0x0000000000000000-4:0:0:0st-nst -> ../../nst4

lrwxrwxrwx 1 root root 10 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfdaa:0x0000000000000000-4:0:2:0st-nst -> ../../nst6

lrwxrwxrwx 1 root root 9 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfdaa:0x0000000000000000-4:0:2:0st -> ../../st6

lrwxrwxrwx 1 root root 10 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfda7:0x0000000000000000-4:0:3:0st-nst -> ../../nst7

lrwxrwxrwx 1 root root 9 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfda7:0x0000000000000000-4:0:3:0st -> ../../st7

lrwxrwxrwx 1 root root 10 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfda1:0x0000000000000000-4:0:1:0st-nst -> ../../nst5

lrwxrwxrwx 1 root root 9 2012-07-19 17:33 pci-0000:07:00.0-fc-0x500104f000acfda1:0x0000000000000000-4:0:1:0st -> ../../st5

lrwxrwxrwx 1 root root 9 2012-07-19 17:34 pci-0000:07:00.0-fc-0x500104f000acfdbf:0x0000000000000000-4:0:5:0st -> ../../st8

lrwxrwxrwx 1 root root 10 2012-07-19 17:34 pci-0000:07:00.0-fc-0x500104f000acfdbf:0x0000000000000000-4:0:5:0st-nst -> ../../nst8

lrwxrwxrwx 1 root root 10 2012-07-20 10:19 pci-0000:07:00.0-fc-0x500104f000acfdad:0x0000000000000000-4:0:4:0st-nst -> ../../nst9

lrwxrwxrwx 1 root root 9 2012-07-20 10:19 pci-0000:07:00.0-fc-0x500104f000acfdad:0x0000000000000000-4:0:4:0st -> ../../st9

lrwxrwxrwx 1 root root 9 2012-07-23 19:42 pci-0000:06:00.1-fc-0x500104f000acfdb9:0x0000000000000000-3:0:2:0st -> ../../st1

lrwxrwxrwx 1 root root 10 2012-07-23 19:42 pci-0000:06:00.1-fc-0x500104f000acfdb9:0x0000000000000000-3:0:2:0st-nst -> ../../nst1

lrwxrwxrwx 1 root root 10 2012-07-23 19:45 pci-0000:06:00.1-fc-0x500104f000acfdbc:0x0000000000000000-3:0:4:0st-nst -> ../../nst3

lrwxrwxrwx 1 root root 9 2012-07-23 19:45 pci-0000:06:00.1-fc-0x500104f000acfdbc:0x0000000000000000-3:0:4:0st -> ../../st3

lrwxrwxrwx 1 root root 10 2012-07-24 14:49 pci-0000:06:00.1-fc-0x500104f000acfd98:0x0000000000000000-3:0:1:0st-nst -> ../../nst0

lrwxrwxrwx 1 root root 9 2012-07-24 14:49 pci-0000:06:00.1-fc-0x500104f000acfd98:0x0000000000000000-3:0:1:0st -> ../../st0

Regards,

Abhishek

VOX

reservation conflicts