cancel
Showing results for 
Search instead for 
Did you mean: 

Repeated error 86 Netbackup 7.5

sheriff_e47
Level 5
Partner Accredited

Hi All,

 

I have Netbackup 7.5 run in Solaris 10. There is these two online backup policies that use 7 tape drives streamingly. But, since 5 days ago, there's always error 86 appear among the streaming

bi_86.PNG

erp_86.PNG

 

Every time one of both policies above running, there will be one or two streaming got error 86 on different tape drives and different media.

These are the messages on solaris:

bash-3.2# tail /var/adm/messages
Apr 28 07:40:48 backupdrc bpjava-msvc[167]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
Apr 29 06:44:52 backupdrc bpjava-msvc[2841]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
Apr 30 11:17:31 backupdrc bpjava-msvc[25779]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
Apr 30 11:41:48 backupdrc bpjava-msvc[29228]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
May  1 06:57:53 backupdrc bpjava-msvc[282]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
May  2 07:48:25 backupdrc bpjava-msvc[23494]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
May  2 08:23:28 backupdrc bpjava-msvc[29221]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
May  2 10:07:52 backupdrc bpjava-msvc[15592]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
May  2 15:28:15 backupdrc bpjava-msvc[5370]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.
May  2 15:36:12 backupdrc bpjava-msvc[6510]: [ID 427199 user.error] pam_dial_auth: terminal-device not specifiedby login, returning Error in underlying service module.

I've attached the media log. (medialog.txt)

So, any one have faced problem like this before? Please help anyone. 

Thank you

 

1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

We need messages file as well on this media server. Not much from NBU point of view in bptm log, other than  errors we've seen already.

Just a thought - please check firmware versions on all tape drive using 'tpautoconf -t'.

I have seen in the past where tapes were written in higher version tape drive and later loaded in older firmware tape drive to be appended to, that this drive was unable to position the tape.

Problem was solved when all drives were upgraded to same firmware level.

View solution in original post

16 REPLIES 16

Andy_Welburn
Level 6

Looks like 2 totally different issues here.

 

- 'password authentication':

http://www.symantec.com/business/support/index?page=content&id=TECH58705

 

- potentially faulty media or hardware

- 3 tapes causing issues on 3 drives or vice versa (altho' personally I'd be more inclined to believe the issue is with the tapes)

http://www.symantec.com/business/support/index?page=content&id=HOWTO90513

Maybe worth interrogating the media errors log to determine frequency of issues per drive or media:

/usr/openv/netbackup/db/media/errors

(mph put together a little script some time back to assist in this .... will see if I can locate)

https://www-secure.symantec.com/connect/downloads/tperrsh-script-solaris-only

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

/var/adm/messages is not showing any device-related errors. But it does not seem to be messages file on media server showing the errors - mediadrc?

Please add VERBOSE entry to /usr/openv/volmgr/vm.conf on mediadrc and restart NBU.
Also check that bptm log folder exists under /usr/openv/netbackup/logs.

Hardware and Media Manager errors will be logged in messages file and NBU I/O operations in bptm log.

The media log report does not include enough info to troubleshoot.

sheriff_e47
Level 5
Partner Accredited

Hi Marianne,

Thanks for the explanation. I have added VERBOSE in vm.conf at mediadrc and restart the NBU. And I've just created directory bptm so, there's no log generated yet. But I will update this after the btptm has its log.

 

 

sheriff_e47
Level 5
Partner Accredited

Hi Andy,

Actually at first I have performed "freeze" to the all error medias, then the error accoured again (at different error media id of course) and I freeze them again. And now it turns out, the other media got error again. So there have been 6 medias I have frozen. What do you think the problem is? Thanks

 

 

bash-3.2# more /usr/openv/netbackup/db/media/errors
04/14/12 08:26:20 JL0023 5 POSITION_ERROR HP.ULTRIUM5-SCSI.006
05/28/12 00:00:49 JL0068 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
06/03/12 21:29:24 JL0068 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
06/03/12 21:40:49 JL0068 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
06/24/12 21:30:12 JL0068 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
06/24/12 21:41:36 JL0068 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
07/01/12 22:10:19 JL0066 5 POSITION_ERROR HP.ULTRIUM5-SCSI.006
07/12/12 16:26:11 JL0101 5 TAPE_ALERT HP.ULTRIUM5-SCSI.006 0x00000000 0x00000040
08/27/12 12:42:33 JL0103 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
08/31/12 21:14:19 JL0111 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
09/02/12 00:44:25 JL0111 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
09/03/12 16:25:15 JL0101 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
09/16/12 21:29:43 JL0103 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
09/23/12 21:34:39 JL0103 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
10/01/12 01:05:04 JL0117 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
10/07/12 21:27:51 JL0117 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
10/23/12 01:29:36 JL0117 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
10/23/12 01:41:56 JL0117 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
01/20/13 07:33:49 JL0007 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
02/03/13 00:35:41 JL0007 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
02/03/13 00:47:13 JL0007 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
02/04/13 01:04:57 JL0188 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
02/10/13 21:30:04 JL0188 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
02/10/13 21:41:06 JL0188 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
02/17/13 21:32:03 JL0188 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
02/17/13 21:43:07 JL0188 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
02/24/13 21:32:32 JL0188 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
02/24/13 21:43:34 JL0188 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
03/04/13 01:23:04 JL0239 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
03/10/13 21:25:09 JL0239 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
03/10/13 21:36:28 JL0239 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
03/17/13 21:59:34 JL0239 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
03/17/13 22:10:55 JL0239 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
04/01/13 01:53:17 JL0253 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/14/13 21:26:49 JL0253 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
04/14/13 21:38:58 JL0253 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
04/21/13 21:28:16 JL0253 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/21/13 21:40:24 JL0253 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/28/13 21:28:49 JL0253 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
04/28/13 21:40:57 JL0253 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
05/13/13 00:22:49 JL0725 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
05/19/13 21:27:32 JL0725 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
05/19/13 21:38:38 JL0725 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
05/26/13 21:27:52 JL0725 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
05/26/13 21:38:56 JL0725 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
06/02/13 05:11:56 JL0717 5 POSITION_ERROR HP.ULTRIUM5-SCSI.006
06/09/13 21:58:41 JL0725 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
06/09/13 22:00:28 JL0725 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
06/09/13 22:09:50 JL0725 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
06/10/13 04:33:36 JL0740 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
06/11/13 15:39:03 JL0740 5 POSITION_ERROR HP.ULTRIUM5-SCSI.006
06/11/13 15:56:46 JL0094 6 OPEN_ERROR HP.ULTRIUM5-SCSI.002
06/15/13 03:26:51 JL0733 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
06/23/13 00:39:38 JL0733 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
06/23/13 00:51:18 JL0733 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
06/30/13 00:38:59 JL0733 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
06/30/13 00:50:37 JL0733 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
07/07/13 00:39:56 JL0733 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
07/07/13 00:51:33 JL0733 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
08/05/13 01:45:55 JL0972 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
09/03/13 20:27:14 JL0972 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
09/16/13 01:51:30 JL0940 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
09/22/13 21:32:29 JL0940 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
09/22/13 21:43:44 JL0940 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
09/29/13 21:30:57 JL0940 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
09/29/13 21:42:11 JL0940 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
10/06/13 21:32:42 JL0940 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
10/06/13 21:43:56 JL0940 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
10/06/13 23:56:40 JL0013 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
10/13/13 21:33:58 JL0013 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
10/13/13 21:46:19 JL0013 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
10/27/13 03:56:53 JL0910 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
01/06/14 01:34:45 JL0259 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
01/12/14 22:35:06 JL0259 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
01/13/14 01:36:02 JL0251 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
01/13/14 01:38:10 JL0251 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
03/23/14 05:19:33 JL0246 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
03/24/14 01:39:35 JL0234 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
03/30/14 00:19:27 JL0246 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
04/07/14 02:19:58 JL0732 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
04/13/14 21:08:11 JL0732 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
04/21/14 19:49:33 JL0749 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
04/22/14 17:18:01 JL0749 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
04/22/14 20:05:38 JL0713 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
04/25/14 23:47:47 JL0203 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/25/14 23:49:54 JL0203 1 TAPE_ALERT HP.ULTRIUM5-SCSI.002 0x00000000 0x00000040
04/26/14 06:32:51 JL0205 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
04/26/14 06:34:58 JL0205 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/26/14 07:48:41 JL0203 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
04/26/14 07:50:59 JL0203 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/26/14 14:32:47 JL0205 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
04/26/14 14:35:40 JL0205 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
04/27/14 23:59:07 JL0729 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
04/28/14 00:01:26 JL0729 0 TAPE_ALERT HP.ULTRIUM5-SCSI.001 0x00000000 0x00000040
04/28/14 06:10:16 JL0973 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
04/28/14 06:12:35 JL0973 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/28/14 07:38:05 JL0729 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
04/28/14 07:40:13 JL0729 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
04/28/14 14:58:32 JL0973 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
04/28/14 15:00:50 JL0973 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/28/14 17:42:15 JL0735 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/28/14 21:36:32 JL0729 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/28/14 21:39:02 JL0729 1 TAPE_ALERT HP.ULTRIUM5-SCSI.002 0x00000000 0x00000040
04/29/14 02:35:56 JL0973 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
04/29/14 02:38:15 JL0973 1 TAPE_ALERT HP.ULTRIUM5-SCSI.002 0x00000000 0x00000040
04/29/14 07:50:14 JL0729 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
04/29/14 07:52:21 JL0729 2 TAPE_ALERT HP.ULTRIUM5-SCSI.003 0x00000000 0x00000040
04/29/14 07:55:43 JL0729 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
04/29/14 07:58:00 JL0729 2 TAPE_ALERT HP.ULTRIUM5-SCSI.003 0x00000000 0x00000040
04/29/14 15:29:08 JL0973 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
04/29/14 15:31:45 JL0973 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/29/14 15:34:36 JL0973 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
04/29/14 15:36:54 JL0973 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/29/14 18:34:35 JL0035 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
04/29/14 18:36:44 JL0035 2 TAPE_ALERT HP.ULTRIUM5-SCSI.003 0x00000000 0x00000040
04/30/14 02:23:30 JL0973 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
04/30/14 02:24:48 JL0035 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
04/30/14 02:25:48 JL0973 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
04/30/14 02:26:57 JL0035 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/30/14 10:44:18 JL0213 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
04/30/14 18:56:51 JL0219 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
05/01/14 02:34:51 JL0219 2 POSITION_ERROR HP.ULTRIUM5-SCSI.003
05/01/14 05:24:52 JL0198 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
05/01/14 11:38:09 JL0192 6 POSITION_ERROR HP.ULTRIUM5-SCSI.007
05/01/14 14:26:18 JL0198 0 POSITION_ERROR HP.ULTRIUM5-SCSI.001
05/01/14 14:28:04 JL0219 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
05/01/14 21:15:13 JL0192 1 POSITION_ERROR HP.ULTRIUM5-SCSI.002
05/02/14 02:22:25 JL0219 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
05/02/14 05:33:26 JL1131 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
05/02/14 07:50:46 JL0192 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005
05/02/14 14:55:55 JL1131 3 POSITION_ERROR HP.ULTRIUM5-SCSI.004
05/02/14 14:56:04 JL0219 4 POSITION_ERROR HP.ULTRIUM5-SCSI.005

sheriff_e47
Level 5
Partner Accredited

Hi Andy Welburn,

At first (3 days ago) I saw the error, I performed "freeze" to the error media. And the next backup, the backup streaming are all normal. Then at next schedule, it happened again at different media id, so i did freeze again.Then it happen again this time. So there have been 6 media I have frozen till now. Here I attach the media/error. Please advice. Thanks

 

Dip
Level 4

Can you paste one of the failed Job Details here? 

sheriff_e47
Level 5
Partner Accredited

Hi All,

Here I attached the bptm log of mediadrc. you can see that erpscs offline got error 86 again.

 

05/04/2014 14:13:39 - Info nbjm (pid=1752) starting backup job (jobid=20916) for client mediadrc, policy Drc_STR_Erpscs_Online, schedule Full_Online_Night
05/04/2014 14:13:39 - Info nbjm (pid=1752) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=20916, request id:{9E1DB160-D35B-11E3-B4A5-002128E8EDEA})
05/04/2014 14:13:39 - requesting resource mediadrc-hcart2-robot-tld-0
05/04/2014 14:13:39 - requesting resource backupdrc.NBU_CLIENT.MAXJOBS.mediadrc
05/04/2014 14:13:39 - requesting resource backupdrc.NBU_POLICY.MAXJOBS.Drc_STR_Erpscs_Online
05/04/2014 14:13:40 - granted resource  backupdrc.NBU_CLIENT.MAXJOBS.mediadrc
05/04/2014 14:13:40 - granted resource  backupdrc.NBU_POLICY.MAXJOBS.Drc_STR_Erpscs_Online
05/04/2014 14:13:40 - granted resource  JL0210
05/04/2014 14:13:40 - granted resource  HP.ULTRIUM5-SCSI.004
05/04/2014 14:13:40 - granted resource  mediadrc-hcart2-robot-tld-0
05/04/2014 14:13:41 - estimated 1764461068 kbytes needed
05/04/2014 14:13:41 - Info nbjm (pid=1752) started backup (backupid=mediadrc_1399187621) job for client mediadrc, policy Drc_STR_Erpscs_Online, schedule Full_Online_Night on storage unit mediadrc-hcart2-robot-tld-0
05/04/2014 14:13:41 - started process bpbrm (pid=26716)
05/04/2014 14:13:42 - Info bpbrm (pid=26716) mediadrc is the host to backup data from
05/04/2014 14:13:42 - Info bpbrm (pid=26716) reading file list from client
05/04/2014 14:13:42 - Info bpbrm (pid=26716) starting bpbkar on client
05/04/2014 14:13:42 - Info bpbkar (pid=26732) Backup started
05/04/2014 14:13:42 - Info bpbrm (pid=26716) bptm pid: 26733
05/04/2014 14:13:42 - Info bptm (pid=26733) start
05/04/2014 14:13:42 - Info bptm (pid=26733) using 1048576 data buffer size
05/04/2014 14:13:42 - Info bptm (pid=26733) using 32 data buffers
05/04/2014 14:13:42 - connecting
05/04/2014 14:13:42 - connected; connect time: 0:00:00
05/04/2014 14:13:43 - Info bptm (pid=26733) start backup
05/04/2014 14:13:43 - Info bptm (pid=26733) Waiting for mount of media id JL0210 (copy 1) on server mediadrc.
05/04/2014 14:13:43 - mounting JL0210
05/04/2014 14:16:03 - Info bptm (pid=26733) media id JL0210 mounted on drive index 3, drivepath /dev/rmt/6cbn, drivename HP.ULTRIUM5-SCSI.004, copy 1
05/04/2014 14:16:03 - mounted JL0210; mount time: 0:02:20
05/04/2014 14:16:03 - positioning JL0210 to file 7
05/04/2014 14:17:40 - Error bptm (pid=26733) ioctl (MTBSF) failed on media id JL0210, drive index 3, I/O error (bptm.c.22526)
05/04/2014 14:17:40 - Info bptm (pid=26733) EXITING with status 86 <----------
05/04/2014 14:17:41 - Error bpbrm (pid=26716) from client mediadrc: ERR - bpbkar exiting because backup is aborting
05/04/2014 14:17:42 - Info bpbkar (pid=26732) done. status: 86: media position error
05/04/2014 14:17:42 - end writing
media position error  (86)

sheriff_e47
Level 5
Partner Accredited

Hi,

I have paste the failed job detail. what do you think seems to be problem? Thanks  

sheriff_e47
Level 5
Partner Accredited

Hi Marianne,

 

Here I attach a new bptm log. Can you please check it and share your analysis? Thanks

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

We need messages file as well on this media server. Not much from NBU point of view in bptm log, other than  errors we've seen already.

Just a thought - please check firmware versions on all tape drive using 'tpautoconf -t'.

I have seen in the past where tapes were written in higher version tape drive and later loaded in older firmware tape drive to be appended to, that this drive was unable to position the tape.

Problem was solved when all drives were upgraded to same firmware level.

mph999
Level 6
Employee Accredited

tperr.sh shows a fairly even spread of erros across the tapes/ drives -  tapes that show more errors the the others are of interst, but the numbers probably aren't high enough to pin-point anything for sure.

JL0251 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 2)
JL0035 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 4)
JL0234 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0117 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 4)
JL0910 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0253 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 7)
JL0740 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0713 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0732 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0246 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0219 has had errors in 4 different drives   (Total occurrences (errors) of this volume is 5)
JL0192 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 3)
JL0066 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0940 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 7)
JL0733 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 7)
JL0094 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0725 has had errors in 4 different drives   (Total occurrences (errors) of this volume is 8)
JL0239 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 5)
JL0068 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 5)
JL0735 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0717 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0259 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0188 has had errors in 4 different drives   (Total occurrences (errors) of this volume is 7)
JL0972 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0729 has had errors in 4 different drives   (Total occurrences (errors) of this volume is 10)
JL0198 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0973 has had errors in 4 different drives   (Total occurrences (errors) of this volume is 12)
JL0749 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0101 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 2)
JL0111 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 2)
JL0103 has had errors in 3 different drives   (Total occurrences (errors) of this volume is 3)
JL0013 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 3)
JL0203 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 4)
JL0023 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL1131 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 2)
JL0213 has had errors in 1 different drives   (Total occurrences (errors) of this volume is 1)
JL0205 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 4)
JL0007 has had errors in 2 different drives   (Total occurrences (errors) of this volume is 3)


HP.ULTRIUM5-SCSI.001 has had errors with 10 different tapes   (Total occurrences (errors) for this drive is 19)
HP.ULTRIUM5-SCSI.002 has had errors with 10 different tapes   (Total occurrences (errors) for this drive is 17)
HP.ULTRIUM5-SCSI.003 has had errors with 9 different tapes   (Total occurrences (errors) for this drive is 15)
HP.ULTRIUM5-SCSI.004 has had errors with 16 different tapes   (Total occurrences (errors) for this drive is 26)
HP.ULTRIUM5-SCSI.005 has had errors with 11 different tapes   (Total occurrences (errors) for this drive is 21)
HP.ULTRIUM5-SCSI.006 has had errors with 5 different tapes   (Total occurrences (errors) for this drive is 5)
HP.ULTRIUM5-SCSI.007 has had errors with 20 different tapes   (Total occurrences (errors) for this drive is 29)

 

The errors file shows :

07/12/12 16:26:11 JL0101 5 TAPE_ALERT HP.ULTRIUM5-SCSI.006 0x00000000 0x00000040
09/03/12 16:25:15 JL0101 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
01/13/14 01:38:10 JL0251 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/25/14 23:49:54 JL0203 1 TAPE_ALERT HP.ULTRIUM5-SCSI.002 0x00000000 0x00000040
04/26/14 06:34:58 JL0205 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/26/14 07:50:59 JL0203 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/26/14 14:35:40 JL0205 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
04/28/14 00:01:26 JL0729 0 TAPE_ALERT HP.ULTRIUM5-SCSI.001 0x00000000 0x00000040
04/28/14 06:12:35 JL0973 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/28/14 07:40:13 JL0729 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
04/28/14 15:00:50 JL0973 3 TAPE_ALERT HP.ULTRIUM5-SCSI.004 0x00000000 0x00000040
04/28/14 21:39:02 JL0729 1 TAPE_ALERT HP.ULTRIUM5-SCSI.002 0x00000000 0x00000040
04/29/14 02:38:15 JL0973 1 TAPE_ALERT HP.ULTRIUM5-SCSI.002 0x00000000 0x00000040
04/29/14 07:52:21 JL0729 2 TAPE_ALERT HP.ULTRIUM5-SCSI.003 0x00000000 0x00000040
04/29/14 07:58:00 JL0729 2 TAPE_ALERT HP.ULTRIUM5-SCSI.003 0x00000000 0x00000040
04/29/14 15:31:45 JL0973 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/29/14 15:36:54 JL0973 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040
04/29/14 18:36:44 JL0035 2 TAPE_ALERT HP.ULTRIUM5-SCSI.003 0x00000000 0x00000040
04/30/14 02:25:48 JL0973 6 TAPE_ALERT HP.ULTRIUM5-SCSI.007 0x00000000 0x00000040
04/30/14 02:26:57 JL0035 4 TAPE_ALERT HP.ULTRIUM5-SCSI.005 0x00000000 0x00000040

This translates to ...

root@womble 2014-03-30 $ /netbackup/scripts/DecodeTA.pl 0x00000000 0x00000040
Flag 58: Microcode failure. Severity: Warning

 

You may wish to contact the hardware vendor to look into this further - tapealerts are sent from the drives, so nothing to do with NBU.

Errors such as these (from bptm log above)

ioctl (MTBSF) failed 

... are typically hardware related (or firmware), could be drive or even HBA, but I'd start with the drive, especially seeing the tapealerts are showing an issue with 'Microcode failure'

 

 

sheriff_e47
Level 5
Partner Accredited

Hi,

I tried this tpautoconf on media server, but command not found:

bash-3.2# hostname
mediadrc
bash-3.2# tpautoconf -t
bash: tpautoconf: command not found

 

But as you all said, I think its related to firmware failure.

Capture1.PNG

So, I will contact our hardware vendor for this firmware issue. Thank you all very much

 

Best Regards,

Sheriff

mph999
Level 6
Employee Accredited

tpautoconf is in /usr/openv/volmgr/bin

It is advisable to set the NBU paths in the OS PATH variable

/usr/openv/netbackup/bin

/usr/openv/netbackup/bin/admincmd

/usr/openv/volmgr/bin

/usr/openv/db/bin

/usr/openv/netbackup/bin/goodies

/usr/openv/netbackup/bin/goodies/support

sheriff_e47
Level 5
Partner Accredited

Hi,

I have contacted our h/w vendor and he said that Almost all drives are running with very old firmware (I2DS), and the latest code is I67S.

bash-3.2# /usr/openv/volmgr/bin/tpautoconf -t
TPAC60 HP      Ultrium 5-SCSI  I2DS HU1049E1LK -1 -1 -1 -1 /dev/rmt/0cbn - - 
TPAC60 HP      Ultrium 5-SCSI  I2DS HU1049E1JU -1 -1 -1 -1 /dev/rmt/2cbn - - 
TPAC60 HP      Ultrium 5-SCSI  I2DS HU1049E1K4 -1 -1 -1 -1 /dev/rmt/5cbn - - 
TPAC60 HP      Ultrium 5-SCSI  I2DS HU1049E1JV -1 -1 -1 -1 /dev/rmt/6cbn - - 
TPAC60 HP      Ultrium 5-SCSI  I2DS HU1050E652 -1 -1 -1 -1 /dev/rmt/4cbn - - 
TPAC60 HP      Ultrium 5-SCSI  I59S HUE2470CD9 -1 -1 -1 -1 /dev/rmt/1cbn - - 
TPAC60 HP      Ultrium 5-SCSI  I2DS HU1050E65F -1 -1 -1 -1 /dev/rmt/3cbn - - 

So is the library firmware. So, we will upgrade the firmware of those drives n librabry. Hope this will solve the problem.

Thank you very much for your big help 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
One drive has a higher firmware than the rest: I59S. Never a good idea. All should be on the same level.

sheriff_e47
Level 5
Partner Accredited

Hi,

I have already upgraded all of those 7 drives on mediadrc to latest version. And the backup jobs of mediadrc are all now done successfully (status 0 for all streaming). So, Many thanks to you all for your big help smiley

bash-3.2# ./tpautoconf -t
TPAC60 HP      Ultrium 5-SCSI  I67S HU1049E1LK -1 -1 -1 -1 /dev/rmt/0cbn - -
TPAC60 HP      Ultrium 5-SCSI  I67S HU1049E1JU -1 -1 -1 -1 /dev/rmt/2cbn - -
TPAC60 HP      Ultrium 5-SCSI  I67S HU1049E1K4 -1 -1 -1 -1 /dev/rmt/5cbn - -
TPAC60 HP      Ultrium 5-SCSI  I67S HU1049E1JV -1 -1 -1 -1 /dev/rmt/6cbn - -
TPAC60 HP      Ultrium 5-SCSI  I67S HU1050E652 -1 -1 -1 -1 /dev/rmt/4cbn - -
TPAC60 HP      Ultrium 5-SCSI  I67S HUE2470CD9 -1 -1 -1 -1 /dev/rmt/1cbn - -
TPAC60 HP      Ultrium 5-SCSI  I67S HU1050E65F -1 -1 -1 -1 /dev/rmt/3cbn - -

 

Best Regards,

Sheriff