08-26-2011 07:03 AM
Hi guys,
We've recently had some issues with a non-functional tape drive on an STK L180, which we've replaced. However, the alerts as the ones seen below have not stopped:
Aug 23 13:28:43 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
Aug 23 13:29:00 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
Aug 23 13:29:18 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
Aug 23 13:29:35 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
My best guess is that it's somehow linked to the Vault jobs that keep hanging, which I gotta manually cancel in order to have (at least one or two) running normally afterwards:
root@backup5 # bpdbjobs | grep Vault | more
1123015 Vault Active vault-daily-eject daily-eject backup34_1 backup34_1 08/26/11 11:39:00 002:52:59
1123010 Vault Done 0 vault-daily-reports daily-reports backup34_1 backup34_1 08/26/11 10:39:03 000:05:59 08/26/11
10:45:02
1123001 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/26/11 08:09:11 000:00:03 08/26/11
08:09:14
1122988 Vault Done 0 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/26/11 06:19:11 001:56:51 08/26/11
08:16:02
1122547 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/25/11 10:39:16 000:00:03 08/25/11
10:39:19
1122539 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/25/11 08:09:20 000:00:03 08/25/11
08:09:23
1122131 Vault Done 150 vault-daily-eject daily-eject backup34_1 backup34_1 08/24/11 11:39:07 030:34:44 08/25/11
18:13:51
1122129 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/24/11 10:39:07 000:00:02 08/24/11
10:39:09
1122119 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/24/11 08:09:10 000:00:03 08/24/11
08:09:13
1121803 Vault Done 63 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/24/11 04:08:58 038:04:39 08/25/11
18:13:37
1121692 Vault Done 150 vault-daily-eject daily-eject backup34_1 backup34_1 08/23/11 11:39:02 006:57:40 08/23/11
18:36:42
1121687 Vault Done 0 vault-daily-reports daily-reports backup34_1 backup34_1 08/23/11 10:38:53 000:05:54 08/23/11
10:44:47
1121675 Vault Done 0 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/23/11 08:10:09 000:00:32 08/23/11
08:10:41
1121220 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/22/11 10:39:13 000:00:03 08/22/11
10:39:16
1121209 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/22/11 08:09:17 000:00:03 08/22/11
08:09:20
1120802 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/21/11 10:39:02 000:00:03 08/21/11
10:39:05
1120796 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/21/11 08:09:06 000:00:03 08/21/11
08:09:09
1120483 Vault Done 63 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/21/11 04:08:51 041:36:12 08/22/11
21:45:03
1120398 Vault Done 150 vault-daily-eject daily-eject backup34_1 backup34_1 08/20/11 11:38:53 058:06:11 08/22/11
21:45:04
1120395 Vault Done 0 vault-daily-reports daily-reports backup34_1 backup34_1 08/20/11 10:30:21 000:08:31 08/20/11
10:38:52
1120390 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/20/11 08:09:26 000:00:03 08/20/11
08:09:29
1119968 Vault Done 296 vault-daily-eject daily-eject backup34_1 backup34_1 08/19/11 11:33:03 000:00:03 08/19/11
11:33:06
1119957 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/19/11 10:39:09 000:00:03 08/19/11
10:39:12
1119950 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/19/11 08:09:13 000:00:03 08/19/11
08:09:16
1119522 Vault Done 296 vault-daily-eject daily-eject backup34_1 backup34_1 08/18/11 11:38:57 000:00:03 08/18/11
11:39:00
1119520 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/18/11 10:38:57 000:00:03 08/18/11
10:39:00
1119512 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/18/11 08:09:01 000:00:03 08/18/11
08:09:04
1119195 Vault Done 63 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/18/11 04:11:32 053:54:28 08/20/11
10:06:00
1119082 Vault Done 0 vault-daily-eject daily-eject backup34_1 backup34_1 08/17/11 11:38:50 000:29:12 08/17/11
12:08:02
Notice that it's the vault-daily-eject that craps out and hangs in most cases, however I cannot 100% link the errors in the logs to the hung jobs... Any ideas as to where I might start my investigations?
NBU 5.0 (unpatched...don't ask...), Solaris 5.9, STK L180. The newly replaced TD has a newer FW version than the others, could that be an issue or should I look for something that blocked?
Thanks for your kind replies,
Dan