08-26-2011 07:05 AM
Hi guys,
We've recently had some issues with a non-functional tape drive on an STK L180, which we've replaced. However, the alerts as the ones seen below have not stopped:
Aug 23 13:28:43 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
Aug 23 13:29:00 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
Aug 23 13:29:18 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
Aug 23 13:29:35 backup5 tldcd[28736]: [ID 200886 daemon.error] External request already in progress on TLD(0), from host backup4
My best guess is that it's somehow linked to the Vault jobs that keep hanging, which I gotta manually cancel in order to have (at least one or two) running normally afterwards:
root@backup5 # bpdbjobs | grep Vault | more
1123015 Vault Active vault-daily-eject daily-eject backup34_1 backup34_1 08/26/11 11:39:00 002:52:59
1123010 Vault Done 0 vault-daily-reports daily-reports backup34_1 backup34_1 08/26/11 10:39:03 000:05:59 08/26/11
10:45:02
1123001 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/26/11 08:09:11 000:00:03 08/26/11
08:09:14
1122988 Vault Done 0 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/26/11 06:19:11 001:56:51 08/26/11
08:16:02
1122547 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/25/11 10:39:16 000:00:03 08/25/11
10:39:19
1122539 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/25/11 08:09:20 000:00:03 08/25/11
08:09:23
1122131 Vault Done 150 vault-daily-eject daily-eject backup34_1 backup34_1 08/24/11 11:39:07 030:34:44 08/25/11
18:13:51
1122129 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/24/11 10:39:07 000:00:02 08/24/11
10:39:09
1122119 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/24/11 08:09:10 000:00:03 08/24/11
08:09:13
1121803 Vault Done 63 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/24/11 04:08:58 038:04:39 08/25/11
18:13:37
1121692 Vault Done 150 vault-daily-eject daily-eject backup34_1 backup34_1 08/23/11 11:39:02 006:57:40 08/23/11
18:36:42
1121687 Vault Done 0 vault-daily-reports daily-reports backup34_1 backup34_1 08/23/11 10:38:53 000:05:54 08/23/11
10:44:47
1121675 Vault Done 0 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/23/11 08:10:09 000:00:32 08/23/11
08:10:41
1121220 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/22/11 10:39:13 000:00:03 08/22/11
10:39:16
1121209 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/22/11 08:09:17 000:00:03 08/22/11
08:09:20
1120802 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/21/11 10:39:02 000:00:03 08/21/11
10:39:05
1120796 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/21/11 08:09:06 000:00:03 08/21/11
08:09:09
1120483 Vault Done 63 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/21/11 04:08:51 041:36:12 08/22/11
21:45:03
1120398 Vault Done 150 vault-daily-eject daily-eject backup34_1 backup34_1 08/20/11 11:38:53 058:06:11 08/22/11
21:45:04
1120395 Vault Done 0 vault-daily-reports daily-reports backup34_1 backup34_1 08/20/11 10:30:21 000:08:31 08/20/11
10:38:52
1120390 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/20/11 08:09:26 000:00:03 08/20/11
08:09:29
1119968 Vault Done 296 vault-daily-eject daily-eject backup34_1 backup34_1 08/19/11 11:33:03 000:00:03 08/19/11
11:33:06
1119957 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/19/11 10:39:09 000:00:03 08/19/11
10:39:12
1119950 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/19/11 08:09:13 000:00:03 08/19/11
08:09:16
1119522 Vault Done 296 vault-daily-eject daily-eject backup34_1 backup34_1 08/18/11 11:38:57 000:00:03 08/18/11
11:39:00
1119520 Vault Done 275 vault-daily-reports daily-reports backup34_1 backup34_1 08/18/11 10:38:57 000:00:03 08/18/11
10:39:00
1119512 Vault Done 275 vault-daily-itc daily-inlinecopies backup34_1 backup34_1 08/18/11 08:09:01 000:00:03 08/18/11
08:09:04
1119195 Vault Done 63 vault-daily-duplicate daily-duplicate backup34_1 backup34_1 08/18/11 04:11:32 053:54:28 08/20/11
10:06:00
1119082 Vault Done 0 vault-daily-eject daily-eject backup34_1 backup34_1 08/17/11 11:38:50 000:29:12 08/17/11
12:08:02
Notice that it's the vault-daily-eject that craps out and hangs in most cases, however I cannot 100% link the errors in the logs to the hung jobs... Any ideas as to where I might start my investigations?
NBU 5.0 (unpatched...don't ask...), Solaris 5.9, STK L180. The newly replaced TD has a newer FW version than the others, could that be an issue or should I look for something that blocked?
Thanks in advance for your englightning replies,
Dan
Solved! Go to Solution.
09-10-2011 02:35 AM
Thanks Marianne, actually this is exactly what I've done last Monday (how unfortunate though that I had not seen your response sooner... twould've spared me a lotta troubles....). I did not however reboot the machines themselves, I've just restarted NBU on all the media servers on which the new tape drive had been reconfigured and the errors have stopped (moreover, a whole bunch of other errors stopped too...).
So, if anyone has similar issues and stumbles upon this thread, do this: restart NBU on all the media servers.
08-26-2011 11:23 PM
Have you tried to reboot the whole lot? Robot as well as robot control host.
09-10-2011 02:35 AM
Thanks Marianne, actually this is exactly what I've done last Monday (how unfortunate though that I had not seen your response sooner... twould've spared me a lotta troubles....). I did not however reboot the machines themselves, I've just restarted NBU on all the media servers on which the new tape drive had been reconfigured and the errors have stopped (moreover, a whole bunch of other errors stopped too...).
So, if anyone has similar issues and stumbles upon this thread, do this: restart NBU on all the media servers.