Unexpected Nebackup restart in cluster environment

[virankumar]

There is an Plan activity to restart the netbackup and its done successfully

And there are two Unexpected restarts happen in the same day ...will you be able to give me support On this case

If you need any logs ill povide

[Marianne]

I have seen that MonitorTimeout on the NetBackup resource is sometimes causing an issue in a very busy environment.
If memory serves my right, The default for the MonitorTimeout attribute is 60 seconds. This means that if the Monitor entry point (that checks for online processes) does not return a success message within 60 seconds, the process is seen as failed and VCS needs to take corrective action. The NBU resource is configured to restart instead of immediate failover. So, all NBU processes will be stopped and restarted.

You need to check Engine_A cluster log to check/confirm if this is the reason for restart.
If so, increase MonitorTimeout to something like 90 or 120 seconds.  MonitorInterval should then at the same time be increased to the same value.

If you are not familiar with VCS concepts, please show this post to your VCS administrator to assist.

[RamNagalla]

hi,

i would advice you to start checking from below logs.

/usr/openv/db/log/server.log

please provide the server.log, and admin log from /usr/openv/netbackup/logs/admin.

what is the OS?

what is the netbackup versions?

does the netbackup restarted or server restart?

[virankumar]

Hi Marianne ,

Thanks for you reply.

I am not familiyar in VCS, Any way i logged a case with vendor, Let see what they says.

[virankumar]

Hai Nagalla,

Below are the logs, you requested.

Netbackup Version : 6.5.6

Os : Solaris 5.10

Netbackup Service Restarted.

============================ Syslog=======================

I. 01/28 00:18:30. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 00:18
I. 01/28 00:18:31. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 00:18
I. 01/28 00:27:35. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 00:27
I. 01/28 00:27:36. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 00:27
I. 01/28 01:00:57. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 01:00
I. 01/28 01:00:57. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 01:00
I. 01/28 01:02:36. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 01:02
I. 01/28 01:02:37. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 01:02
I. 01/28 01:20:58. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 01:20
I. 01/28 01:20:58. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 01:20
I. 01/28 01:21:01. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 01:21
I. 01/28 01:21:01. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 01:21
I. 01/28 02:02:04. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 02:02
I. 01/28 02:02:04. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 02:02
I. 01/28 02:07:50. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 02:07
I. 01/28 02:07:51. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 02:07
I. 01/28 02:35:24. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 02:35
I. 01/28 02:35:24. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 02:35
I. 01/28 02:40:07. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 02:40
I. 01/28 02:40:08. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 02:40
I. 01/28 02:57:40. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 02:57
I. 01/28 02:57:40. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 02:57
I. 01/28 03:04:51. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 03:04
I. 01/28 03:04:51. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 03:04
I. 01/28 03:05:49. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 03:05
I. 01/28 03:05:50. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 03:05
I. 01/28 03:20:46. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 03:20
I. 01/28 03:20:46. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 03:20
I. 01/28 03:34:11. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 03:34
I. 01/28 03:34:11. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 03:34
I. 01/28 03:50:17. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 03:50
I. 01/28 03:50:17. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 03:50
I. 01/28 03:54:12. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 03:54
I. 01/28 03:54:12. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 03:54
I. 01/28 04:14:13. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 04:14
I. 01/28 04:14:13. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 04:14
I. 01/28 04:30:55. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 04:30
I. 01/28 04:30:55. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 04:30
I. 01/28 04:34:14. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 04:34
I. 01/28 04:34:14. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 04:34
I. 01/28 04:54:15. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 04:54
I. 01/28 04:54:15. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 04:54
I. 01/28 05:14:17. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 05:14
I. 01/28 05:14:17. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 05:14
I. 01/28 05:25:40. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 05:25
I. 01/28 05:25:40. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 05:25
I. 01/28 05:34:18. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 05:34
I. 01/28 05:34:18. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 05:34
I. 01/28 05:39:24. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 05:39
I. 01/28 05:39:24. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 05:39
I. 01/28 05:54:19. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 05:54
I. 01/28 05:54:19. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 05:54
I. 01/28 06:14:20. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 06:14
I. 01/28 06:14:20. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 06:14
I. 01/28 06:39:24. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 06:39
I. 01/28 06:39:24. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 06:39
I. 01/28 06:40:36. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 06:40
I. 01/28 06:40:36. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 06:40
I. 01/28 07:00:12. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:00
I. 01/28 07:00:12. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:00
I. 01/28 07:10:16. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:16. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:16. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:16. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:17. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:17. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:55. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:55. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:55. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:10:55. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:10
I. 01/28 07:30:18. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:30
I. 01/28 07:30:18. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:30
I. 01/28 07:50:19. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:50
I. 01/28 07:50:19. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 07:50
I. 01/28 07:53:21. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:53
I. 01/28 07:53:22. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 07:53
I. 01/28 08:10:20. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 08:10
I. 01/28 08:10:20. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 08:10
I. 01/28 08:30:22. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 08:30
I. 01/28 08:30:22. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 08:30
I. 01/28 08:33:59. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 08:33
I. 01/28 08:34:00. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 08:34
I. 01/28 08:50:23. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 08:50
I. 01/28 08:50:23. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 08:50
I. 01/28 09:10:24. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 09:10
I. 01/28 09:10:24. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 09:10
I. 01/28 09:30:25. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 09:30
I. 01/28 09:30:25. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 09:30
I. 01/28 09:34:00. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 09:34
I. 01/28 09:34:00. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 09:34
I. 01/28 09:50:26. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 09:50
I. 01/28 09:50:27. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 09:50
I. 01/28 10:10:28. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 10:10
I. 01/28 10:10:28. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 10:10
I. 01/28 10:30:29. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 10:30
I. 01/28 10:30:29. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 10:30
I. 01/28 10:34:00. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 10:34
I. 01/28 10:34:00. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 10:34
I. 01/28 10:50:30. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 10:50
I. 01/28 10:50:30. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 10:50
I. 01/28 11:10:31. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 11:10
I. 01/28 11:10:31. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 11:10
I. 01/28 11:21:39. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 11:21
I. 01/28 11:21:39. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 11:21
I. 01/28 11:30:32. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 11:30
I. 01/28 11:30:32. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 11:30
I. 01/28 12:10:14. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 12:10
I. 01/28 12:10:14. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 12:10
I. 01/28 12:23:17. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 12:23
I. 01/28 12:23:18. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 12:23
I. 01/28 12:43:19. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 12:43
I. 01/28 12:43:19. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 12:43
I. 01/28 13:03:20. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 13:03
I. 01/28 13:03:20. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 13:03
I. 01/28 13:03:22. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 13:03
I. 01/28 13:03:22. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 13:03
I. 01/28 13:23:21. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 13:23
I. 01/28 13:23:21. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 13:23
I. 01/28 13:39:28. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 13:39
I. 01/28 13:39:28. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 13:39
I. 01/28 13:43:22. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 13:43
I. 01/28 13:43:22. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 13:43
I. 01/28 14:03:24. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 14:03
I. 01/28 14:03:24. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 14:03
I. 01/28 14:23:25. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 14:23
I. 01/28 14:23:25. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 14:23
I. 01/28 14:26:12. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 14:26
I. 01/28 14:26:12. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 14:26
I. 01/28 14:43:26. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 14:43
I. 01/28 14:43:26. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 14:43
I. 01/28 14:47:38. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 14:47
I. 01/28 14:47:38. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 14:47
I. 01/28 15:03:27. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 15:03
I. 01/28 15:03:27. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 15:03
I. 01/28 15:23:29. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 15:23
I. 01/28 15:23:29. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 15:23
I. 01/28 15:32:59. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 15:32
I. 01/28 15:32:59. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 15:32
I. 01/28 15:43:30. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 15:43
I. 01/28 15:43:30. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 15:43
I. 01/28 16:03:31. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 16:03
I. 01/28 16:03:31. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 16:03
I. 01/28 16:13:36. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 16:13
I. 01/28 16:13:36. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 16:13
I. 01/28 16:23:32. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 16:23
I. 01/28 16:23:32. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 16:23
I. 01/28 16:43:33. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 16:43
I. 01/28 16:43:33. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 16:43
I. 01/28 16:58:18. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 16:58
I. 01/28 16:58:19. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 16:58
I. 01/28 17:35:12. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 17:35
I. 01/28 17:35:12. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 17:35
I. 01/28 17:41:45. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 17:41
I. 01/28 17:41:45. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 17:41
I. 01/28 17:55:13. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 17:55
I. 01/28 17:55:14. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 17:55
I. 01/28 18:17:50. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 18:17
I. 01/28 18:17:56. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 18:17
I. 01/28 18:44:26. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 18:44
I. 01/28 18:44:26. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 18:44
I. 01/28 19:14:12. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 19:14
I. 01/28 19:14:12. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 19:14
I. 01/28 19:33:35. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 19:33
I. 01/28 19:33:35. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 19:33
I. 01/28 19:53:36. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 19:53
I. 01/28 19:53:36. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 19:53
I. 01/28 20:03:31. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 20:03
I. 01/28 20:03:31. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 20:03
I. 01/28 20:30:48. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 20:30
I. 01/28 20:30:49. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 20:30
I. 01/28 20:41:09. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 20:41
I. 01/28 20:41:09. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 20:41
I. 01/28 20:50:50. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 20:50
I. 01/28 20:50:50. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 20:50
I. 01/28 21:05:53. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 21:05
I. 01/28 21:05:54. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 21:05
I. 01/28 21:06:38. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 21:06
I. 01/28 21:06:39. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 21:06
I. 01/28 21:19:05. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 21:19
I. 01/28 21:19:05. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 21:19
I. 01/28 21:29:00. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 21:29
I. 01/28 21:29:00. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 21:29
I. 01/28 21:47:30. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 21:47
I. 01/28 21:47:31. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 21:47
I. 01/28 22:01:57. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 22:01
I. 01/28 22:01:57. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 22:01
I. 01/28 22:20:12. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 22:20
I. 01/28 22:20:12. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 22:20
I. 01/28 22:29:14. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 22:29
I. 01/28 22:29:14. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 22:29
I. 01/28 22:46:22. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 22:46
I. 01/28 22:46:22. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 22:46
I. 01/28 22:50:54. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 22:50
I. 01/28 22:50:54. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 22:50
I. 01/28 23:07:17. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 23:07
I. 01/28 23:07:18. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 23:07
I. 01/28 23:19:33. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 23:19
I. 01/28 23:19:34. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 23:19
I. 01/28 23:43:58. Starting checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 23:43
I. 01/28 23:43:59. Finished checkpoint of "NBDB" (NBDB.db) at Mon Jan 28 2013 23:43
I. 01/28 23:58:59. Starting checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 23:58
I. 01/28 23:58:59. Finished checkpoint of "BMRDB" (BMRDB.db) at Mon Jan 28 2013 23:58
I. 01/29 00:20:58. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 00:20
I. 01/29 00:20:58. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 00:20
I. 01/29 00:33:03. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 00:33
I. 01/29 00:33:04. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 00:33
I. 01/29 00:53:05. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 00:53
I. 01/29 00:53:05. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 00:53
I. 01/29 01:05:46. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 01:05
I. 01/29 01:05:47. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 01:05
I. 01/29 01:38:46. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 01:38
I. 01/29 01:38:48. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 01:38
I. 01/29 01:49:33. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 01:49
I. 01/29 01:49:34. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 01:49
I. 01/29 02:25:14. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 02:25
I. 01/29 02:25:14. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 02:25
I. 01/29 02:28:57. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 02:28
I. 01/29 02:28:58. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 02:28
I. 01/29 02:52:08. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 02:52
I. 01/29 02:52:08. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 02:52
I. 01/29 03:16:21. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 03:16
I. 01/29 03:16:21. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 03:16
I. 01/29 03:25:37. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 03:25
I. 01/29 03:25:37. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 03:25
I. 01/29 03:53:38. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 03:53
I. 01/29 03:53:38. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 03:53
I. 01/29 04:08:30. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 04:08
I. 01/29 04:08:31. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 04:08
I. 01/29 04:23:52. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 04:23
I. 01/29 04:23:52. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 04:23
I. 01/29 04:38:43. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 04:38
I. 01/29 04:38:44. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 04:38
I. 01/29 04:52:32. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 04:52
I. 01/29 04:52:32. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 04:52
I. 01/29 04:59:01. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 04:59
I. 01/29 04:59:01. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 04:59
I. 01/29 05:14:08. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 05:14
I. 01/29 05:14:09. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 05:14
I. 01/29 05:24:35. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 05:24
I. 01/29 05:24:35. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 05:24
I. 01/29 05:45:04. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 05:45
I. 01/29 05:45:05. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 05:45
I. 01/29 05:57:50. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 05:57
I. 01/29 05:57:51. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 05:57
I. 01/29 06:05:06. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 06:05
I. 01/29 06:05:06. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 06:05
I. 01/29 06:25:07. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 06:25
I. 01/29 06:25:07. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 06:25
I. 01/29 06:52:32. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 06:52
I. 01/29 06:52:33. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 06:52
I. 01/29 07:00:13. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:00
I. 01/29 07:00:14. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:00
I. 01/29 07:06:20. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:20. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:20. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:20. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:20. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:21. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:55. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:55. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:55. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:06:55. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:06
I. 01/29 07:26:21. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:26
I. 01/29 07:26:21. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 07:26
I. 01/29 07:43:06. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:43
I. 01/29 07:43:06. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 07:43
I. 01/29 08:26:21. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 08:26
I. 01/29 08:26:21. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 08:26
I. 01/29 08:43:06. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 08:43
I. 01/29 08:43:06. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 08:43
I. 01/29 08:46:23. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 08:46
I. 01/29 08:46:23. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 08:46
I. 01/29 09:43:06. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 09:43
I. 01/29 09:43:06. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 09:43
I. 01/29 09:46:23. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 09:46
I. 01/29 09:46:23. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 09:46
I. 01/29 10:06:06. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 10:06
I. 01/29 10:06:06. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 10:06
I. 01/29 10:26:07. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 10:26
I. 01/29 10:26:07. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 10:26
I. 01/29 10:43:06. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 10:43
I. 01/29 10:43:06. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 10:43
I. 01/29 10:46:08. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 10:46
I. 01/29 10:46:08. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 10:46
I. 01/29 11:03:41. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 11:03
I. 01/29 11:03:41. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 11:03
I. 01/29 11:42:36. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 11:42
I. 01/29 11:42:36. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 11:42
I. 01/29 11:46:34. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 11:46
I. 01/29 11:46:34. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 11:46
I. 01/29 12:06:35. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 12:06
I. 01/29 12:06:35. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 12:06
I. 01/29 12:19:58. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 12:19
I. 01/29 12:19:58. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 12:19
I. 01/29 12:26:36. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 12:26
I. 01/29 12:26:36. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 12:26
I. 01/29 12:46:37. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 12:46
I. 01/29 12:46:38. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 12:46
I. 01/29 13:05:13. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 13:05
I. 01/29 13:05:13. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 13:05
I. 01/29 13:06:39. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:06
I. 01/29 13:06:39. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:06
I. 01/29 13:26:40. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:26
I. 01/29 13:26:40. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:26
I. 01/29 13:46:41. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:46
I. 01/29 13:46:41. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:46
I. 01/29 13:58:07. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 13:58
I. 01/29 13:58:07. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 13:58
I. 01/29 13:59:06. Connection terminated abnormally
I. 01/29 13:59:06. Disconnected SharedMemory client's AppInfo: IP=10.134.64.12;HOST=bnsdc-mast-01;OS=SunOS 5.
10 Generic_127111-01 ;PID=0x6427;VERSION=9.0.2.3854;API=ODBC;TIMEZONEADJUSTMENT=0
I. 01/29 13:59:06. Connection terminated abnormally
I. 01/29 13:59:06. Disconnected SharedMemory client's AppInfo: IP=10.134.64.12;HOST=bnsdc-mast-01;OS=SunOS 5.
10 Generic_127111-01 ;PID=0x17de;VERSION=9.0.2.3854;API=ODBC;TIMEZONEADJUSTMENT=0
I. 01/29 13:59:06. Connection terminated abnormally
I. 01/29 13:59:06. Disconnected SharedMemory client's AppInfo: IP=10.134.64.12;HOST=bnsdc-mast-01;OS=SunOS 5.
10 Generic_127111-01 ;PID=0x6337;VERSION=9.0.2.3854;API=ODBC;TIMEZONEADJUSTMENT=0
I. 01/29 13:59:59. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 13:59
I. 01/29 13:59:59. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 13:59
I. 01/29 13:59:59. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:59
I. 01/29 13:59:59. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 13:59
I. 01/29 14:00:01. Database server stopped at Tue Jan 29 2013 14:00
I. 01/29 14:08:51. Adaptive Server Anywhere Network Server Version 9.0.2.3854
I. 01/29 14:08:51.
I. 01/29 14:08:51. Copyright (c) 1989-2007 Sybase, Inc.
I. 01/29 14:08:51. Portions Copyright (c) 2002-2007, iAnywhere Solutions, Inc.
I. 01/29 14:08:51. All rights reserved. All unpublished rights reserved.
I. 01/29 14:08:51.  
I. 01/29 14:08:51. This software contains confidential and trade secret information of
I. 01/29 14:08:51. iAnywhere Solutions, Inc.
I. 01/29 14:08:51. Use, duplication or disclosure of the software and documentation
I. 01/29 14:08:51. by the U.S. Government is subject to restrictions set forth in a license
I. 01/29 14:08:51. agreement between the Government and iAnywhere Solutions, Inc. or
I. 01/29 14:08:51. other written agreement specifying the Government's rights to use the
I. 01/29 14:08:51. software and any applicable FAR provisions, for example, FAR 52.227-19.
I. 01/29 14:08:51.  
I. 01/29 14:08:51. iAnywhere Solutions, Inc., One Sybase Drive, Dublin, CA 94568, USA
I. 01/29 14:08:51.
I. 01/29 14:08:51. 32 physical processor(s) detected.
I. 01/29 14:08:51. Per-processor licensing model. The server is limited to use 100 processor(s).
I. 01/29 14:08:51. This server is licensed to:
I. 01/29 14:08:51.     "NetBackup"
I. 01/29 14:08:51.     "Symantec Corporation"
I. 01/29 14:08:51. Running on SunOS 5.10 Generic_127111-01
I. 01/29 14:08:51. Authenticated Server licensed for use with Authenticated Applications only
I. 01/29 14:08:51. 51200K of memory used for caching
I. 01/29 14:08:51. Minimum cache size: 51200K, maximum cache size: 512000K
I. 01/29 14:08:51. Using a maximum page size of 4096 bytes
I. 01/29 14:08:51. Starting database "NBDB" (/catalog/db/data/NBDB.db) at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Starting database "BMRDB" (/catalog/db/data/BMRDB.db) at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Opening dbspace 'EMM_DATA' in file '/catalog/db/data/EMM_DATA.db' for database 'NBDB'
I. 01/29 14:08:51. Opening dbspace 'BMR_DATA' in file '/catalog/db/data/BMR_DATA.db' for database 'BMRDB'
I. 01/29 14:08:51. Opening dbspace 'EMM_INDEX' in file '/catalog/db/data/EMM_INDEX.db' for database 'NBDB'
I. 01/29 14:08:51. Opening dbspace 'BMR_INDEX' in file '/catalog/db/data/BMR_INDEX.db' for database 'BMRDB'
I. 01/29 14:08:51. Transaction log: /catalog/db/data/NBDB.log
I. 01/29 14:08:51. Transaction log: /catalog/db/data/BMRDB.log
I. 01/29 14:08:51. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Database "BMRDB" (BMRDB.db) started at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. This database is licensed for use with:
I. 01/29 14:08:51.     Application: CommonRepository
I. 01/29 14:08:51.     Company: VERITAS
I. 01/29 14:08:51. Performance warning: Server cache size is too small for database "BMRDB"
I. 01/29 14:08:51. Database "NBDB" (NBDB.db) started at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. This database is licensed for use with:
I. 01/29 14:08:51.     Application: CommonRepository
I. 01/29 14:08:51.     Company: VERITAS
I. 01/29 14:08:51. Database server started at Tue Jan 29 2013 14:08
I. 01/29 14:08:51. Trying to start SharedMemory link ...
I. 01/29 14:08:51.     SharedMemory link started successfully
I. 01/29 14:08:51. Trying to start TCPIP link ...
I. 01/29 14:08:51. Starting on port 13785
I. 01/29 14:08:51.     TCPIP link started successfully
I. 01/29 14:08:51. Now accepting requests
I. 01/29 14:28:52. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 14:28
I. 01/29 14:28:52. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 14:28
I. 01/29 14:30:14. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 14:30
I. 01/29 14:30:14. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 14:30
I. 01/29 14:47:00. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 14:47
I. 01/29 14:47:00. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 14:47
I. 01/29 14:48:53. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 14:48
I. 01/29 14:48:53. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 14:48
I. 01/29 15:08:54. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 15:08
I. 01/29 15:08:54. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 15:08
I. 01/29 15:09:56. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 15:09
I. 01/29 15:09:56. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 15:09
I. 01/29 15:28:55. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 15:28
I. 01/29 15:28:55. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 15:28
I. 01/29 15:48:56. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 15:48
I. 01/29 15:48:56. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 15:48
I. 01/29 16:07:19. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:07
I. 01/29 16:07:19. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:07
I. 01/29 16:08:57. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 16:08
I. 01/29 16:08:57. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 16:08
I. 01/29 16:20:11. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:20
I. 01/29 16:20:11. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:20
I. 01/29 16:28:59. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 16:28
I. 01/29 16:28:59. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 16:28
I. 01/29 16:42:36. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:42
I. 01/29 16:42:37. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:42
I. 01/29 16:49:00. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 16:49
I. 01/29 16:49:00. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 16:49
I. 01/29 16:50:30. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:50
I. 01/29 16:50:30. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 16:50
I. 01/29 17:00:30. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:00
I. 01/29 17:00:30. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:00
I. 01/29 17:18:11. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:18
I. 01/29 17:18:11. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:18
I. 01/29 17:24:12. Starting database "utility_db" (utility_db) at Tue Jan 29 2013 17:24
I. 01/29 17:24:12. Database "utility_db" (utility_db) started at Tue Jan 29 2013 17:24
I. 01/29 17:24:12. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:24
I. 01/29 17:24:12. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:24
I. 01/29 17:24:12. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 17:24
I. 01/29 17:24:13. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 17:24
I. 01/29 17:24:14. Database server stopped at Tue Jan 29 2013 17:24
I. 01/29 17:45:01. Adaptive Server Anywhere Network Server Version 9.0.2.3854
I. 01/29 17:45:01.
I. 01/29 17:45:01. Copyright (c) 1989-2007 Sybase, Inc.
I. 01/29 17:45:01. Portions Copyright (c) 2002-2007, iAnywhere Solutions, Inc.
I. 01/29 17:45:01. All rights reserved. All unpublished rights reserved.
I. 01/29 17:45:01.  
I. 01/29 17:45:01. This software contains confidential and trade secret information of
I. 01/29 17:45:01. iAnywhere Solutions, Inc.
I. 01/29 17:45:01. Use, duplication or disclosure of the software and documentation
I. 01/29 17:45:01. by the U.S. Government is subject to restrictions set forth in a license
I. 01/29 17:45:01. agreement between the Government and iAnywhere Solutions, Inc. or
I. 01/29 17:45:01. other written agreement specifying the Government's rights to use the
I. 01/29 17:45:01. software and any applicable FAR provisions, for example, FAR 52.227-19.
I. 01/29 17:45:01.  
I. 01/29 17:45:01. iAnywhere Solutions, Inc., One Sybase Drive, Dublin, CA 94568, USA
I. 01/29 17:45:01.
I. 01/29 17:45:01. 32 physical processor(s) detected.
I. 01/29 17:45:01. Per-processor licensing model. The server is limited to use 100 processor(s).
I. 01/29 17:45:01. This server is licensed to:
I. 01/29 17:45:01.     "NetBackup"
I. 01/29 17:45:01.     "Symantec Corporation"
I. 01/29 17:45:01. Running on SunOS 5.10 Generic_127111-01
I. 01/29 17:45:01. Authenticated Server licensed for use with Authenticated Applications only
I. 01/29 17:45:01. 51200K of memory used for caching
I. 01/29 17:45:01. Minimum cache size: 51200K, maximum cache size: 512000K
I. 01/29 17:45:01. Using a maximum page size of 4096 bytes
I. 01/29 17:45:01. Starting database "NBDB" (/catalog/db/data/NBDB.db) at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. Starting database "BMRDB" (/catalog/db/data/BMRDB.db) at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. Opening dbspace 'BMR_DATA' in file '/catalog/db/data/BMR_DATA.db' for database 'BMRDB'
I. 01/29 17:45:01. Opening dbspace 'EMM_DATA' in file '/catalog/db/data/EMM_DATA.db' for database 'NBDB'
I. 01/29 17:45:01. Opening dbspace 'BMR_INDEX' in file '/catalog/db/data/BMR_INDEX.db' for database 'BMRDB'
I. 01/29 17:45:01. Opening dbspace 'EMM_INDEX' in file '/catalog/db/data/EMM_INDEX.db' for database 'NBDB'
I. 01/29 17:45:01. Transaction log: /catalog/db/data/NBDB.log
I. 01/29 17:45:01. Transaction log: /catalog/db/data/BMRDB.log
I. 01/29 17:45:01. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. Database "NBDB" (NBDB.db) started at Tue Jan 29 2013 17:45
I. 01/29 17:45:01. This database is licensed for use with:
I. 01/29 17:45:01.     Application: CommonRepository
I. 01/29 17:45:01.     Company: VERITAS
I. 01/29 17:45:02. Database "BMRDB" (BMRDB.db) started at Tue Jan 29 2013 17:45
I. 01/29 17:45:02. This database is licensed for use with:
I. 01/29 17:45:02.     Application: CommonRepository
I. 01/29 17:45:02.     Company: VERITAS
I. 01/29 17:45:02. Performance warning: Server cache size is too small for database "BMRDB"
I. 01/29 17:45:02. Database server started at Tue Jan 29 2013 17:45
I. 01/29 17:45:02. Trying to start SharedMemory link ...
I. 01/29 17:45:02.     SharedMemory link started successfully
I. 01/29 17:45:02. Trying to start TCPIP link ...
I. 01/29 17:45:02. Starting on port 13785
I. 01/29 17:45:02.     TCPIP link started successfully
I. 01/29 17:45:02. Now accepting requests
I. 01/29 18:01:43. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 18:01
I. 01/29 18:01:43. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 18:01
I. 01/29 18:11:57. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 18:11
I. 01/29 18:11:57. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 18:11
I. 01/29 18:21:44. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 18:21
I. 01/29 18:21:45. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 18:21
I. 01/29 18:41:02. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 18:41
I. 01/29 18:41:02. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 18:41
I. 01/29 18:41:46. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 18:41
I. 01/29 18:41:46. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 18:41
I. 01/29 19:02:50. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 19:02
I. 01/29 19:02:50. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 19:02
I. 01/29 19:09:08. Cache size adjusted to 395344K
I. 01/29 19:10:08. Cache size adjusted to 481024K
I. 01/29 19:18:22. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 19:18
I. 01/29 19:18:22. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 19:18
I. 01/29 19:38:23. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 19:38
I. 01/29 19:38:23. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 19:38
I. 01/29 19:58:25. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 19:58
I. 01/29 19:58:25. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 19:58
I. 01/29 20:00:16. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 20:00
I. 01/29 20:00:16. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 20:00
I. 01/29 20:45:08. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 20:45
I. 01/29 20:45:08. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 20:45
I. 01/29 20:58:25. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 20:58
I. 01/29 20:58:25. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 20:58
I. 01/29 21:18:26. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 21:18
I. 01/29 21:18:26. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 21:18
I. 01/29 21:37:21. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 21:37
I. 01/29 21:37:21. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 21:37
I. 01/29 21:38:27. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 21:38
I. 01/29 21:38:27. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 21:38
I. 01/29 21:45:26. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 21:45
I. 01/29 21:45:26. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 21:45
I. 01/29 21:55:25. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 21:55
I. 01/29 21:55:26. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 21:55
I. 01/29 21:58:28. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 21:58
I. 01/29 21:58:28. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 21:58
I. 01/29 22:05:27. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 22:05
I. 01/29 22:05:27. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 22:05
I. 01/29 22:18:29. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 22:18
I. 01/29 22:18:29. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 22:18
I. 01/29 22:38:30. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 22:38
I. 01/29 22:38:30. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 22:38
I. 01/29 22:46:11. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 22:46
I. 01/29 22:46:11. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 22:46
I. 01/29 22:55:26. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 22:55
I. 01/29 22:55:27. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 22:55
I. 01/29 22:58:31. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 22:58
I. 01/29 22:58:31. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 22:58
I. 01/29 23:05:26. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:05
I. 01/29 23:05:26. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:05
I. 01/29 23:16:14. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:16
I. 01/29 23:16:14. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:16
I. 01/29 23:18:32. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 23:18
I. 01/29 23:18:32. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 23:18
I. 01/29 23:25:25. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:25
I. 01/29 23:25:25. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:25
I. 01/29 23:36:17. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:36
I. 01/29 23:36:17. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:36
I. 01/29 23:38:33. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 23:38
I. 01/29 23:38:33. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 23:38
I. 01/29 23:45:26. Starting checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:45
I. 01/29 23:45:27. Finished checkpoint of "NBDB" (NBDB.db) at Tue Jan 29 2013 23:45
I. 01/29 23:58:34. Starting checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 23:58
I. 01/29 23:58:34. Finished checkpoint of "BMRDB" (BMRDB.db) at Tue Jan 29 2013 23:58
I. 01/30 00:06:20. Starting checkpoint of "NBDB" (NBDB.db) at Wed Jan 30 2013 00:06

======================================

admin logs.

[Michael_G_Ander]

A shutdown can be caused by lack disk space where the nbdb database is located, think the limit is 10% of the disk size.

The limit can be set in ..\Veritas\NetBackup\var\global\emm.conf with the parameter DISK_MB_AVAILABLE

Regards

Michael

[RamNagalla]

I. 01/29 13:59:06. Connection terminated abnormally

I. 01/29 13:59:06. Disconnected SharedMemory client's AppInfo: IP=10.134.64.12;HOST=bnsdc-mast-01;OS=SunOS 5.
10 Generic_127111-01 ;PID=0x6427;VERSION=9.0.2.3854;API=ODBC;TIMEZONEADJUSTMENT=0
I. 01/29 13:59:06. Connection terminated abnormally
I. 01/29 13:59:06. Disconnected SharedMemory client's AppInfo: IP=10.134.64.12;HOST=bnsdc-mast-01;OS=SunOS 5.
10 Generic_127111-01 ;PID=0x17de;VERSION=9.0.2.3854;API=ODBC;TIMEZONEADJUSTMENT=0
I. 01/29 13:59:06. Connection terminated abnormally
I. 01/29 13:59:06. Disconnected SharedMemory client's AppInfo: IP=10.134.64.12;HOST=bnsdc-mast-01;OS=SunOS 5.
10 Generic_127111-01 ;PID=0x6337;VERSION=9.0.2.3854;API=ODBC;TIMEZONEADJUSTMENT=0

_____________

I. 01/29 14:08:51. Performance warning: Server cache size is too small for database "BMRDB"

_______

so its seems becuase of catch size .

please increase the catch size and see how it works...

[virankumar]

Thanks Nagalla & Mich

[virankumar]

Thanks Marianne as well................

[Marianne]

Ok - you have NBU 6.5.6 installed on Solaris 10.

You can find the engine_A.log file under /var/VRTSvcs/log on any of the cluster nodes.

Please copy this file to engine_A.txt and also copy /var/adm/messages on active node to messages.txt and post as File Attachments.

Let us know what time the cluster restart happened.

PS: Are you aware of the fact that NBU 6.x is no longer supported?

[Marianne]

Oops - I took too long to post.

Thanks to Nagalla who found your issue.

PLEASE upgrade NBU ASAP as NBU 6.5 ran out of support in Oct last year.

[virankumar]

Hi arianne,

Attached the requested log. Upgrade will not possible at these time.

any other option can we have.

[Marianne · Accepted solution]

Unfortunately messages file is too new - starts 21 Jan.

engine_A log also does not contain all the detail that I've expected, for example:

2012/12/03 21:09:21 VCS ERROR V-16-2-13027 (bnsdc-mast-01) Resource(nbu_server) - monitor procedure did not complete within the expected time.

But we cannot see action taken by VCS.

NBU was taken offline using VCS at the time we see the 'unexpected' disconnect in log posted above:

2013/01/29 13:58:15 VCS INFO V-16-1-50135 User admin fired command: hares -offline nbu_server  bnsdc-mast-01  from 10.134.64.12
2013/01/29 14:02:25 VCS INFO V-16-2-13001 (bnsdc-mast-01) Resource(nbu_server): Output of the completed operation (offline) 

And started again a couple of minutes later:

2013/01/29 14:08:49 VCS INFO V-16-1-50135 User admin fired command: hares -online nbu_server  bnsdc-mast-01  from 10.134.64.12

Here we see 'unexpected offline' of VCS:

2013/01/29 17:17:07 VCS ERROR V-16-2-13067 (bnsdc-mast-01) Agent is calling clean for resource(nbu_server) because the resource became OFFLINE unexpectedly, on its own.

But 'clean' process is battling to complete:

2013/01/29 17:18:09 VCS ERROR V-16-2-13006 (bnsdc-mast-01) Resource(nbu_server): clean procedure did not complete within the expected time.
2013/01/29 17:53:56 VCS INFO V-16-1-50133 User patrol has logged in from 127.0.0.1
2013/01/29 20:53:22 VCS ERROR V-16-2-13079 (bnsdc-mast-01) Resource(nbu_server): The last 10 invocations of the clean procedure have failed.
2013/01/29 21:26:08 VCS ERROR V-16-2-13079 (bnsdc-mast-01) Resource(nbu_server): The last 20 invocations of the clean procedure have failed.

2013/01/30 07:21:06 VCS ERROR V-16-2-13079 (bnsdc-mast-01) Resource(nbu_server): The last 200 invocations of the clean procedure have failed.
2013/01/30 07:33:51 VCS INFO V-16-2-13001 (bnsdc-mast-01) Resource(nbu_server): Output of the completed operation (clean) 

Looking for NetBackup processes that need to be terminated.
Killing jnbSA/jbpSA processes...

Looking for Media Manager processes that need to be terminated.


Looking for VxDBMS processes that need to be terminated.
/usr/openv/netbackup/bin/bp.kill_all FORCEKILL SKIPBPCD 2>&1 < /dev/null succeeded.
2013/01/30 07:33:51 VCS INFO V-16-2-13078 (bnsdc-mast-01) Resource(nbu_server) - clean completed successfully after 203 failed attempts.

So - maybe you can tell us what happened between the afternoon of 29 Jan and the morning of 30 Jan?

VCS managed to restart NBU after this:

2013/01/30 07:33:51 VCS ERROR V-16-2-13073 (bnsdc-mast-01) Resource(nbu_server) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 2) the resource.
2013/01/30 07:34:11 VCS INFO V-16-2-13001 (bnsdc-mast-01) Resource(nbu_server): Output of the completed operation (online) 
NetBackup Database Server started.

PLEASE convince your management that NBU as well as SF/HA need to be upgraded as a matter of urgency.

NBU 6.x ran out of support in October last year.
Seems SF/HA 5.0 was installed in Feb 2007 and never patched? Not good.........