Lessons learned on NetBackup Operations Manger Alerts.

[J_H_Is_gone]

NOM has some great Alerts built into it. Just a little setup and you are ready to go. Then you learn a few things.

When setting up the alerts you learn that some items can cover all the masters. Like ‘Frozen media'. I don’t care which server has a frozen a tape, I just want to know that it happened, and the alert will tell me which master. So I create the alert with the NOM server as Context. All good.

But ‘Low Avail Media’ … Well that is different. If you have more then one master, each has their own library, and each library may be of a different size. So on MasterA less then 50 tapes would raise alarms with me, on MasterB you might not even have 50 tapes, so only having 5 left is your issue. So here you have to create two different Alert Policies. One for each master and you set your alarm threshold as needed.

If your location is one that has many many volume pools and you want to monitor tapes in each volume pool then NOM might not help you as you cannot choose by volume pool. (Maybe in the next release? Hint Hint)

So now we moved to the Severity that you can set for the policy. When setting up the policy you can choose: Warning, Critical, Informational, and Major. Seems good.

Until you look over the left on the NOM console and you see the Alert Summary where you get: Informational, Warning, Error, and Critical… They don’t quite match. Seems that Major shows up under Error. (Maybe they will fix that in the next release.)

Now you get to email notification.Does not matter what you set it to, if you setup with an email address you get an email. You can set them up without email notification, which will have the report in NOM but not email you. Like maybe Policy Change. As some locations might not have issues with other people changing the Policy some might. I just want to record that something was change but don’t send me an email on it. The person who made the change can go into NOM and acknowledge the alert and add a comment as to what they changed.

Other things you want to know right away so you set them up with the email account of your Netbackup group. And if you have a BlackBerry you can make them show up as Level1 and make it play music. I have found this very useful when sleeping and we get notified that all the jobs are failing (Not that it has ever happened to me!). That music wakes me up and I know I have to get up and look at the issue.

But now you learn that whenever you ‘Clear’ an alert you also get an email! (Wish I could turn this off). We have made it a policy to NOT clear during the night, but only in the morning. As whoever is working the alert has already sent an email saying they are looking at it. I don’t need to know when they ‘clear’ and delete the alert from NOM.
More then once we have been trying to find out WHY did I get this alert, only 5 minutes latter noticing the ‘(CLEAR)’ in the subject of the email.

Some things you don’t need so it is nice that they are not active from the start. If you don’t use disk for backups no need to have an alert monitoring for it.

But after creating the Alert and you find you want to change it, as it is really easy, give it a check mark and select Change. Then save. The policy has the change but NOM does not seem to know it has changed. You have to remember to Deactivate the policy and then Reactivate it for the change to take effect.

In creating polices, you can see that some can be very specific if you need them to be. I took the ‘High Job Failure Rate” and told it to include all exit status, but exclude status like 1,71,150. So I only get the email when the real failures are high. But here it is all relative. If I am doing a restore that fails and one that works within 1 hour I now have a 50% failure rate for that hour. (Here I wish I could state what time I want this Alert to be active).

I was also able to take the ‘Job Finalized’ alert and set it up to only tell me if something ended with a 233. And because you can customize the name of the alert, the email stands out.

So we learn that Alerts are good. 
Some alerts have to be specific, and some can be general.
Some have to be examined as to how relative they are to the time of day.
Alert Emails have to be viewed very carefully.
And consideration as to who gets the emails and which Alerts need to send them.

So don’t take the alerts at face value, see how you can make them work for you. 
Everybody has different needs. 
Make the Alerts match your needs.

[Karthikeyan_Sun]

Thanks that was Very useful information Guru.

Am Planing to buy BlackBerry to get notified; but that will disturb my Sleep :)

Thats making me to stop :)

[Carlos_V]

What is the solution to eliminate the mail (CLEAR), I don't like this kind of messages.