Netbackup Appliance 5220 2.5.2 Alerts Email

[thejollyboys]

Hi,

How can I stop the really annoying alerts from being emailed from the Netbackup Appliance followinf upgrading to 2.5.2 please?

I get something like the following every few minutes.

I have acknowleges and clear acknowledged the alerts but still they get emailed?

Compute Node fred

Time Monitoring Ran: Mon Mar 11 21:39:03 2013 UTC

+--------------------------------------------------------------------------------------------+

|                                      RAID Infomation                                       |

|+------------------------------------------------------------------------------------------+|

||ID|Name|Status |Capacity| Type |Disks|Write Policy|Enclosure|HotSpare | State |Acknowledge||

||  |    |       |        |      |     |            |   ID    |Available|       |           ||

||--+----+-------+--------+------+-----+------------+---------+---------+-------+-----------||

||  |    |       |        |      |0 1 2|            |         |         |       |           ||

||3 |VD-0|Optimal|4.541TB |RAID-6|3 4 5|WriteThrough|0        |yes      |Warning|No         ||

||  |    |       |        |      |6    |            |         |         |       |           ||

|+------------------------------------------------------------------------------------------+|

|                              Adapter Information                                           |

|+-----------------------------------------------------------------------------+             |

||  |                     |Adapter| BBU  |BBU Learn | BBU  |       |           |             |

||ID|    Adapter model    |Status |Status|  Cycle   |charge| State |Acknowledge|             |

||  |                     |       |      |  active  |      |       |           |             |

||--+---------------------+-------+------+----------+------+-------+-----------|             |

||  |Integrated Intel(R)  |       |      |          |      |       |           |             |

||1 |RAID Controller      |OK     |OK    |No        |57 %  |Warning|No         |             |

||  |SROMBSASMP2          |       |      |          |      |       |           |             |

|+-----------------------------------------------------------------------------+             |

+--------------------------------------------------------------------------------------------+

Thanks.

[Andrew_Madsen]

I have a ticket opened on this. Mine say "

Appliance is seeing a high disk IO service time at this moment. There are <0> backup jobs running currently, and a high disk service time may mean a bottleneck at disk level. You can ignore this message if response time for backups is acceptable. This e-mail notification can be acknowledged from CLISH:

Main->Settings->Alerts AcknowledgeErrors

Node does not have any errors."

Yet nothing is going on.

[thejollyboys]

I also have that message too Wolfsbane.

[AntBar]

Hi, I have exactly this message too...

[lburnside]

I have the same message and the same annoying alerts.  I have been told that this is a 2.5.2 problem and that engineering is working on a fix for it.

[Mark_Solutions]

Maybe try renaming the /opt/NBUAppliance/scripts/CallHomeDataGather.sh or callhome.pl or hash out the cron job (crontab -e) so that it doesnt actually run the check every 15 minutes - just until they have a fix available.

Hope this helps

[BackupBunny]

Finally got the upgrade installed - and now this email starts spamming! Very annoying at the lack of attention to detailed testing.

[Andrew_Madsen]

An EEB is in testing and should be out this week or next.

[Mark_Solutions]

Did anyone try renaming the files as i suggested or changing the cron job? I would be interested to know if it resolved it for you while waiting for the EEB

[Andrew_Madsen]

I did not. Mine eventually quieted down to one or two a day.

[AntBar]

Nope. I prefer to wait official support response...

[MortenSeeberg]

Had the issue as well, strangely enough only one of the customers 2 appliances. Mails died down after approx. 1-1½ day.

[EnriquePereira]

It is happening the same to me. I upgraded 4 appliances and two of them (at this moment) are sending the mail.

[DMF-UK]

Any update on when the EEB will be released?

[Andrew_Madsen]

Last thing I got a couple of days ago from Symantec:

Just got some updated info on this.

Their findings thus far are that this mostly affects the OS disk and would not have any impact on backup/restore performance. They are triggered if the disk wait (queue) is higher than 25ms for 30+ minutes for any disk, but testing so far has indicated the issue is primarily OS disks. Background tasks such as rebuilds, patrol reads, and consistency checks can lead to this behavior.

I know they have developed and tested some fixes, but haven't released them as of today. Their suggested workaround for now is to simply acknowledge the alerts. Also, if the alert condition clears itself (ie - the wait time drops back down) then the alert should automatically clear itself.

We can also use this command to view the status to confirm this is the issue:

iostat -kx 300 2

You would see which device has the high 'await' time in ms.

I don't have an ETA for the permanent fix yet, but as soon as I have new info, I'll get it to you.

[MortenSeeberg]

Not sure one of my customers agree :-), after receiving 400 emails in 2-3 days from the 2 appliances I turned of the job in crontab....

[Andrew_Madsen]

So you can only hope there is not some other issue with the device like a drive loss. You might just remove the local user email from the support settings so at least the device only pesters Symantec.  

[EnriquePereira]

It seems that the EEB has been released. I am trying to install it.

[EnriquePereira]

Downloaded. ET3116615.

I will install tomorrow at the customer.

[thejollyboys]

I've installed the patch and we still have the exact same problem?

Have a case open but no progress fast.