Forum Discussion

allaboutunix's avatar
10 years ago

Resource faults issue

Hi Team, Many times we face resource faults alerts on our AIX server,but when we check it shows all are running fine.  Also, when checked logs it shows not (initiated by VCS) alerts. Our mai...
  • Wally_Heim's avatar
    10 years ago

    Hi allaboutunix,

    From the log snippet that you provided it is you NIC resource (csgnic) in the ClusterService group that is going faulted from time to time that is causing problems for the proxy resources that point to it.

    NIC resources are persistent resources and show the state of the NIC hardware that they are configured to monitor.  If the NIC resource probes as online for persistent resources the fault is cleared automatically and the resource state is shown as online.

    The proxy resource mirrors the state of another resource that it is configured to monitor.  In this case, I would guess that proxy_DB_INSPRD is configured to mirror the csgnic resource state.  So in this case, its faults would also be automatically cleared when the csgnic resource probes as online.

    You can check the NIC_A.log file on the clnibm014 node for issues around 2015/04/21 12:57:32.  This might give you a better idea as to why the csgnic resource is having issues at that time.

    This also seems to be a temporary issue and one that resources itself in a short amount of time.  Possibly system load related??

    Anyway, you can try increasing the ToleranceLimit attribute on the csgnic to a value of say 1 or 2 (default is zero) to try and tolerate a breif period of incorrect probing state of the nic resource.  But this is just a way to hide these events from happening and not really a way to fix the underlying issue.

    Thank you,

    Wally

  • mikebounds's avatar
    10 years ago

    Agree with Wally that NIC seems to be issue and setting ToleranceLimit may help, but you should also make sure you have set NetworkHosts on your NIC resource.  Setting this attribute is best practice for any interface, but it is required for virtual interfaces on AIX which you seem to be using - see extract from bundled agents guide:

    NetworkHosts

    List of hosts on the same network that are pinged to determine if the
    network connection is alive. Enter the IP address of the host, instead
    of the host name, to prevent the monitor from timing out. DNS lookup
    causes the ping to hang. If more than one network host is listed, the
    monitor returns ONLINE if at least one of the hosts is reachable.
    If you do not specify network hosts, the monitor tests the NIC by
    sending pings to the broadcast address on the NIC.
    For a virtual device, you must configure the NetworkHosts attribute.
    Symantec recommends configuring more than one host to take care
    of the NetworkHost itself failing.
    Type and dimension: string-vector
    Example: { "166.96.15.22", "166.97.1.2" }

    Mike

  • allaboutunix's avatar
    10 years ago

    Hi Heim/Mark,

    Is there any way to fix it permanently?

    We have checked and didn't fing any logs of year 2015, all are older one's.

  • Marianne's avatar
    10 years ago

    We have checked and didn't fing any logs of year 2015, all are older one's.

    This means that there was probably a real issue at hardware level that got fixed in the meantime.

    Remember that VCS is reporting faults/issues, not causing it.
    Troubleshooting of NIC issues must be done at OS and hardware level.