Forum Discussion

taesun's avatar
taesun
Level 3
14 years ago

What's the messages ?

hello.

I have SFCFS5.0 on hpux 11.31

I saw messages : 

 

/opt/VRTSvcs/bin/CFSMount/monitor[47]: 4657 Killed
vx_ha_read[37]: 4669 Killed 
 
What's  mean the "4669 Killed"  ?
 
VCS engine_A.log
===================================================================================
skip ...
2010/11/18 17:33:01 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount3): Output of the completed operation (monitor) 
/opt/VRTSvcs/bin/CFSMount/monitor[47]: 4657 Killed
vx_ha_read[37]: 4669 Killed
/opt/VRTSvcs/bin/CFSMount/monitor[55]: 4676 Killed
2010/11/18 17:33:01 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount4): Output of the completed operation (monitor) 
/opt/VRTSvcs/bin/CFSMount/monitor[47]: 4675 Killed
2010/11/18 17:33:01 VCS ERROR V-16-2-13067 (eachap2t) Agent is calling clean for resource(cfsmount4) because the resource became OFF
LINE unexpectedly, on its own.
2010/11/18 17:33:01 VCS ERROR V-16-2-13067 (eachap2t) Agent is calling clean for resource(cfsmount3) because the resource became OFF
LINE unexpectedly, on its own.
2010/11/18 17:33:02 VCS NOTICE V-16-10021-5511 (eachap2t) CFSMount:cfsmount4:clean:clean unexpected offline : Mount Point : /nbsftp
2010/11/18 17:33:02 VCS NOTICE V-16-10021-5511 (eachap2t) CFSMount:cfsmount3:clean:clean unexpected offline : Mount Point : /nbsrun
2010/11/18 17:33:05 VCS NOTICE V-16-10021-5504 (eachap2t) CFSMount:cfsmount4:clean:Force Umount : Mount Point : /nbsftp
2010/11/18 17:33:06 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount4): Output of the completed operation (clean) 
/opt/VRTSvcs/bin/CFSMount/clean[106]: test: argument expected
2010/11/18 17:33:06 VCS INFO V-16-2-13068 (eachap2t) Resource(cfsmount4) - clean completed successfully.
2010/11/18 17:33:07 VCS NOTICE V-16-10021-5504 (eachap2t) CFSMount:cfsmount3:clean:Force Umount : Mount Point : /nbsrun
2010/11/18 17:33:07 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount3): Output of the completed operation (clean) 
/opt/VRTSvcs/bin/CFSMount/clean[106]: test: argument expected
2010/11/18 17:33:07 VCS INFO V-16-2-13068 (eachap2t) Resource(cfsmount3) - clean completed successfully.
2010/11/18 17:33:07 VCS INFO V-16-1-10307 Resource cfsmount4 (Owner: unknown, Group: cfs) is offline on eachap2t (Not initiated by V
CS)
2010/11/18 17:33:08 VCS INFO V-16-6-15004 (eachap2t) hatrigger:Failed to send trigger for resfault; script doesn't exist
2010/11/18 17:33:08 VCS INFO V-16-1-10307 Resource cfsmount3 (Owner: unknown, Group: cfs) is offline on eachap2t (Not initiated by V
CS)
2010/11/18 17:33:09 VCS INFO V-16-6-15004 (eachap2t) hatrigger:Failed to send trigger for resfault; script doesn't exist
skip ...
 
 
 
SYSTEM LOG
==================================================================================
skip ...
Nov 18 14:40:52 eachap2t su: + 19 root-jeuss
Nov 18 16:30:54 eachap2t mountd[4846]: Cannot stat /etc/dfs/sharetab: No such file or directory
Nov 18 17:33:01 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13067 Thread(5) Agent is calling clean f
or resource(cfsmount4) because the resource became OFFLINE unexpectedly, on its own.
Nov 18 17:33:01 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13067 Thread(3) Agent is calling clean f
or resource(cfsmount3) because the resource became OFFLINE unexpectedly, on its own.
Nov 18 17:33:01 eachap2t Had[13145]: VCS ERROR V-16-1-13067 (eachap2t) Agent is calling clean for resourc
e(cfsmount4) because the resource became OFFLINE unexpectedly, on its own.
Nov 18 17:33:01 eachap2t Had[13145]: VCS ERROR V-16-1-13067 (eachap2t) Agent is calling clean for resourc
e(cfsmount3) because the resource became OFFLINE unexpectedly, on its own.
Nov 18 17:33:05 eachap2t vmunix: Phase 9 - /dev/vx/dsk/eachapvg11/lvol4 - Set Primary nodeid to 0
Nov 18 17:33:06 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13068 Thread(5) Resource(cfsmount4) - cl
ean completed successfully.
Nov 18 17:33:06 eachap2t vmunix: Phase 9 - /dev/vx/dsk/eachapvg11/lvol3 - Set Primary nodeid to 0
Nov 18 17:33:07 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13068 Thread(3) Resource(cfsmount3) - cl
ean completed successfully.
Nov 18 22:30:54 eachap2t mountd[4846]: Cannot stat /etc/dfs/sharetab: No such file or directory
business continuity
Cluster Server
Clustering
Error messages
High availability
HP-UX
  • Gaurav_S's avatar
    Gaurav_S
    14 years ago
    Hello, I believe your monitor script was killed when it was actually executing to check the resource status.. This script was killed because resource went offline outside to VCS ... now I am not sure that it is by design that as soon clean was called it cleaned up the monitor process or there is something else... Do you know what was the actual cause of above issue ? Did you see any storage issue, luns went offline or diskgroup was deported /disabled or filesystems were unmounted manually etc ? Gaurav
  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    14 years ago

    That looks to me like PID of process ...

    can you attach the /opt/VRTSvcs/bin/CFSMount/monitor  script here ? Unfortunately I don't have a HP server with CFS installed to check the same...

     

    Gaurav

  • taesun's avatar
    taesun
    Level 3
    14 years ago

     

    [root@TEST-eachap1t]/>$ cat /opt/VRTSvcs/bin/CFSMount/monitor
    #!/usr/bin/ksh
    # @(#)src/common/vxfs/CFSMount/scripts/monitor  1.14 03/28/06 13:37:49 - 1999-2004
    #ident "@(#)vcscvm:src/common/vxfs/CFSMount/scripts/monitor  1.14"
    #
    # Copyright (c) 1999-2004 Symantec Corporation.
    # All rights reserved.
    #
    # THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS OF
    # SYMANTEC CORPORATION.  USE, DISCLOSURE OR REPRODUCTION IS PROHIBITED
    # WITHOUT THE PRIOR EXPRESS WRITTEN PERMISSION OF SYMANTEC CORPORATION.
    #
    # The Licensed Software and Documentation are deemed to be "commercial
    # computer software" and "commercial computer software documentation"
    # as defined in FAR Sections 12.212 and DFARS Section 227.7202.
    #
     
    #
    # Verify that the file system specified by the MountPoint attribute
    # is mounted as a cluster mount.
    #
    # This should be re-written in C++ so that the VCS daemon can
    # run it more efficiently.
    # We want to use the VX_CFS_MOUNT_INFO ioctl, which means we
    # want both vxfs/3 header files and thor/0.2 header files.
    # Refer to incident 41878.
    # A fallback would be to just use "stat", which tells us if
    # "something" is mounted, but doesn't tell us if a CFS resource
    # is mounted.
    #
    # $1 = resource name
    # $2 = MountPoint
    # $3 = BlockDevice
    # $4 = MountOpt
    #
     
    . /opt/VRTSvcs/bin/CFSMount/CFSMount.env
    . /opt/VRTSvxfs/cfs/lib/tr_lib.sh
    VCSHOME="${VCS_HOME:-/opt/VRTSvcs}"
     
    CFSRES="$1"
    MOUNTPOINT="$2"
     
    HARES=${VCSHOME}/bin/hares
    . $VCSHOME/bin/ag_i18n_inc.sh
    VCSAG_SET_ENVS $CFSRES
     
    MOUNTTYPE=`$FSCLUSTADM mounttype $MOUNTPOINT 2>/dev/null`
    echo "$4" | egrep "snapof=" > /dev/null 2>&1
    if [ $? -eq 0 ] && [ "$MOUNTTYPE" = "local" ]
    then
        exit 110;
    fi
     
    primary_name=`vx_ha_read "$HARES -value $CFSRES Primary"`
    current_primary=`$FSCLUSTADM -v showprimary $MOUNTPOINT 2>/dev/null`
     
    if [ "$MOUNTTYPE" = "primary" ]
    then
            if [ "$primary_name" != "$current_primary" ]
            then
                    vx_ha_write "$HARES -modify $CFSRES Primary \
                             "$current_primary""
            fi
    fi
     
    if [ "$MOUNTTYPE" = "primary" ] || [ "$MOUNTTYPE" = "secondary" ]
    then
            exit 110;
    else
            if [ "$MOUNTTYPE" = "disabledFs" ]
            then
                    VCSAG_LOG_MSG "E" "File System Disabled: \
                                    MountPoint : $MOUNTPOINT" \
                                    5507  "$MOUNTPOINT"
            fi
            exit 100;
    fi
  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    14 years ago
    Hello, I believe your monitor script was killed when it was actually executing to check the resource status.. This script was killed because resource went offline outside to VCS ... now I am not sure that it is by design that as soon clean was called it cleaned up the monitor process or there is something else... Do you know what was the actual cause of above issue ? Did you see any storage issue, luns went offline or diskgroup was deported /disabled or filesystems were unmounted manually etc ? Gaurav