Understanding the V-16-1-10600 error message for Windows
The hacli command communicates with the high availability daemon (HAD) process for any operations on objects such as clusters, systems, service groups, and resources. If the hacli command cannot communicate with the HAD process, the following error message occurs: VCS ERROR V-16-1-10600 Cannot connect to VCS engine This message is displayed when: The cluster cannot start, because GAB cannot seed the cluster. As a result, HAD cannot start on a node. Veritas Cluster High Availability Engine (Had.exe) hangs in the Starting state. GAB requires a minimum number of nodes be online to successfully seed the cluster. The minimum number of nodes is defined in the C:\Program Files\VERITAS\comms\gab\gabtab.txt file. When all nodes go down and one or more nodes do not come online, it is possible that the minimum node requirement is not reached. Symantec Product Authentication Service (vxat or vxeat depending on version) is having problems even if HAD is in the Running state. To determine the cause of the error: Check if the HAD error message is displayed in the Application Event Viewer, as shown in the figure below: Figure 1 - Command Prompt and Application Event Viewer. Check the Services Control Panel to see if the Veritas High Availability Engine Startup service is in the Starting state. Use one of the following commands to determine the state of the cluster: C:\>hastatus -sum or C:\>hasys –state To fix the error in the case where HAD cannot start: Check the gabtab.txt file for information that defines the seeding requirement: the minimum number of nodes for this cluster that must be available before HAD can start. HAD will not start until the cluster is seeded, even on only a subset of nodes. HAD must be running on a node for cluster operations to commence on that node. If the Veritas High Availability Engine Startup service is in the Starting state, you must manually seed GAB. Enter the following commands: C:\>gabconfig -c –x (This command configures GAB, if needed, and seeds a node or nodes, regardless of the minimum specified in the gabtab.txt file.) C:\>hasys –state (This command displays the current status of all cluster nodes that are online.) Note: Starting with version 6.0.2, the VCS Administrator’s Guide instructs users to use gabconfig -x to manually seed the cluster, while the VCS Administrator’s Guides for 6.0.1 and earlier versions specify using gabconfig -c -x instead. The -x option forces a subset of the required minimum nodes to seed. In a two node cluster that has only one node online, this seeds the one node and allows HAD to start on that node. Once HAD has started on one node, bringing the other node online allows it to join the cluster, as long as the private network communication between nodes is working. If the configuration file is valid, the node should transition from LOCAL_BUILD to RUNNING. However, you must run the hasys –state command repeatedly to see if the status changes, as the command shows a snapshot of the current state. In the case where Symantec Product Authentication Service is having issues, you need to fix and reconfigure the Symantec Product Authentication Service to resolve the HAD connectivity issue. For more information, see: About the high availability daemon (HAD) About the Symantec Product Authentication Service About Group Membership Services and Atomic Broadcast (GAB) VCS ERROR V-16-1-10600 Cannot connect to VCS engine Had.exe, the Veritas Cluster High Availability Engine, hangs in a "Starting" state Symantec Storage Foundation and High Availability documentation for other releases and platforms can be found on the SORT website.Understanding the V-16-1-10600 error message for UNIX
The V-16-1-10600 error message is generated in response to an operation on objects such as clusters, systems, groups, and resources. The error message may display in the engine log when a Symantec Cluster Server (VCS) resource is configured for a Solaris local zone. The error may also occur after a power outage when one node fails to join a cluster after issuing the hastart command. When a VCS resource is configured for a Solaris local zone This instance of the error indicates that when a resource is configured to run inside a Solaris local zone, the VCS commands cannot connect to the VCS engine (HAD) directly. This error is most likely caused by running VCS commands (e.g. halog) when the halogin environment is not set up properly. For example, in the VCS entry point scripts, it is common to call halog (through the VCSAG_LOG_MSG or VCSAG_LOGDBG_MSG functions) to log a message to the engine log. In order for the VCS commands to connect to the VCS engine, you should set up the halogin environment in the local zone. Therefore, Symantec recommends that you use the hazonesetup command to set up a proper halogin environment for the VCS commands to run successfully inside a Solaris local zone. To setup VCS zones using the hazonesetup command, enter the following: # hazonesetup –g servicegroup_name –r zoneres_name –z zone_name –p password –a –s system1,system2, ….systemN Where: servicegroup_name – the name of the service group zoneres_name – the name of the VCS Zone resource to be created zone_name – the name of the Solaris local zone password – a new password for the service group administrator Note that after you run the hazonesetup command, you may find the .vcshost and .vcspwd files in the local zone. The /etc/VRTSvcs/.vcshost file contains the hostname of the global zone. The .vcspwd file contains the encrypted password for the VCS zone user. If the DeleteVCSZoneUser attribute of the Zone resource is set to 1, when VCS takes a zone resource offline, the .vcspwd file is removed. For more information, see: When VCS takes a zone resource offline the .vcspwd file is removed Attributes of zone agent hazonesetup(1M) manual page: Solaris For more information, see: About Solaris Zones Configuring VCS in zones When one node fails to join a cluster after a power outage This instance of the error often occurs with the VCS WARNING V-16-1-11046 Local system not available error. This error occurs when a power outage causes VCS to not shutdown properly. If this error occurs, you must manually start the Global Atomic Broadcast (GAB) process and other processes using the following commands: # svcadm enable gab # svcadm enable llt # svcadm enable vcs For more information about manually starting the GAB, Low Latency Transport (LLT), and VCS, see: Starting LLT, GAB, and VCS after manual configuration Symantec Storage Foundation and High Availability documentation for other releases and platforms can be found on the SORT website.
CFS NFS configuration
Hello team, I'm trying to configure a CFS NFS HA service, so I've executed this command on the cluster: cfsshare config -p nfs apbasedg locks /locks and then configured all the shares. The cfsnfs_locks volume was already created and mounted on the vcs, but now it's not able to mount it anymore: cfsshare display VCS WARNING V-16-1-10260 Resource does not exist: cfsnfs_locks CNFS metadata filesystem : Protocols Configured : none #RESOURCE MOUNTPOINT PROTOCOL OPTIONS scs-ESP-nfsshare_ESPgoldclient /ESP/goldclient NFS scs-ESP-nfsshare_ESPusrsaptrans /ESP/usr/sap/trans NFS scs-ESP-nfsshare_sapmntESP /sapmnt/ESP NFS scs-ESP-nfsshare_userdataESP /userdata/ESP NFS #cfsmntadm display List of mount points registered with cluster-configuration but not associated with any node : /locks # vxdisk -g apbasedg list DEVICE TYPE DISK GROUP STATUS vmdk0_10 auto:sliced apbasedg_0 apbasedg online shared this is the volume where the cfsnfs_locks volume is available v cnfs_locks fsgen ENABLED 2097152 - ACTIVE - - pl cnfs_locks-01 cnfs_locks ENABLED 2097152 - ACTIVE - - but I'm not able to mount the volume: # cfsmount /locks Error: V-35-33: cfsmount: Cluster mount /locks not associated with any node It worked at first, but now no more. Can you please help Thanks
NFS agent for CFS share
Dear All, I have setup CFSshare. I can see under cfsshare display, but if I run "cfsshare share /emm2 rw" is NOT sharing but from the Java Console of the cluster. The NFS resource has got a question MARK. So I am really not too what to do now., root@node_1# cfsmount /emm5 rw Mounting... [/dev/vx/dsk/mobile_dg/mobile_vol_4] already mounted at /emm5 on node_0 [/dev/vx/dsk/mobile_dg/mobile_vol_4] already mounted at /emm5 on node_1 root@node_1 # cfsshare share /emm5 Warning: V-35-465: Resource [share3] is not online on system [node_0]. Warning: V-35-465: Resource [share3] is not online on system [node_1]. root@node_1 # cfsshare display CNFS metadata filesystem : /locks Protocols Configured : NFS #RESOURCE MOUNTPOINT PROTOCOL OPTIONS share1 /emm2 NFS rw share2 /emm3 NFS rw share3 /emm5 NFS rw root@node_1 # Regards, Thokozani
SFHA Solutions 6.0.1: Understanding and responding to common VCS resource issues
When you work with Veritas Cluster Server (VCS) resources, you may encounter one of the following situations: The resource goes offline unexpectedly. The command to bring the resource online is rejected. The monitor procedure does not complete within the expected time. For background information about VCS resources, see: About resources and resource dependencies Categories of resources About resource types When a resource goes offline A resource may go offline unexpectedly and outside of VCS control. By default, VCS detects this situation and automatically fails over the resource. Error message V-16-2-13067 is written to the agent log or engine log file: Agent is calling clean for resource(%s) because the resource became OFFLINE unexpectedly, on its own For more information on the agent log and engine log, see: VCS message logging When you receive this error, no action is required. VCS takes action on the faulted resource. However, if you do not want VCS to take action when the resource goes offline, increase the value of the ToleranceLimit attribute. This attribute defines how many times the monitor entry point reports the resource offline before it declares the resource faulted. Note: If you set this value to a higher number, it could delay detecting a genuinely faulted resource. For more information, see: About the ToleranceLimit attribute VCS behavior when a resource goes offline When the command to bring the resource online is rejected VCS can bring a resource online only if it can probe the resource and determine the resource’s initial state. If VCS cannot probe the resource before you issue the hares –online command, error message V-16-1-10283 is written to the engine log file: Resource has not been probed on system system_name To resolve this issue, you can: Wait until the agent probes the resource before you enter the hares –online command. Check the agent log and engine log to identify and correct any reported issues by the Monitor agent function. A very common issue is that the resource is in the UNKNOWN state due to a configuration issue. If you are a Solaris user running Storage Foundation and High Availability (SFHA) 5.0 MP3, the following TechNote describes the error in the context of your environment: TechNote 88123 - Resource has not been probed To learn more about the Monitor agent function and probing resources, see: About agent functions Probing a resource hares (1M) manual pages: AIX HP-UX Linux Solaris When the Monitor agent function does not complete within the expected time The Monitor agent function tests the status of a resource to determine whether it is online or offline. If the Monitor agent function does not complete within the expected time, and there are no other issues with the system, error message V-16-2-13027 is written to the agent log or engine log: Resource(%s) - monitor procedure did not complete within the expected time For information on correcting this error, see: TechNote 70442 Veritas Storage Foundation and High Availability documentation for other releases and platforms can be found on the SORT website.
SFHA Solutions 6.0.1: Troubleshooting the Veritas Command Server service
The Veritas Command Server (CmdServer) service is used to handle the command queries submitted from the client to the server and vice versa. When a Veritas Cluster Server (VCS) cluster is in the secure mode, the CmdServer service is enabled automatically on all nodes in the cluster. However, the CmdServer service may be unable to start if: Port 14150 on the node is blocked by the firewall. This port is used by the CmdServer to communicate with the clients. For a complete list of the ports and services used by Veritas Storage Foundation and High Availability (SFHA), see: About SFHA services and ports The node is in an IPv6 environment. This only applies to VCS release 5.1SP1 on AIX, HP-UX, Linux and Solaris platforms. If port 14150 is blocked, use the following steps to start the CmdServer service: Add an exception for port 14150 in the firewall settings. On AIX, HP-UX, Linux, or Solaris, start the CmdServer service: /opt/VRTSvcs/bin/CmdServer On Windows, open the Services control panel, and start the CmdServer service. Select Start > Run > services.msc. This solution is provided in UMI V-16-10-13. If the node is in an IPv6 environment, use the following steps to start the CmdServer service: Add the following line in the /etc/hosts file of the node: ::1 localhost On AIX, HP-UX, Linux, or Solaris, start the CmdServer service: /opt/VRTSvcs/bin/CmdServer On Windows, open the Services control panel, and start the CmdServer service. Select Start > Run > services.msc. For information about secure services, see: About security services Enabling and disabling secure mode for the cluster Veritas™ Cluster Server Administrator's Guide Veritas Storage Foundation and High Availability documentation for other releases and platforms can be found on the SORT website.
Veritas Installer: Understanding the V-9-40-3050 error message: Mismatch in CPU clock frequency
Veritas Installer: Understanding the V-9-40-3050 error message: Mismatch in CPU clock frequency The V-9-40-3050 message is displayed because the processors on all the nodes within a cluster do not have the same clock frequency. For the solution, see: UMI V-9-40-3050 This is an error code related to the SF Oracle RAC product installer. The installer enables you to install and configure the product, verify pre-installation requirements, and view the product's description. For more information, see: About the Veritas Installer Installing SF Oracle RAC using the Veritas script-based installation program SF Oracle RAC cluster setup modelsVxVM: Understanding the V-5-1-587 error message
The V-5-1-587 error message indicates that a disk import failed. The second portion of the message briefly describes the reason for the error. Depending on the reason, you can either clear the locks or force the disk group to be imported to solve the issue. UMI V-5-1-587 Clearing the locks Second Part of the Message Description Disk is in use by another host Indicates the disk group that you tried to import has disk(s) moved from a crashed system, or from a system that fails to detect the group before the move. You must clear the locks stored on the disks. No valid disk found containing disk group Indicates the disk group does not contain any valid disks (but it might contain disks). A mismatch between the host ID in the configuration copies and that stored in the /etc/vx/volboot file may make the disks invalid. To solve these issues, do one of the following: Clear the locks on a specific set of devices: # vxdisk clearimport devicename ... Clear the locks during import: # vxdg -C import diskgroup Warning: When you use the vxdisk clearimport or vxdg -C import command to clear the locks, make sure that the disks in this disk group are not in use by another host. If the disks are in use by another host, clearing the locks allows the disks to be accessed at the same time from multiple hosts. This can result in data corruption. Forcing the disk group to be imported Second Part of the Message Description Disk for disk group not found Indicates the disk(s) is not accessible after the last successful import of the disk group. This is a recoverable error. Disk group has no valid configuration copies Indicates a fatal error. You must repair the hardware or create a new disk group, and recover the disk group configuration and data. To solve these issues, force the disk group to be imported: # vxdg -f import diskgroup The -f option forces an import if one or more disks are currently unusable or inaccessible. When you use the -f option to force the import of an incomplete disk group, it counts as a successful import. The next time this disk group is imported, the –f option is no longer required. The import completes successfully as long as all the disks in this disk group are accessible. Warning: Be careful when you use the -f option. It can cause the same disk group to be imported twice from different sets of disks. Consequently, the disk group configuration may become inconsistent. You can also use the vxdiskadm utility to clear the locks or force the disk groups to be imported: Select the menu item “Enable access to (deport) a disk group” to deport a disk group. Select the menu item “Enable access to (import) a disk group” to import a disk group. The vxdiskadm import operation checks for host import locks and prompts to see if you want to clear any that are found. It also starts volumes in the disk group. For more information on importing the disk groups, see: Importing a disk group Forcibly importing a disk group Troubleshooting issues with importing disk groups vxdg (1M) 6.0.1 manual pages: AIX HP-UX Linux Solaris vxdiskadm (1M) 6.0.1 manual pages: AIX HP-UX Linux Solaris Veritas Storage Foundation and High Availability documentation for other releases can be found on the SORT website.VxFS: Understanding the V-3-20837 warning message
The Veritas File System (VxFS) fsck command examines VxFS file systems for consistency. Because VxFS file systems record pending file system updates in an intent log, the fsck command typically replays the intent log instead of doing a full structural file system check. When you run the fsck command to repair a file system, the following warning message may appear: UX:vxfs fsck: WARNING: V-3-20837: file system had I/O error(s) on user data. The message suggests that there were one or more I/O errors on user data. You do not have to take action on this message. However, you should note that the fsck command found data errors. Frequent data errors are often the first sign that a spinning disk is nearing the end of its life. Most modern disk drives include Self-Monitoring, Analysis and Reporting Technology (SMART) logic for anticipating failures. The fsck command can also be used for: Ensuring that file systems are clean Recovering a disabled file system Forcibly removing a volume in a multi-volume file system fsck_vxfs (1M) manual pages: AIX HP-UX Linux Solaris fsck command use cases and fsck_vxfs (1M) manual pages for other releases can be found on the SORT website.SORT Mobile: View error code descriptions and solutions on your iPhone, iPad, or iPod Touch
The Symantec Operations Readiness Tools mobile application (SORT Mobile), enables you to look up Symantec error codes on your iPhone, iPad, or iPod Touch. You can now look up an error code anywhere: in a customer meeting, on the road, or at lunch with a client. In addition to Error Code Lookup, SORT Mobile supports these key features from the SORT website: Symantec Performance Value Unit (SPVU) Calculator. Perform SPVU and server tier calculations for the Storage Foundation product family. Patch Lookup. Look up patches based on the incident ID, patch name, or patch ID. Installation and Upgrade Checklist. Look up product requirements including memory, disk space, and architecture (Storage Foundation and NetBackup only). News and Alerts. Display the latest SORT news (Storage Foundation and NetBackup only). SORT Mobile runs natively on iPhones, iPads, and iPod Touches running iOS 5.0 or later. To learn more about SORT Mobile and to download it to your iPhone, iPad, or iPod Touch, go to: http://sort.symantec.com/mobile