Veritas Cluster Server - Unable to start vxcached

[H_A_N_B_U]

Hi to all,

I tried to add node on the existig VCS cluster using /opt/VRTS/install/installsfha601 -addnode command. All of the services started except on the vxcached. Below is my environment:

• OS: Solaris 10
• SFHA Version: 6.0.3

Please see vxcached logs below:

0 17:49:09 ## Begin prestart checking for vxcached on php-sparct41-dp9-app03
0 17:49:09 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C; LANG=C; export LC_ALL LANG; /usr/bin/ps -ef | /usr/bin/grep 'vxcached' | /usr/bin/grep -v grep | /usr/bin/grep -v defunct 2>&1; echo _CMD_EXIT_=\$?" 2>&1
0 17:49:09 cmd exit=1 (duration: 0 seconds)
0 17:49:09 ## The prestart checking return '0'
0 17:49:09 ## Begin starting vxcached on php-sparct41-dp9-app03
0 17:49:09 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C LANG=C /usr/bin/nohup /usr/lib/vxvm/bin/vxcached root > /dev/null 2>&1 < /dev/null &" 2>&1
0 17:49:09 cmd exit=0 (duration: 0 seconds)
0 17:49:11 ## The start_sys() return '1'
0 17:49:11 ## Begin poststart checking for vxcached on php-sparct41-dp9-app03
0 17:49:11 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C; LANG=C; export LC_ALL LANG; /usr/bin/ps -ef | /usr/bin/grep 'vxcached' | /usr/bin/grep -v grep | /usr/bin/grep -v defunct 2>&1; echo _CMD_EXIT_=\$?" 2>&1
0 17:49:11 cmd exit=1 (duration: 0 seconds)
0 17:49:11 ## The poststart checking return '0'
0 17:49:21 ## Sleep 10, and do poststart checking again
0 17:49:21 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C; LANG=C; export LC_ALL LANG; /usr/bin/ps -ef | /usr/bin/grep 'vxcached' | /usr/bin/grep -v grep | /usr/bin/grep -v defunct 2>&1; echo _CMD_EXIT_=\$?" 2>&1
0 17:49:21 cmd exit=1 (duration: 0 seconds)
0 17:49:21 ## The poststart checking return '0'
0 17:49:21 ## vxcached is not successfully started on php-sparct41-dp9-app03
0 17:49:21 ## The proc_start_sys() return '0'
0 17:50:04 ## Starting vxcached on php-sparct41-dp9-app03
0 17:50:04 ## Begin prestart checking for vxcached on php-sparct41-dp9-app03
0 17:50:04 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C; LANG=C; export LC_ALL LANG; /usr/bin/ps -ef | /usr/bin/grep 'vxcached' | /usr/bin/grep -v grep | /usr/bin/grep -v defunct 2>&1; echo _CMD_EXIT_=\$?" 2>&1
0 17:50:04 cmd exit=1 (duration: 0 seconds)
0 17:50:04 ## The prestart checking return '0'
0 17:50:04 ## Begin starting vxcached on php-sparct41-dp9-app03
0 17:50:04 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C LANG=C /usr/bin/nohup /usr/lib/vxvm/bin/vxcached root > /dev/null 2>&1 < /dev/null &" 2>&1
0 17:50:04 cmd exit=0 (duration: 0 seconds)
0 17:50:06 ## The start_sys() return '1'
0 17:50:06 ## Begin poststart checking for vxcached on php-sparct41-dp9-app03
0 17:50:06 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C; LANG=C; export LC_ALL LANG; /usr/bin/ps -ef | /usr/bin/grep 'vxcached' | /usr/bin/grep -v grep | /usr/bin/grep -v defunct 2>&1; echo _CMD_EXIT_=\$?" 2>&1
0 17:50:06 cmd exit=1 (duration: 0 seconds)
0 17:50:06 ## The poststart checking return '0'
0 17:50:16 ## Sleep 10, and do poststart checking again
0 17:50:16 cmd /usr/bin/rsh php-sparct41-dp9-app03 "LC_ALL=C; LANG=C; export LC_ALL LANG; /usr/bin/ps -ef | /usr/bin/grep 'vxcached' | /usr/bin/grep -v grep | /usr/bin/grep -v defunct 2>&1; echo _CMD_EXIT_=\$?" 2>&1
0 17:50:16 cmd exit=1 (duration: 0 seconds)
0 17:50:16 ## The poststart checking return '0'
0 17:50:16 ## vxcached is not successfully started on php-sparct41-dp9-app03
0 17:50:16 ## The proc_start_sys() return '0'

Any help will appreciate.

Paul

[frankgfan]

The issue is vxdmpadm related, not cvs.

The Veritas Volume Manager cache daemon, vxcached, is invoked by the VxVM kernel to monitor the cache objects that are used by space-optimized snapshots. When usage of a cache volume by space-optimized snapshots reaches the high watermark level that has been set for the cache, vxcached automatically grows the cache volume if this is required and configured.

What you can do are

1. Check and make sure each node has the same Veritas software nstalled (same version same patch level)

2. Each node has the same Veritas license applied

3. All the nodes can access all the share storage (with the same r/w permissions)

4. Run the command below and post the output here

hastatus -sum

gabconfig -a

vxdctl license  < n an existing node as well as on the newly added node

Frank

[frankgfan]

My apology for the typo in previous replay.

hate auto-correction

vxdmpadm meant to be vxvm.

need to turn off the auto type correction on my ipad