VxVM vxconfigd ERROR V-5-1-8561 ddl_fill_dmp_info:

[Florin_M_]

Hi community,

Recently there was a project to add a new EMC AX4-5 enclosure on top of the existing AX4-5 Enclosure (a new array of disks).

Current cluster is a 6 blade cluster where all blades share the same LUNs (partitions) using VCSFS and PowerPath.

After adding the new LUNs and then running on the master blade of the cluster and scanning the devices (devfsadm -Cv) then the following error was encountered:

# vxdctl enable
VxVM vxdctl ERROR V-5-1-307 vxconfigd is not running, cannot enable

# vxconfigd -k
VxVM vxconfigd ERROR V-5-1-8561 ddl_fill_dmp_info: Path 0x4080280 not found in devlist: Error 0
VxVM vxconfigd ERROR V-5-1-1589 enable failed: System error occurred in the client
    Error 0; aborting

Veritas also creates a core dump file, Here is the pstack output:

# pstack core
core 'core' of 64:    vxconfigd -x syslog -m boot
 fed656ba strcmp   (84215b0, bcddc5c) + 10a
 08120aaf dmp_getpathindex (8047b3c, 84215b0, 9fb5000) + 77
 08120492 dmp_countenclr_lun (8047b3c, 804729a, 9fb5000) + 72
 0811f6d3 dmp_dumpenclr (8047b3c, 9fc0408, 9fb5000) + fb
 0811f586 dmp_append (8047b3c, 9fc0408, 0, 2710, 9fb5000) + 12a
 0811f120 config_dmpdump (8883110, 9fc0408, 3e000000) + 1d8
 081259f0 config_all (8883110, 3e000000, 1) + 57c
 08125397 req_dg_getconfig_all (8883110, 82bec48) + 3f
 0812f55b request_loop (8047e68, 8047de8, feffb818, 1, feffb818, 3b80000) + 76b
 0810a2fc main     (5, 8047e2c, 8047e44) + b74
 080771d2 ???????? (5, 8047ed0, 8047eda, 8047edd, 8047ee4, 8047ee7)

Engine log:

2011/12/08 09:31:27 VCS ERROR V-16-10001-1004 CVMVxconfigd:???:monitor:Core file saved once already
2011/12/08 09:31:28 VCS INFO V-16-2-13001  Resource(cvm_vxconfigd): Output of the completed operation (monitor)
/core: ELF 32-bit LSB core file 80386 Version 1, from 'vxconfigd'

Any ideas ?

P.S. All logical paths are shown active by running "powermt display dev=all" towards the EMC array.

Thanks,

Florin.

[Marianne]

Which OS?

Which VxVM version (including patch level)?

[Florin_M_]

Its Solaris 10

SunOS 5.10 Generic_142901-07 i86pc i386 i86pc

% pkginfo -l VRTSvxvm
   PKGINST:  VRTSvxvm
      NAME:  Binaries for VERITAS Volume Manager by Symantec
  CATEGORY:  system
      ARCH:  i386
   VERSION:  5.0,REV=04.15.2007.12.15
   BASEDIR:  /
    VENDOR:  Symantec Corporation
      DESC:  Virtual Disk Subsystem
    PSTAMP:  VERITAS-5.0_x86_MP3_RP3.3:2009-11-12
  INSTDATE:  May 27 2010 07:25
   HOTLINE:  http://support.veritas.com/phonesup/phonesup_ddProduct_.htm
     EMAIL:  support@veritas.com
    STATUS:  completely installed
     FILES:      868 installed pathnames
                  32 shared pathnames
                  13 linked files
                 111 directories
                 413 executables
              287438 blocks used (approx)

[Marianne]

LOTS of vxconfigd fixes in SF 5.1:

http://sfdoccentral.symantec.com/sf/5.1/solaris/html/sfcfs_notes/ch01s07s02.htm

[Gaurav_S]

Hi,

just had a quick search on Symantec site, nothing found though into known panic strings...

Just looking at pstack output, it suggests that panic is happening during :dmp_getpathindex:  so the function suggests by name that DMP trying to get index of paths & dumping core ..

I would recommend to have following steps (just a try)

--Ensure the powerpath version you are running is compatible with vxvm version you are using.

-- Ensure all devices looks OK from powerpath.

Try out a device tree cleanup:

http://www.symantec.com/docs/TECH76780

OR

http://www.symantec.com/docs/TECH78376

also, have a pkg consistency check...

# pkgchk VRTSvxvm

Gaurav