03-03-2016 09:12 AM
Hello everyone,
Please I need help regarding a volume I cannot mount. It is 2 Symmetrix EMC SAN meta (emcpower), forming one dg (supposedly clonedg), divided into vol01 and vol02. Each day, the metas are cloned from the principal server and mounted, but yesterday we had this issue :
/dev/vx/dsk/clonedg/vol01 460G 379G 77G 84% /clone/hydra/oracle df: `/clone/hydra/ora11g': Input/output error (for /dev/vx/dsk/clonedg/vol02)
Note that vol01 works perfectly, but not 02.
Last time it happened, it did a umount -l and waited for the next day for the clone and it worked.
So I umounted it again and then tried to clone and remount, but this time, it does not work.
I then tried to deport the dg, in the hope to import it, but no luck :
# vxdg -C import clonedg VxVM vxdg ERROR V-5-1-10978 Disk group clonedg: import failed: No valid disk found containing disk group # vxdg -f import clonedg VxVM vxdg ERROR V-5-1-10978 Disk group clonedg: import failed: No valid disk found containing disk group
I also did :
vxdctl initdmp vxdisk scandisks
and now I have this :
# vxdisk list DEVICE TYPE DISK GROUP STATUS cciss/c0d0 auto:none - - online invalid emcpowera auto:cdsdisk - - error emcpowera auto - - error emcpowera auto - - error emcpowera auto - - error emcpowera auto - - error emcpowera auto - - error emcpowerb auto:cdsdisk - - error emcpowerb auto - - error emcpowerb auto - - error emcpowerb auto - - error emcpowerb auto - - error emcpowerb auto - - error emcpowerb auto - - error emcpowerc auto - - error emcpowerc auto - - error emcpowerc auto:cdsdisk - - error emcpowerc auto - - error emcpowerc auto - - error emcpowerc auto - - error emcpowerd auto:cdsdisk - - online udid_mismatch emcpowerd auto - - error emcpowere auto:cdsdisk - - online udid_mismatch emcpowere auto - - error emcpowerf auto - - error emcpowerf auto:cdsdisk - - online emcpowerf auto - - error emcpowerf auto - - error emcpowerf auto - - error emcpowerf auto - - error emcpowerg auto:LVM - - LVM
Note that many volume are in error, and they are duplicated.
From here, I don't know what to do, since I'm not expert in vxfs.
so any help would be appreciated!
Thank you.
03-03-2016 09:15 AM
I forgot to mention that since the dg is not recognized and cannot be imported, now I cannot even mount vol01.
03-03-2016 09:49 AM
What are the outputs of
# vxdmpadm listctlr all
# vxdisk list emcpowera
# vxdmpadm listenclosure all
# modinfo |grep -i vx
G
03-04-2016 01:23 AM
Hello G. Thank you for your answer.
I am writing you from a phone connected to my workstation via TViewer...
Here is the result of the commands :
vxdmpadm listctlr all CTLR-NAME ENCLR-TYPE STATE ENCLR-NAME ===================================================== c360 OTHER_DISKS ENABLED other_disks emcp PP_EMC ENABLED pp_emc0 [root@miranda log]# vxdisk list emcpowera Device: emcpowera devicetag: emcpowera type: auto flags: error private autoconfig pubpaths: block=/dev/vx/dmp/emcpowera char=/dev/vx/rdmp/emcpowera guid: {37e4d0f2-fff2-11e0-9100-459b072bef83} udid: EMC%5FSYMMETRIX%5F000290103833%5F334D5000 site: - errno: Duplicate device Multipathing information: numpaths: 1 emcpowera state=enabled [root@miranda log]# modinfo vxfs filename: /lib/modules/2.6.18-274.el5/veritas/vxfs/vxfs.ko version: 5.1.00.00 description: Veritas File System Send bug reports to support@veritas.com author: Symantec Corporation license: Proprietary srcversion: 5A5F73DB92E11B69B7574C9 depends: vermagic: 2.6.18-8.el5 SMP mod_unload gcc-4.1 parm: vxfs_ninode:Maximum number of vxfs inodes (int) parm: vx_ifree_timelag:Seconds before scanning inode free list (long) parm: vxfs_mbuf:Maximum memory used for vxfs buffer cache (charp) parm: vx_dexh_sz:Large directory hashing threshold (uint) parm: vxfs_maxdiospace:Maximum memory wired down for direct IO (charp) parm: vxfs_mindiospace:Minimum memory wired down for direct IO (charp) parm: vxfs_pmem:Physical memory to use as input for tuning curves (charp) parm: vxfs_icachechunk:Maximum number of pages for each memory chunk used for VxFS inodes (int) parm: vx_fiostats_tunable:Memory used for file I/O stats (int) parm: vx_fiostats_winterval:FCL winterval for file I/O stats (int) parm: vxfs_start_node_id:Highest node ID - override for low memory scanning (int) parm: vxfs_naio_threads_max:Maximum number of threads for native AIO (uint) parm: vxfs_naio_threads_min:Minimum number of threads for native AIO (uint) parm: vxfs_naio_plug_enable:Throttle native AIO on memory usage (uint) parm: vx_maxlink:Maximum number of hardlinks (uint) parm: vxfs_sendfile_chunk:Number of pages to fetch for large sendfile transfers (uint) parm: vxfs_sendfile_limit:Maximum number of processes in a sendfile main loop (uint) parm: vxfs_sendfile_nfs:Setting to non-zero changes VxFS behaviour with sendfile from NFS (uint) parm: vxfs_io_proxy_vxvm:Include VxVM devices in IO hand-off decisions (int) parm: vxfs_io_proxy_level:Hand-off IO to worker threads for `raw` block devices when less than this number of bytes free on kernel stack (int) parm: vxfs_io_proxy_pools:Number of IO pools for hand-off. (int) parm: vxfs_io_proxy_time:If an IO has been waiting on an IO queue for this number of clock ticks, then create a new IO hand-off thread (int) parm: vxfs_io_proxy_max:Maximum number of threads per IO handoff queue (int) # vxdmpadm listenclosure all vxdmpadm: Cannot allocate memory
The last command actually hanged for quite a bit.
03-04-2016 06:46 AM
Hello G,
I finally solved the problem :
1) rebooted the server
2)
vxdctl initdmp
vxdisk scandisks
3)
vxconfigrestore -p
vxconfigrestore -c
4) mount the FS
The config looks clean, hope it'll stay that way :)