07-21-2014 10:23 PM
07-21-2014 10:27 PM
07-21-2014 10:32 PM
Hi,
Could you provide more details .. what version of OS & SF ? what is the storage in use ?
Can you also paste output of fdisk & "vxprint -qthg <diskgroup>" just to verify the original size. the disks reporting size in xxz are storage disks only ?
G
07-21-2014 11:55 PM
Hi pgm10s,
Also, pls paste the output of "vxdisk -eo alldgs list". And have you check compatible list for the SF version you have ?
07-22-2014 07:11 AM
Thank you.
I will request the information from the customer.
07-22-2014 07:12 AM
Thank you.
I will request the information from the customer.
07-22-2014 07:14 AM
Hello
We use SFHA 6.1 HF3 on SUSE11SP3. The SAN storage is 3PAR.
All disks (LUN) are 3PAR disks.
07-22-2014 07:53 AM
hsea-apdr-s06:~ # vxdisk -eo alldgs list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
disk_0 auto:none - - online invalid sdb -
disk_1 auto:LVM - - LVM sdc -
p63000_0 auto:cdsdisk - (fendg) online sdaj -
p63000_1 auto:cdsdisk - (fendg) online sdba -
p63000_2 auto:cdsdisk - (fendg) online sdal -
p63000_3 auto:cdsdisk sapdata1 ORADG online sdbc -
p63000_4 auto:cdsdisk sapdata2 ORADG online sdbd -
p63000_5 auto:cdsdisk sapdata3 ORADG online sdbe -
p63000_6 auto:cdsdisk sapdata4 ORADG online sdap -
p63000_7 auto:cdsdisk sapdata5 ORADG online sdbg -
p63000_8 auto:cdsdisk sapdata6 ORADG online sdar -
p63000_9 auto:cdsdisk sapdata7 ORADG online sdbi -
p63000_10 auto:cdsdisk sapdata8 ORADG online sdbj -
p63000_11 auto:cdsdisk sapdata9 ORADG online sdbk -
p63000_12 auto:cdsdisk sapdata10 ORADG online sdbl -
p63000_13 auto:cdsdisk saparch ORADG online sdbm -
p63000_14 auto:cdsdisk oradg01 ORADG online sdax -
p63000_15 auto:cdsdisk sapdg01 SAPDG online sdbo -
sda auto:LVM - - online invalid sda -
hsea-apdr-s06:~ # vxprint -qthg ORADG
dg ORADG default default 25000 1403842370.36.hsea-apdr-s05
dm oradg01 p63000_14 auto 65536 209639920 -
dm saparch p63000_13 auto 65536 1048436800 -
dm sapdata1 p63000_3 auto 65536 2076022208 -
dm sapdata2 p63000_4 auto 65536 2076022208 -
dm sapdata3 p63000_5 auto 65536 2076022208 -
dm sapdata4 p63000_6 auto 65536 2076022208 -
dm sapdata5 p63000_7 auto 65536 5404118821994739088 -
dm sapdata6 p63000_8 auto 65536 5404118821994739088 -
dm sapdata7 p63000_9 auto 65536 1258178672 -
dm sapdata8 p63000_10 auto 65536 1258178672 -
dm sapdata9 p63000_11 auto 65536 1258178672 -
dm sapdata10 p63000_12 auto 65536 1258178672 -
v VmirrlogA - ENABLED ACTIVE 20971520 SELECT - fsgen
pl VmirrlogA-01 VmirrlogA ENABLED ACTIVE 20971520 CONCAT - RW
sd oradg01-01 VmirrlogA-01 oradg01 0 20971520 0 p63000_14 ENA
v VmirrlogB - ENABLED ACTIVE 20971520 SELECT - fsgen
pl VmirrlogB-01 VmirrlogB ENABLED ACTIVE 20971520 CONCAT - RW
sd oradg01-02 VmirrlogB-01 oradg01 20971520 20971520 0 p63000_14 ENA
v VoriglogA - ENABLED ACTIVE 20971520 SELECT - fsgen
pl VoriglogA-01 VoriglogA ENABLED ACTIVE 20971520 CONCAT - RW
sd oradg01-03 VoriglogA-01 oradg01 41943040 20971520 0 p63000_14 ENA
v VoriglogB - ENABLED ACTIVE 20971520 SELECT - fsgen
pl VoriglogB-01 VoriglogB ENABLED ACTIVE 20971520 CONCAT - RW
sd oradg01-04 VoriglogB-01 oradg01 62914560 20971520 0 p63000_14 ENA
v Vsaparch - ENABLED ACTIVE 1048227840 SELECT - fsgen
pl Vsaparch-01 Vsaparch ENABLED ACTIVE 1048227840 CONCAT - RW
sd saparch-01 Vsaparch-01 saparch 0 1048227840 0 p63000_13 ENA
v Vsapbackup - ENABLED ACTIVE 10485760 SELECT - fsgen
pl Vsapbackup-01 Vsapbackup ENABLED ACTIVE 10485760 CONCAT - RW
sd oradg01-05 Vsapbackup-01 oradg01 83886080 10485760 0 p63000_14 ENA
v Vsapcheck - ENABLED ACTIVE 10485760 SELECT - fsgen
pl Vsapcheck-01 Vsapcheck ENABLED ACTIVE 10485760 CONCAT - RW
sd oradg01-06 Vsapcheck-01 oradg01 94371840 10485760 0 p63000_14 ENA
v Vsapdata1 - ENABLED ACTIVE 2076020736 SELECT - fsgen
pl Vsapdata1-01 Vsapdata1 ENABLED ACTIVE 2076020736 CONCAT - RW
sd sapdata1-01 Vsapdata1-01 sapdata1 0 2076020736 0 p63000_3 ENA
v Vsapdata2 - ENABLED ACTIVE 2076020736 SELECT - fsgen
pl Vsapdata2-01 Vsapdata2 ENABLED ACTIVE 2076020736 CONCAT - RW
sd sapdata2-01 Vsapdata2-01 sapdata2 0 2076020736 0 p63000_4 ENA
v Vsapdata3 - ENABLED ACTIVE 2076020736 SELECT - fsgen
pl Vsapdata3-01 Vsapdata3 ENABLED ACTIVE 2076020736 CONCAT - RW
sd sapdata3-01 Vsapdata3-01 sapdata3 0 2076020736 0 p63000_5 ENA
v Vsapdata4 - ENABLED ACTIVE 2076020736 SELECT - fsgen
pl Vsapdata4-01 Vsapdata4 ENABLED ACTIVE 2076020736 CONCAT - RW
sd sapdata4-01 Vsapdata4-01 sapdata4 0 2076020736 0 p63000_6 ENA
v Vsapdata5 - ENABLED ACTIVE 2075813888 SELECT - fsgen
pl Vsapdata5-01 Vsapdata5 ENABLED ACTIVE 2075813888 CONCAT - RW
sd sapdata5-01 Vsapdata5-01 sapdata5 0 2075813888 0 p63000_7 ENA
v Vsapdata6 - ENABLED ACTIVE 2075813888 SELECT - fsgen
pl Vsapdata6-01 Vsapdata6 ENABLED ACTIVE 2075813888 CONCAT - RW
sd sapdata6-01 Vsapdata6-01 sapdata6 0 2075813888 0 p63000_8 ENA
v Vsapdata7 - ENABLED ACTIVE 1258178560 SELECT - fsgen
pl Vsapdata7-01 Vsapdata7 ENABLED ACTIVE 1258178560 CONCAT - RW
sd sapdata7-01 Vsapdata7-01 sapdata7 0 1258178560 0 p63000_9 ENA
v Vsapdata8 - ENABLED ACTIVE 1258178560 SELECT - fsgen
pl Vsapdata8-01 Vsapdata8 ENABLED ACTIVE 1258178560 CONCAT - RW
sd sapdata8-01 Vsapdata8-01 sapdata8 0 1258178560 0 p63000_10 ENA
v Vsapdata9 - ENABLED ACTIVE 1258178560 SELECT - fsgen
pl Vsapdata9-01 Vsapdata9 ENABLED ACTIVE 1258178560 CONCAT - RW
sd sapdata9-01 Vsapdata9-01 sapdata9 0 1258178560 0 p63000_11 ENA
v Vsapdata10 - ENABLED ACTIVE 1258178560 SELECT - fsgen
pl Vsapdata10-01 Vsapdata10 ENABLED ACTIVE 1258178560 CONCAT - RW
sd sapdata10-01 Vsapdata10-01 sapdata10 0 1258178560 0 p63000_12 ENA
v Vsapreorg - ENABLED ACTIVE 20971520 SELECT - fsgen
pl Vsapreorg-01 Vsapreorg ENABLED ACTIVE 20971520 CONCAT - RW
sd oradg01-07 Vsapreorg-01 oradg01 104857600 20971520 0 p63000_14 ENA
v Vsaptrace - ENABLED ACTIVE 10485760 SELECT - fsgen
pl Vsaptrace-01 Vsaptrace ENABLED ACTIVE 10485760 CONCAT - RW
sd oradg01-08 Vsaptrace-01 oradg01 125829120 10485760 0 p63000_14 ENA
07-22-2014 08:05 AM
please review the text file.
07-22-2014 07:20 PM
Hi pgm10s
From the output of vxdisk and vxprint, the status of disk p63000_7 and p63000_8 are good and normal. And doing a research on the size expression I found following info from http://en.wikipedia.org/wiki/File_size
Zettabyte | ZB | 270 | 1021 | 1,180,591,620,717,411,303,424 | (1,024) EB |
So technically it's just a huge size of a LUN. And from the output of vxprint we could see that the block number of the 2 disks is 5404118821994739088. Here one block is 512 bytes, so you see here it just fits into 2.34 ZB which is exactly match the output of vxlist.
But I don't know how you get such huge LUNs from disk array even from 3par. Is that a thin provisioning LUN with unlimited size? That's the only possibility I could think of.
07-23-2014 02:58 AM
I Agree, these are normal luns however of very huge size in Zetabytes & hence appearing as xxz
Be careful in using this size of volume as you can't have that big filesystem (vxfs), max size in 6.1 for FS is 256TB with 8k block size
G
07-24-2014 12:43 AM
hsea-apdr-s06:/home/software/data_collector/vrtsexplorer_reports # fdisk -l | grep sdar
WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.
Disk /dev/sdc doesn't contain a valid partition table
WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.
Disk /dev/sdar (Sun disk label): 65479 heads, 19216 sectors, 65535 cylinders
/dev/sdar3 u 0 3036722832 1518361416 5 Whole disk
/dev/sdar8 u 0 3036722832 1518361416 f Unknown
Disk /dev/sdar3 (Sun disk label): 65479 heads, 19216 sectors, 65535 cylinders
/dev/sdar3p3 u 0 3036722832 1518361416 5 Whole disk
/dev/sdar3p8 u 0 3036722832 1518361416 f Unknown
Disk /dev/sdar8 (Sun disk label): 65479 heads, 19216 sectors, 65535 cylinders
/dev/sdar8p3 u 0 3036722832 1518361416 5 Whole disk
/dev/sdar8p8 u 0 3036722832 1518361416 f Unknown
Disk /dev/mapper/system-11264 doesn't contain a valid partition table
Disk /dev/mapper/system-dump doesn't contain a valid partition table
Disk /dev/mapper/system-home doesn't contain a valid partition table
Disk /dev/mapper/system-opt doesn't contain a valid partition table
Disk /dev/mapper/system-oracle doesn't contain a valid partition table
Disk /dev/mapper/system-sapmntPAS doesn't contain a valid partition table
Disk /dev/mapper/system-tmp doesn't contain a valid partition table
Disk /dev/mapper/system-usr doesn't contain a valid partition table
Disk /dev/mapper/system-var doesn't contain a valid partition table
hsea-apdr-s06:/home/software/data_collector/vrtsexplorer_reports # vxdisk list sdar
Device: p63000_8
devicetag: p63000_8
type: auto
hostid: hsea-apdr-s06
disk: name=sapdata6 id=1403841988.28.hsea-apdr-s05
group: name=ORADG id=1403842370.36.hsea-apdr-s05
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3
flags: online ready private autoconfig noautoimport imported
pubpaths: block=/dev/vx/dmp/p63000_8s3 char=/dev/vx/rdmp/p63000_8s3
guid: {8a945000-1086-11e4-8925-79ab3a332144}
udid: HP%5FHSV340%5F500143801138D870%5F600143801259CD390000500001AA0000
site: -
version: 3.1
iosize: min=512 (bytes) max=2048 (blocks)
public: slice=3 offset=65792 len=5404118821994739088 disk_offset=0
private: slice=3 offset=256 len=65536 disk_offset=0
update: time=1406039982 seqno=0.40
ssb: actual_seqno=0.0
headers: 0 240
configs: count=1 len=51360
logs: count=1 len=4096
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 disabled
config priv 000256-051423[051168]: copy=01 offset=000192 disabled
log priv 051424-055519[004096]: copy=01 offset=000000 disabled
lockrgn priv 055520-055663[000144]: part=00 offset=000000
Multipathing information:
numpaths: 4
sdar state=enabled type=primary
sdbh state=enabled type=secondary
sdab state=enabled type=secondary
sdl state=enabled type=primary
hsea-apdr-s06:/home/software/data_collector/vrtsexplorer_reports #
07-25-2014 08:41 AM
FYI - SF 6.1 has some enhanced vxdisk options for help with sizes i.e "vxdisk -o size list"
Also see this post
https://www-secure.symantec.com/connect/blogs/displaying-volume-and-disk-sizes-human-friendly-format-mb-gb-tb-volume-manager-vxvm
cheers
tony
11-18-2014 04:55 AM
Solution: If data loss is not a constraint
Remove the disk from DG
Uninitialize the disk
#vxdctl enable
#vxdisk scandisks
Initialize the disk.
Add the disk in DG
format the disk and mount it.
11-18-2014 04:57 AM
Solution: Without loosing the data.
ISSUE: vxlist report wrong disk size
ERROR CODE/MESSAGE:
disk p63000_7 sapdata5 ORADG 2.34z 2.34z imported <<<<<<<<<<<<
disk p63000_8 sapdata6 ORADG 2.34z 2.34z imported <<<<<<<<<<<<
dm sapdata5 p63000_7 auto 0.03g 2576884661671.99g - <<<<<<<<<<<<<<<<
dm sapdata6 p63000_8 auto 0.03g 2576884661671.99g - <<<<<<<<<<<<<<<<
ENVIRONMENT/CONDITIONS:
OS : SLES 11
SF : 6.1.0.100
SOLUTION/WORKAROUND:
Provided below details -
-- There is a issue with the disk label because of which the disk size is showing in zeta bytes.
-- To resolve this issue we need to remove the disk from the disk group, relabel the disk then add it in the disk group.
-- If you are not going to resize any of the volumes, in that case there will not be any impact on the disk. These two disks should not be used for any resize operation. But still I would recommend you to follow the steps and correct the disk.
-- As per discussion with the Backline, they have mentioned that if you have resize the lun size from 660GB to 99GB then by any chance have you collected the debug logs of that particular time? Those debug logs will help us to know what exactly was happened with these disks.
Need to follow below steps to correct the disk label which will resolve the incorrect disk size issue.
=> Vsapdata5 and Vsapdata6 volumes are affected because of this issue.
dm sapdata5 p63000_7 auto 0.03g 2576884661671.99g -
dm sapdata6 p63000_8 auto 0.03g 2576884661671.99g -
disk p63000_7 sapdata5 ORADG 2.34z 2.34z imported
disk p63000_8 sapdata6 ORADG 2.34z 2.34z imported
v Vsapdata5 - ENABLED ACTIVE 989.82g SELECT - fsgen
pl Vsapdata5-01 Vsapdata5 ENABLED ACTIVE 989.82g CONCAT - RW
sd sapdata5-01 Vsapdata5-01 sapdata5 0.00g 989.82g 0.00g p63000_7 ENA
v Vsapdata6 - ENABLED ACTIVE 989.82g SELECT - fsgen
pl Vsapdata6-01 Vsapdata6 ENABLED ACTIVE 989.82g CONCAT - RW
sd sapdata6-01 Vsapdata6-01 sapdata6 0.00g 989.82g 0.00g p63000_8 ENA
=> Mirror the volume, if there is a free space then use it else need to get the new luns for mirroring the volume.
------------------------------------------------------------------------------------------
If you are going to add new disk, follow the below steps -
-- Assign the new luns from the SAN and make sure they are visible properly in the OS.
-- Initialize the new disk
#vxdisksetup -i DISKname
Example - vxdisksetup -i DISKname
-- Add the disk in the disk group
#vxdg -g DGname adddisk <DM name>=<DA Name>
Example : vxdg -g ORADG adddisk sparedisk=<New Disk Name>
------------------------------------------------------------------------------------------
#vxassist -b -g <disk group> mirror <volume>
Example - vxassist -b -g ORADG mirror Vsapdata5
v Vsapdata5 - ENABLED ACTIVE 989.82g SELECT - fsgen
pl Vsapdata5-01 Vsapdata5 ENABLED ACTIVE 989.82g CONCAT - RW
sd sapdata5-01 Vsapdata5-01 sapdata5 0.00g 989.82g 0.00g p63000_7 ENA
v Vsapdata6 - ENABLED ACTIVE 989.82g SELECT - fsgen
pl Vsapdata6-01 Vsapdata6 ENABLED ACTIVE 989.82g CONCAT - RW
sd sapdata6-01 Vsapdata6-01 sapdata6 0.00g 989.82g 0.00g p63000_8 ENA
=> Monitor the sync with vxtask list command. Vxtask can be used to check the status of a mirror resynchronization. If needed, it can also be used to pause, resume or abort the operation.
# vxtask list
# vxtask -l list
=> Once the mirroring is done, please dis associate the old plex.
#vxplex -g <Disk Group> -o rm dis <old plex>
=> Remove both the disk from the disk group
#vxdg -g <Disk Group> rmdisk <DM Name>
Example - vxdg -g ORADG rmdisk sapdata5
p63000_7 auto:cdsdisk sapdata5 ORADG online sdbg -
p63000_8 auto:cdsdisk sapdata6 ORADG online sdar -
=> Uninitialize both the disk
#vxdiskunsetup -C <Disk Name>
Example - #vxdiskunsetup -C p63000_7
p63000_7 auto:cdsdisk sapdata5 ORADG online sdbg -
p63000_8 auto:cdsdisk sapdata6 ORADG online sdar -
=> Write the partition table on the disk
#fdisk /dev/DISKname #####Example - fdisk /dev/sdbg
type "w"
quite
#partprobe >>>> it will update the partition table.
Below are the disk paths
sdbg <<<<< Use this path p63000_7 sapdata5 ORADG ENABLED
sdaq p63000_7 sapdata5 ORADG ENABLED
sdk p63000_7 sapdata5 ORADG ENABLED
sdaa p63000_7 sapdata5 ORADG ENABLED
sdar p63000_8 sapdata6 ORADG ENABLED
sdbh p63000_8 sapdata6 ORADG ENABLED
sdab p63000_8 sapdata6 ORADG ENABLED
sdl p63000_8 sapdata6 ORADG ENABLED
=> Initialize both the disk and add it back in the disk group
#vxdisksetup -i DISKname
Example - vxdisksetup -i p63000_7
p63000_7 auto:cdsdisk sapdata5 ORADG online sdbg -
p63000_8 auto:cdsdisk sapdata6 ORADG online sdar -
=> Add both the disk in the disk group
#vxdg -g DGname adddisk <DM name>=<DA Name>
Example : vxdg -g ORADG adddisk sapdata5=p63000_7
=> Mirror the volume with the same disk, perform this operation on both the volumes.
#vxassist -b -g <disk group> mirror <volume> alloc=DISKname
Example - vxassist -b -g ORADG mirror Vsapdata5 alloc=sapdata5
v Vsapdata5 - ENABLED ACTIVE 989.82g SELECT - fsgen
pl Vsapdata5-01 Vsapdata5 ENABLED ACTIVE 989.82g CONCAT - RW
sd sapdata5-01 Vsapdata5-01 sapdata5 0.00g 989.82g 0.00g p63000_7 ENA
v Vsapdata6 - ENABLED ACTIVE 989.82g SELECT - fsgen
pl Vsapdata6-01 Vsapdata6 ENABLED ACTIVE 989.82g CONCAT - RW
sd sapdata6-01 Vsapdata6-01 sapdata6 0.00g 989.82g 0.00g p63000_8 ENA
=> Monitor the sync with vxtask list command. Vxtask can be used to check the status of a mirror resynchronization. If needed, it can also be used to pause, resume or abort the operation.
# vxtask list
# vxtask -l list
=> Once the mirroring is done, please dis associate the old plex.
#vxplex -g <Disk Group> -o rm dis <old plex>
=> If you have used the spare disk, Remove it from the disk group
#vxdg -g <Disk Group> rmdisk <DM Name>
Example - vxdg -g ORADG rmdisk <spare disk>
=> Uninitialize both the spare disk and remove it from veritas control
#vxdiskunsetup -C <Disk Name>
11-19-2014 05:02 AM
Thanks for detailed feedback.
Was solution provided by Symantec Support?
If so, was a TN issued?