02-13-2013 06:18 AM
Hi guys,
I am a little curious about defragmentation. Probably someone can explain defragmentation.
First let me start what I have done so far:
1. Created a DG, Volume (1GB) etc (testing system only)
2. Created about 1000 1MB files
3. Deleted every 2nd file
And now I am curious. When I ran fsadm -E /local (/local is the mounted test volume of course) I get the following output:
...
Free Extents By Size
1: 2 2: 2 4: 3
8: 7 16: 6 32: 6
64: 5 128: 5 256: 6
512: 3 1024: 326 2048: 2
...
It shows clearly 326 free 1024K extents. From my understanding the file system is fragmentated now? Because when I create a large file now it would use all the 1024K extents and not one 128M extent, two 64M extent etc? Am I wrong/right?
And how can I rearrange the extents (if it is possible). I have tried various fsadm options like -e, -d and -C but without success.
BTW:
pkginfo -l VRTSvxfs
...
PSTAMP: 5.1.103.000-5.1SP1RP3_x86-2012-09-16-VERITAS-FS-142635-12
...
uname -a
SunOS sol10 5.10 Generic_142910-17 i86pc i386 i86pc
Any suggestions?
Solved! Go to Solution.
02-14-2013 03:16 AM
Yes, this is what I said in my last post - here were some of my tests (on RHEL 5.3, SF 5.1 - no SP):
With:
Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3952 25 1 23748blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 24.81% Free blocks in extents smaller than 8 blks: 0.93% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 548: 1 16: 0 32: 17764: 87 128: 2 256: 1Pass 1 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3962 0 0 0 0 0Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3952 25 1 23748blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 24.81% Free blocks in extents smaller than 8 blks: 0.93% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 548: 1 16: 0 32: 17764: 87 128: 2 256: 1
Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3917 25 1 23788blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 24.94% Free blocks in extents smaller than 8 blks: 1.09% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 648: 1 16: 0 32: 17764: 87 128: 2 256: 1Pass 1 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3927 194 194 0 0 0Pass 2 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3927 67 67 0 0 0Pass 3 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3927 0 0 0 0 0Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks4141 24 1 23784blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 14.16% Free blocks in extents smaller than 8 blks: 0.30% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 178: 2 16: 1 32: 10264: 75 128: 4 256: 3
Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3587 32 1 27552blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 35.42% Free blocks in extents smaller than 8 blks: 0.06% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 38: 0 16: 1 32: 30464: 86 128: 2 256: 1Pass 1 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 2 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 3 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 4 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 5 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3587 32 1 27537blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 35.16% Free blocks in extents smaller than 8 blks: 0.06% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 3 2: 1 4: 38: 0 16: 0 32: 30264: 87 128: 2 256: 1
02-13-2013 06:39 AM
See the vxfs admin guide for vxfs desribes as fragmented:
To determine the degree of fragmentation, use the following factors:■ Percentage of free space in extents of less than 8 blocks in length■ Percentage of free space in extents of less than 64 blocks in length■ Percentage of free space in extents of length 64 blocks or greaterAn unfragmented file system has the following characteristics:■ Less than 1 percent of free space in extents of less than 8 blocks in length■ Less than 5 percent of free space in extents of less than 64 blocks in length■ More than 5 percent of the total file system size available as free extents inlengths of 64 or more blocksA badly fragmented file system has one or more of the following characteristics:■ Greater than 5 percent of free space in extents of less than 8 blocks in length■ More than 50 percent of free space in extents of less than 64 blocks in length■ Less than 5 percent of the total file system size available as free extents inlengths of 64 or more blocks
02-13-2013 07:04 AM
OK, did it (after formatting the volume):
c=0; while [[ $c -lt 29116 ]]; do let c=c+1; dd if=/dev/zero of=$c bs=32768 count=1 > /dev/null 2>&1; done
fsadm -E /local
Extent Fragmentation Report
Total Average Average Total
Files File Blks # Extents Free Blks
28637 32 1 54311
blocks used for indirects: 8
% Free blocks in extents smaller than 64 blks: 0.07
% Free blocks in extents smaller than 8 blks: 0.01
% blks allocated to extents 64 blks or larger: 0.00
Free Extents By Size
1: 1 2: 1 4: 1
8: 0 16: 0 32: 1
64: 0 128: 2 256: 1
512: 1 1024: 2 2048: 1
4096: 0 8192: 2 16384: 2
c=0; while [[ $c -lt 29116 ]]; do let c=c+2; rm $c; done
fsadm -E /local
Extent Fragmentation Report
Total Average Average Total
Files File Blks # Extents Free Blks
29104 32 1 520167
blocks used for indirects: 8
% Free blocks in extents smaller than 64 blks: 89.54
% Free blocks in extents smaller than 8 blks: 0.00
% blks allocated to extents 64 blks or larger: 0.00
Free Extents By Size
1: 1 2: 1 4: 1
8: 0 16: 0 32: 14555
64: 2 128: 2 256: 1
512: 1 1024: 2 2048: 1
4096: 0 8192: 2 16384: 2
I have a lot of free 32K extents now. OK, the % of extents smaller than 64K has increased - pretty obvious since 32K is smaller than 64K. But what is the difference of having a lot of 1MB exntents free and a lot more 32KB extents free? And again, when can I see that my volume is fragmentated?
Thanks so far,
Marek
02-13-2013 10:20 AM
If you are trying to create a 5MB file then if you have 1MB sized free extents then you can write to 5 extents, but if you only have 32K extents free, then you will need to write to 160 extents - this will involve a lot more work and in particular the file will have to use indirect extents (see https://www-secure.symantec.com/connect/forums/output-fsadm). But on saying this, if you wrote a GB file, then if you only had 1MB sized free extents, the file would need to write to over 1000 extents, so perhaps fragmentation should be a function of the size of the filesystem. Also it would be useful if the fsadm defrag report said something like "your filesystem does/does not need defragmenting" with some easy indication of how badly it is fragmented, but it just seems to report on % of free block in extents, smaller then 8k, smaller than 64k, size of 64k and larger.
In theory if the filesystem is fragmented by the definition in the guide, then I think it should be defraged, but I did some tests and filesystems were not defraged when more than 5 percent of free space was in extents of less than 64 blocks and it didn't even always defrag when filesystem had more than 50 percent of free space in extents of less than 64 blocks (badly fragmented). But the filesytem was defraged for filesystems where more than 1 percent of free space was in extents of less than 8 blocks (using fsadm -E -e -s /local)
Mike
02-14-2013 02:45 AM
It seems that fsadm does not really recognize a lot of free 32 blks extents as a problem. It must be smaller. So I formatted my volume again and created a lot of 1K files like my example before with the 32K files and deleted every second file. Then I ran fsadm -E /local:
fsadm -E /local
Extent Fragmentation Report
Total Average Average Total
Files File Blks # Extents Free Blks
46799 1 1 921961
blocks used for indirects: 8
% Free blocks in extents smaller than 64 blks: 5.08
% Free blocks in extents smaller than 8 blks: 5.08
% blks allocated to extents 64 blks or larger: 0.00
Free Extents By Size
1: 46307 2: 245 4: 1
It shows clearly a lot of free 1 blks extents. Than I ran fsadm -e -d -s /local and fsadm -E /local to verify:
fsadm -E /local
Extent Fragmentation Report
Total Average Average Total
Files File Blks # Extents Free Blks
46799 1 1 922768
blocks used for indirects: 0
% Free blocks in extents smaller than 64 blks: 0.14
% Free blocks in extents smaller than 8 blks: 0.01
% blks allocated to extents 64 blks or larger: 1.67
Free Extents By Size
1: 2 2: 1 4: 13
8: 23 16: 24 32: 21
64: 22 128: 178 256: 177
Finally the extents were reorganized. So it really seems that fsadm can/will (who knows) only reorganize extents of a very small block size. Where 1 blks extent seems to be small enough to be reorginzed but not 32 blks extents.
Marek
02-14-2013 03:16 AM
Yes, this is what I said in my last post - here were some of my tests (on RHEL 5.3, SF 5.1 - no SP):
With:
Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3952 25 1 23748blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 24.81% Free blocks in extents smaller than 8 blks: 0.93% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 548: 1 16: 0 32: 17764: 87 128: 2 256: 1Pass 1 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3962 0 0 0 0 0Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3952 25 1 23748blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 24.81% Free blocks in extents smaller than 8 blks: 0.93% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 548: 1 16: 0 32: 17764: 87 128: 2 256: 1
Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3917 25 1 23788blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 24.94% Free blocks in extents smaller than 8 blks: 1.09% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 648: 1 16: 0 32: 17764: 87 128: 2 256: 1Pass 1 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3927 194 194 0 0 0Pass 2 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3927 67 67 0 0 0Pass 3 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3927 0 0 0 0 0Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks4141 24 1 23784blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 14.16% Free blocks in extents smaller than 8 blks: 0.30% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 178: 2 16: 1 32: 10264: 75 128: 4 256: 3
Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3587 32 1 27552blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 35.42% Free blocks in extents smaller than 8 blks: 0.06% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 2 2: 1 4: 38: 0 16: 1 32: 30464: 86 128: 2 256: 1Pass 1 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 2 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 3 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 4 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Pass 5 StatisticsExtents Reallocations Ioctls ErrorsSearched Attempted Issued FileBusy NoSpace Totaltotal 3600 2 1 0 0 0Extent Fragmentation ReportTotal Average Average TotalFiles File Blks # Extents Free Blks3587 32 1 27537blocks used for indirects: 0% Free blocks in extents smaller than 64 blks: 35.16% Free blocks in extents smaller than 8 blks: 0.06% blks allocated to extents 64 blks or larger: 0.00Free Extents By Size1: 3 2: 1 4: 38: 0 16: 0 32: 30264: 87 128: 2 256: 1