Forum Discussion

jahanbin's avatar
jahanbin
Level 3
13 years ago

SEV 8 Collection and empty folders

We have collections enabled on our journaling stores.  the information is collected and stored in cab files correctly but the original folders structure stay intact.  These folders are empty and don’t seem to be used.  We use netbackup for backup.  Netbackup spends hours just crawling these empty folders.  Is there an easy way of getting rid of these folders?  Can I just delete them?

  • These links may provide some more information: 

     

    http://98.129.119.162/connect/fr/forums/collections-folders-remain

    Question: "EV2007 Should collection folders remain after their contents are passed onto their final storage location?  I just noticed on one of our servers there are 21,782 empty folders in the collection area, using up about 127MB of disk space for allocation tables."

    Answer:(Maynard K) "Yes, this structure will remain long after savesets are committed to your Centera.  Can they be removed?  I have heard two tales in regards to this.  I have encountered an issue in which I had to try and add in folder's that were missing because my StorageFileWatch process was choking a structure that wasn't complete."

    http://98.129.119.162/connect/de/forums/ev9-large-number-empty-directories-disk

    Question: "

    I'm running EV9.0 on Windows 2003 r2 SP2 and have a very nice, agressive collection schedule configured that works very well. Only thing - it leaves behind all the directories from which the archived files are collected from, so even though one of my partitions has been collected from about 2 million files down to about 80 thousand files, I still have about 2 million directories on disk, most of which are now empty.

    I suspect the sheer numbers of empty directories are contributing to backup performance degradation and very slow report generation of disk usage information (eg: diruse.exe, treesize, etc). There is also 600GB of a 2TB LUN that I also can't account for when comparing the disk properties free space to the folder/ file size properties from any of the disk usage reports. (The disk properties shows 600GB more in use than any disk usage report total).

    What would be the impact of manually deleting these empty directories? For example, when an item is retrieved from a CAB file, does EV expect the original directory structure to still be in place, causing the process to fail if I have removed those directories? Or will it store the archdvs file along side the CAB file, rendering the orignal directory structure irrelevant, or possibly will is recreate the original directory structure as required?"

     

    Answer: (JesusWept2) "

    Enterprise Vault should recreate the directory no problem.

    So what i would do if i were you, find an item through search that has been collected and then open the item through the browser, then locate the ArchDVS file on disk, then rename or delete the folder its located in, then try and open the item again.

    You should see it open and you should see that EV has recreated the directory and reextracted the item to an ArchDVS file."

     

    OK So i just tested it and it *does* recreate the folders
    Heres the steps i followed

    Step 1: 
    Go to http://yourEVServer/EnterpriseVault/Search.asp?advanced

    Step 2:
    Uncheck "Search Attachments",
    change Sort Results to "Archived Date, Oldest First",
    change Details to "Full"

    Step 3:
    Press the Search button

    Step 4:
    Make a note of the "SeqNum" on a particular result (i.e. SeqNum: 1)

    Step 5:
    Run the following query to get its CAB file location

    SELECT A.ArchiveName, S.IndexSeqNo, S.IdChecksumHigh, S.IdChecksumLow, S.IdDateTime, S.ArchivedDate, S.IdTransaction, (PE.PartitionRootPath + '\' + C.RelativeFileName)
    FROM EnterpriseVaultDirectory.dbo.Archive A,
         EnterpriseVaultDirectory.dbo.Root R,
         EnterpriseVaultDirectory.dbo.PartitionEntry PE,
         EnterpriseVaultDirectory.dbo.VaultStoreEntry VSE,
         yourVaultStore.dbo.Saveset S,
         yourVaultStore.dbo.Collection C,
         yourVaultStore.dbo.ArchivePoint AP
    WHERE S.CollectionIdentity = C.CollectionIdentity
      AND S.ArchivePointIdentity = AP.ArchivePointIdentity
      AND AP.ArchivePointId = R.VaultEntryId
      AND R.RootIdentity = A.RootIdentity
      AND A.VaultStoreEntryId = VSE.VaultStoreEntryId
      AND VSE.VaultStoreEntryId = PE.VaultStoreEntryId
      AND PE.IdPartition = C.IdPartition
      AND S.IndexSeqNo = 1
      AND A.ArchiveName LIKE 'Your Name%'

     

    Step 6:
    From the results i can see that the CAB file this item is located in is in:
    E:\Vault Stores\myVault Ptn1\2010\01-20\B\Collection12345.CAB

    My Transaction ID is : B03A658C-C9B1-43E8-E748-F87CCD744251
    My Archived Date is : 2010-01-20 20:54:00.000

    EV will extract the item from the CAB to \yyyy\mm-dd\[first TID char]\[next three TID chars]

    So in this case it would extract to the following location
    E:\VaultStores\myVault Ptn1\2010\01-20\B\03A\B03A658CC9B143E8E748F87CCD744251.ARCHDVS

    Step 7:
    Browse to that location and ensure the directory is empty

    Step 8:
    Go to the Browser with your search open and then open up that item that you found in Search.asp

    Step 9:
    Go back to the location and ensure a new ARCHDVS file now exists

    Step 10:
    Delete or rename the three letter TID directory (i.e rename \03A\ or delete it)

    Step 11:
    Go back to the Browser search and open the item again

    Step 12:
    Ensure that the directory and the file have been recreated

4 Replies

  • we just covered this topic in another thread. please have a look here:

    https://www-secure.symantec.com/connect/forums/ev9-large-number-empty-directories-disk

  • Andrew,

    Thanks for the link.  Unfortunately that doesn't fully my concerns.  I responded to that thread so hopefully someone has the answear.


    Jahanbin

  • thanks for posting in that thread. there's some very valuable information from JesusWept2 that should be right up your alley

  • These links may provide some more information: 

     

    http://98.129.119.162/connect/fr/forums/collections-folders-remain

    Question: "EV2007 Should collection folders remain after their contents are passed onto their final storage location?  I just noticed on one of our servers there are 21,782 empty folders in the collection area, using up about 127MB of disk space for allocation tables."

    Answer:(Maynard K) "Yes, this structure will remain long after savesets are committed to your Centera.  Can they be removed?  I have heard two tales in regards to this.  I have encountered an issue in which I had to try and add in folder's that were missing because my StorageFileWatch process was choking a structure that wasn't complete."

    http://98.129.119.162/connect/de/forums/ev9-large-number-empty-directories-disk

    Question: "

    I'm running EV9.0 on Windows 2003 r2 SP2 and have a very nice, agressive collection schedule configured that works very well. Only thing - it leaves behind all the directories from which the archived files are collected from, so even though one of my partitions has been collected from about 2 million files down to about 80 thousand files, I still have about 2 million directories on disk, most of which are now empty.

    I suspect the sheer numbers of empty directories are contributing to backup performance degradation and very slow report generation of disk usage information (eg: diruse.exe, treesize, etc). There is also 600GB of a 2TB LUN that I also can't account for when comparing the disk properties free space to the folder/ file size properties from any of the disk usage reports. (The disk properties shows 600GB more in use than any disk usage report total).

    What would be the impact of manually deleting these empty directories? For example, when an item is retrieved from a CAB file, does EV expect the original directory structure to still be in place, causing the process to fail if I have removed those directories? Or will it store the archdvs file along side the CAB file, rendering the orignal directory structure irrelevant, or possibly will is recreate the original directory structure as required?"

     

    Answer: (JesusWept2) "

    Enterprise Vault should recreate the directory no problem.

    So what i would do if i were you, find an item through search that has been collected and then open the item through the browser, then locate the ArchDVS file on disk, then rename or delete the folder its located in, then try and open the item again.

    You should see it open and you should see that EV has recreated the directory and reextracted the item to an ArchDVS file."

     

    OK So i just tested it and it *does* recreate the folders
    Heres the steps i followed

    Step 1: 
    Go to http://yourEVServer/EnterpriseVault/Search.asp?advanced

    Step 2:
    Uncheck "Search Attachments",
    change Sort Results to "Archived Date, Oldest First",
    change Details to "Full"

    Step 3:
    Press the Search button

    Step 4:
    Make a note of the "SeqNum" on a particular result (i.e. SeqNum: 1)

    Step 5:
    Run the following query to get its CAB file location

    SELECT A.ArchiveName, S.IndexSeqNo, S.IdChecksumHigh, S.IdChecksumLow, S.IdDateTime, S.ArchivedDate, S.IdTransaction, (PE.PartitionRootPath + '\' + C.RelativeFileName)
    FROM EnterpriseVaultDirectory.dbo.Archive A,
         EnterpriseVaultDirectory.dbo.Root R,
         EnterpriseVaultDirectory.dbo.PartitionEntry PE,
         EnterpriseVaultDirectory.dbo.VaultStoreEntry VSE,
         yourVaultStore.dbo.Saveset S,
         yourVaultStore.dbo.Collection C,
         yourVaultStore.dbo.ArchivePoint AP
    WHERE S.CollectionIdentity = C.CollectionIdentity
      AND S.ArchivePointIdentity = AP.ArchivePointIdentity
      AND AP.ArchivePointId = R.VaultEntryId
      AND R.RootIdentity = A.RootIdentity
      AND A.VaultStoreEntryId = VSE.VaultStoreEntryId
      AND VSE.VaultStoreEntryId = PE.VaultStoreEntryId
      AND PE.IdPartition = C.IdPartition
      AND S.IndexSeqNo = 1
      AND A.ArchiveName LIKE 'Your Name%'

     

    Step 6:
    From the results i can see that the CAB file this item is located in is in:
    E:\Vault Stores\myVault Ptn1\2010\01-20\B\Collection12345.CAB

    My Transaction ID is : B03A658C-C9B1-43E8-E748-F87CCD744251
    My Archived Date is : 2010-01-20 20:54:00.000

    EV will extract the item from the CAB to \yyyy\mm-dd\[first TID char]\[next three TID chars]

    So in this case it would extract to the following location
    E:\VaultStores\myVault Ptn1\2010\01-20\B\03A\B03A658CC9B143E8E748F87CCD744251.ARCHDVS

    Step 7:
    Browse to that location and ensure the directory is empty

    Step 8:
    Go to the Browser with your search open and then open up that item that you found in Search.asp

    Step 9:
    Go back to the location and ensure a new ARCHDVS file now exists

    Step 10:
    Delete or rename the three letter TID directory (i.e rename \03A\ or delete it)

    Step 11:
    Go back to the Browser search and open the item again

    Step 12:
    Ensure that the directory and the file have been recreated