deleting of email archive does not reduce space used on file system

[elbutre]

hi

i am running EV 10 (just upgraded from EV9) and have the following issue

i have 28 email archive occupying 119gb of data according to evault reporting, the total space used on the file system is 130gb.

when i deleted an old email archive evault reporting correctly reduces the amount of data from 119gb to 117gb (2gb email archive) but the space occupied on the file system remains 130gb

the EV server runs on a windows 2008 r2 server using normal NTFS partion (no compression)

i am about to delete 3 more email archives which total about 15gb in size. i am hoping to achieve a reduction in disk space used on the file  system as at this rate it will continue to grow while the total email archive size according to EV reporting actually shrinks.

would anyone know why this discprency exists? and how to resolve it if possible?

many thanks

[FreKac2]

Possible reasons:

Due to single instance storage the items that should be deleted are shared by other archives, if that is the case EV won't delete the items from disk since other archives use them.

Collections, if you use collections (.cab files) they won't be re-packaged until the threshold for sparse collections is reached.

[TonySterling]

Are you sure the vault store partition is the only thing on the file system, for example, are your indexes on the same drive?

As FreKac pointed out, collections can effect this as well.

[elbutre]

yes i use both collections and single instant storage.

i think it must be the collections, how do i check the threshold settings?

i don't think it is single instant storage as i would think some emails are shared but probably 90% or unique to that persons archive

i am only checking the folder with the vault partitions on that drive and not the entire drive for size.

many thanks

[TonySterling · Accepted solution]

I recommend this article and technote:

https://www-secure.symantec.com/connect/articles/how-collections-and-sparse-collections-work

How do Sparse Collections and the SparseCollectionPercentage setting work?

Article: TECH87400

|

Created: 2009-01-02

|

Updated: 2011-09-02

|

Article URL http://www.symantec.com/docs/TECH87400

This should help you understand how that is going to work.

[GCSPT]

On the single instance side of things remember it is not necessarily the whole item we are sharing. The users may have individual mail bodies\headers erc but the attachments may be the same which may take up the majority of space. Osis white paper from EV 8 but this hasn't changed for 10 I don't think. https://www-secure.symantec.com/connect/articles/whitepaper-enterprise-vault-80-optimized-single-instance-storage-osis

[elbutre]

hi

i have ran the 3 SQL statements as per the technote 87400

i am not sure if this is working correctly for me or not after reading this line

By default, If the count of items that still exist in a collection reaches 5% (For EV 2007) or 30% (for EV 8) of the total number of items in the collection, then the collector restores the active saveset files in the collection file back to their original location. 

I am running ev10 sp1 so assume it is 30% just like EV 8 (and higher??)

but when i run the SQL statement i can see that when i set the % to 30% there are 2562 cab files affected

so already i have 2562 cab files that should not be there as i have not altered the default settings or am i misunderstanding how this works?

i found some cab files with less than 5% (600 cab files)

although when i looked at the size of these cab files they were quite small so i suspect they have been compressed.

best regards

[JesusWept3]

you are still running the collections process on that machine right? because if it doesn't, then the sparse collections process will never ever run

oh and also CAB files used by collections are never compressed, they're store only

[elbutre]

yes every day between 10am and 4pm, all files older than 10days, max cab size is 10mb

thanks re compression. my bad

I just looked up the SparseCollectionPercentage in the Parition Entry table 

for partition 1 (i have 2) it is set to 5 (started as kvs v5 or v6)

for partition 2 it is set to 30 (started with evault v9)

which is the default setting. 

i guess i will have to alter the sparsecollection percentage to 30 for partition 1 to free up some space but it will have quite an impact on the server so not sure it is worthwhile.

don't understand how some cab files have less than 5% item count. they should have been emptied out under the sparsecollection process?

regards

[JesusWept3]

thats if there really is 5% free or not, the collection table isn't the most accurate in the world unfortunately
You might find that if you do a count on the Saveset table, that there are more items that belong to that collection than the collection table reports

So for instance if you look at the table and it shows it has 4 out of 100 (<5%) and then you do a SELECT COUNT(*) FROM Saveset WHERE CollectionIdentity = 12345 you may find that it has references to 6 savesets (>5%) which would stop the whole thing

If that is the case you would either up the percentages for the sparse collection, but really you would want to run EVSVR to repair the cab counts

[elbutre]

so i ran SELECT COUNT(*) FROM Saveset WHERE CollectionIdentity = 563  (there is a collection file called collection563.cab which has 1 out of 80 files

after running this sql query it returns 1

is this the query you meant, sorry no big sql query expert

i also copied the cab file to my computer to check the contents and it contains 80 .dvs files

if i did the correct sql query it looks like my ev server is not working as per its 5% setting

so am not sure if i were to increase the percentage from 5% to 30% whether it would actually do something

i guess i could increase it to 10% and then see what happens (over weekend when it is not busy)

how would i run EVSVR to repair the cab counts?