EV 8 Storage Sizing for Exchange and Jounal Archiving

[Merv]

Hi All,

Need some help with sizing. I'm trying to size on the assumption of:

1. 600 user Exchange 2007 mbx with journalling enabled.

2. Assuming avg 100 mails recv per user per day.

3. 5 Year retention period.

4. Journalling enabled on a seperate vault store on the same vault store group


According to the EV 8 Perf Guide :

1. DVS(saveset):
Total number of items = 100(avg mails per day)x365(days)x5(years)x600(mbx) = 109500000 items
Total Size = 109500000(item) x 16kb(avg size of EV item) = 1670 GB

2. DVSSP(saveset shared part):
20%(assumption that 20% of emails have attachments x 109500000 items x 250 KB(Avg size of attachment)x60% (Compression)/2(assumption of number of sharers)
= 1566 GB

3. DVSSC(saveset convert content):
5%x1566 = 78GB

Total = DVS+DVSSP+DVSSC = 1670+1566+78 = 3316 GB

Next Part is about journaling:
Assumption  journalling is enabled for all mailboxes. It mentions in the perf guide:
"If items in the mailboxes have already been journaled, and the journal and mailbox partitions participate in sharing within a Vault Store Group, the shared parts have already been stored and will not be stored again. The only additional space is that used to store the DVS files. There are some exceptions to this rule, and some extra DVSSP files may be created."

So I can just add the the DVS files which will be :
1. DVS(saveset):
Total number of items = 100(avg mails per day)x365(days)x5(years)x600(mbx) = 109500000 items
Total Size = 109500000(item) x 16kb(avg size of EV item) = 1670 GB

That's giving me a Grand total = 4986 GB
Thus will 5 TB be the amt of storage required. This takes into account that there is a aggressive archviving policy and thus does not take into account deletions or any sort on Exchange or EV and storage expiry. Hoever this looks like alot. Looks like I missed something out?

Thanks / Merve

[MichelZ]

Merv

Your calculation seems to be OK, I think.

What I would change:

- 100 Mails/User/Day is very high, I believe. I would disclose myself as a Power User, and ~60 Mails/Day is much for me.
- 60% Deduplication/Compression is a high number, I would set the to less, like 40%

Keep in mind that you need storage for Indexing, too.

Cheers
Michel

[GertjanA]

Hello Merve,

Your sizing calculation probably (when done from the guide) is correct.
If you plan journal retention to also be 5 years, the above most likely is correct.
If you plan to have journal retention to be less (6 months in my environment), then you will NEED to use two sharing groups.
Otherwise expiry on the journalstore will not delete items because they are being shared in the vault-store of the mailbox.

succes.

Gertjan

[Wayne_Humphrey]

@GertjanA

I disagree, you sort of right but wrong.....

One VSG, even thou the item expires from the Journal VS, the ref count on the Journal VS will be 0 and will be higher in the Mailbox VS. You right to think it will not be deleted from disk, however there will be two copies if you use two VSGs therefore increasing the size of storage anyways, so therefor there is no benefit whatsoever going different VSG. You much better off having a single VSG.

--wayne

[GertjanA]

Hi Wayne,

sort of off-topic on this one, but here it goes.

We use journal archiving with retention of 6 months, and mailboxarchiving with retention indefinite.

If using 1 VSG, with (therefor) 1 SIS sharing, can i be sure that the mail-item is always SISsed (if I can call it this way) by the mailboxarchivingservers? Then it will not be a problem having all in one VSG

However, if an items is firsty SISsed by the journal archiving server, and needs to expire after 6 months, there are references to the item in the mailboxarchives, which have a retention of indefinite. Therefor, no expiry will take place on this sissed item (right?)
Which might lead to an enormous journalarchive..

My preference is still using a vsg for journaling and 1 for mailboxarchiving. And yes, I am aware of the potential extra storage needed, but we're already using 9TB for journalarchiving, so another 1 TB should not pose a problem ;-))

Gertjan

[TonySterling]

@GertjanA

Normally I would recommend 1 VSG but I would have two VSG's as well if you are only keeping journal items for 6 months as long as you are actually running storage expiry. 

It is one of those "same difference" kinda of scenarios.  Depending on when mailbox archiving occurs you many only have an overlap of duplication for a few months before it is expired from the journal archive.

All that said, if you are going to use this approach it is imparitive that the company runs expiry.  if they don't then you will have massive duplication.

my 2 pence on the subject.  :)

[MichelZ]

I'll agree with Wayne here...

I mean... 
If it DOES SIS, then you will only have 1 copy.
-> If the Journal one expires, it will delete the "Journal" DVS, and will leave the Mailbox DVS intact, as well as the shared parts

If it DOES NOT SIS, then you will have 2 copies.
-> If the Journal one expires, it will delete the Journal DVS and all shared parts

Difference to two VSG?
In a two VSG Scenario, you ALWAYS have 2 copies... (which gets us to 2x Indexing, 2x Storage I/O, etc etc)

Or am I missing something here?

Cheers

[Wayne_Humphrey · Accepted solution]

OK here goes,

Scenario 1
Journal item --> VGS1 --> JNVS1 --> SaveSet
Mailbox item --> VGS2 --> MBXVS1 --> SaveSet

No OSIS between the two items.

Journal item will expire, and free up space on VSG1 but item will remain in VSG2 forever.

Scenario 2
Journal item --> VGS1 --> JNVS1 --> SaveSet
Mailbox item --> VGS1 --> MBXVS1 --> SaveSet

OSIS between the two, so if you have a 5 year retention on Journal Data and a 7 year retention on MBX, you will save 5 years worth of data being OSISed.

If an item expires in JNVS1, it will get removed from the JNVS DB, but not from the MBXVS DB, so there will still be one item on disk, but in your Scenario 1, there will always be 2.

What I am saying is 6 months is nothing most people have 5 or 7 years for journaled data, if we speaking a handful of months then yes, there is no biggie on space however there is still no benefit from using Scenario 1 for a handful of months. What is the justification on using two VSG?

--wayne

[Liam_Finn1]

One item you seem to be not including. If you have 600 users mailbox archives plus journaling plus File system, you need to take into account that a medium percentage of the users will at some stage be searching the archive for emails.

Searching means high disk IOPS which means you not only need to consider the size of the data but you need to consider the IOPS required to service the requests from EV and the users searches.

Do not make the mistake of thinking capacity only you must think of number of spindles needed to provide the IOPS for performance

As a rule of thumb I always recommend low disk capacity, high disk number and whenever passable put it on RAID 10. An average 15K disk provides 180 IOPS so in a RAID 10 your write speed is X but your read speed which is what is needed for searching and retrieval is two times X

Just ensure that is kept in mind as i have seen so many designs fail to meet the need because people think space not IOPS for performance

[TonySterling]

You will always have 2x Indexing as each archive will have its own. 

What I was commenting on was the fact that the items in the Journal mailbox will only be kept for 6 months.  If the mailbox policy, for example, is set to archive older than 3 months you would only have 2x storage for items that were not deleted in the mailbox for 3 months.

I totally agree with the single VSG if the retention for the journal items is close to the retention for the mailbox items but in this scenario I don't think sharing between Vault Stores is that big of deal.

[GertjanA]

I am not sure I can explain this well enough but here goes.
if using 1 vsg
General setup:
Mailbox archiving runs during the night, retention indefinite
Journal archiving 24/7/365, retention 6 months, obviously expiry runs daily

user sends mail to other user. SIS being made by journalarchiving (files stored on journal server)
nightly archiving run, no sis as it already due to the item being sissed by journalarchiving. After 6 months, journal server tries to expire item, but it cannot as there is a reference to a mail-archived item, which has no expiry (retention indefinite). Hence, the journal archive will not be emptied.

if using 2 vsg's
General setup:
Mailbox archiving runs during the night, retention indefinite
Journal archiving 24/7/365, retention 6 months

user sends mail
sis by journal
night mailboxarchiving sis2
expiry journal works
item in mailbox remains.

In our environment, storage is not realy an issue
I've discussed this before, and this seems for us to be the way to go on 8.0.

Gertjan

[Merv]

 WOW! Guys i didn't expect everyone and I mean all the EV greats(are you guys on  summer break? ) to reply to this one and so fast..sure beats regular paid EV support. I'm gonna like update my facebook profile like now and put Fan of the Enterprise Vault Forums.

Anyway, good to know the figures checkout. Also thanks to Scanner for pointing out performance, we have planned HDS AMS 2000 or is it 200 I think FC Disk for EV. However we can't afford RAID 1. We going with normal archive and journalling no FSA.However ew probably scale it down for 200 users intially and a less aggressive archiving policy as i don't think we have 5 TB allocated. 

Another good point is other storage factors like DB and Indexes.. 12 % of 5 TB does add up. With EV taking up so much disk I got things to worry about in my calculations like  the new fingerprint DB's and all..argh!

Everyone knows that HDD prices keep falling someone like moores law and CPU speed increase so management may decide not to size for the entire retention period but monitor/manage capacity leaving maybe a 2 years allowance. Do you guys have examples / ideas on planning for capacity management and EV?

Another option is it we enable storage migration to maybe cheaper SATA disk for older data or even Tape. A 146 GB FC disk cost almost twice as much as a 1 TB SATA nowadays..don't know what's it going to be like in 2 years.

Does anyone know if Symantec has any any Excel spreadsheet like tools or if you guys have already got and would like to share a bunch of formlae similar to the exchange teams storage sizing calculator - E2K7_MBX_Storage_Calculator_16.9.xls?It would sure make things alot easier where we can enter a bunch of variables and viola you get some nice results. EV has a whole bunch of options(mail,journal,fsa,pst,CA,DA.etc),SIS,VSG configurations(like the above discussions),DB and Index requirements etc..

Michel,

60 incoming mails only? We need to spam you ....havn't you got SCOM/MOM/Backup/Replication/etcEmail alerts configured for EV events?I still a remember an old post by Scanner on the subject ..alert emails are a good way to start your day or wake you up in the middel of the night!

[Wayne_Humphrey]

I did a spreadsheet calculator when i was in Symantec Consultancy in the old days, think the last time I touched it was v5.  I will see if i get time to write a new one for v8 and the new OSIS model.  I think its a good idea and we can get everyone input inserted of one persons opinion.

--wayne

[Merv]

I've done a basic spreadsheet based on my above sizing requirements. Does not take into consideration much besides the basic assumptions given by the Perf Guide.

Anyway also found this whitepaper. Am going through it now and will post any comments here.

White paper link:
www.sun.com/storage/archive/EV7TechnicalWhitePaper-SizingforExchangeArchivingForEV.pdf

EV Sizing - simple one with just Exchange Mail and Journaling based on all mailboxes. Will update this after I've read the whiepaper :) for comong things like FSA and journaling for selected users. 

rapidshare.com/files/255729791/EV_Sizing.xls.html

Also will be good to have another for Perf considerations as this is just plain capacity.

Cheers

[MichelZ]

Merv

I'd advice you NOT to use the Sizing guide for EV7, as EV8 has a completely new storage model.

Cheers
Michel

[Merv]

Thanks just realised that ..with OSIS model now in place this EV7 sizing whitepaper is probably very far off... so we need a new one for EV 8. Any volunteers?