Credentials prompt accessing archived emails in public folder

[Tifosa]

Hi and help please,

We already have a call logged with our support team already over this but if anyone from Symantec can help it would be helpful.

We've started getting prompted for user credentials when we try and access (not recall, although that is failing as well) a lot of public folder archived emails.

We have 8 public folders servers, running through 1 EV 9.0.2 EV server (this server is only used for the public folder servers). We also have 4 mailbox EV servers which for user vaults is running OK.

MBXVault1.domain.com, MBXVault2.domain.com, MBXVault3.domain.com, MBXVault4.domain.com and PFVault.domain.com

All EV servers are 2008 64bit

The error....

1. Users get prompted... Windows Security. Connecting to MBXVault 4.domain.com.

2. When they enter in the usernames it reprompt with login box

3. NOTE: The "Connecting to" server is a mailbox server and NOT the EV public folder server. The server is random depending on what EV server the users are on.

I've checked IIS permissions - Basic and IWA is ENABLED. ASP.NET Impersonation is ENABLED

I've checked the EV Desktop policy and the Intranet Zones are correct. They contain the EV alias' and FQDN of the servers

I've checked the Public Folder policy and the 'Inherit Permissions" is set to ON

I've cheked the Internet Explorer proxy settings, they are blank. Automatically Detect Settings, is enabled

Outlook EV logging on maximum shows this;

15/10/2012 14:22:48.584[5608]: ~DesktopCommonConfig::GetSetting: 0x0
15/10/2012 14:22:48.584[5608]: ~CDesktop::get_Setting: 0x0
15/10/2012 14:22:48.584[5608]: CShortcutItem::Display: 0x0
15/10/2012 14:22:48.584[5608]: CDesktop::CheckIfClientDisabled: 0x0
15/10/2012 14:22:48.585[5608]: ~CDesktop::CheckIfClientDisabled: 0x0
15/10/2012 14:22:48.585[5608]: CShortcutItem::Display...Return Last Failed error: 0x80004005
15/10/2012 14:22:48.585[5608]: ~CShortcutItem::Display: 0x80004005

DTRACEing on the EV Servers show this;

EV:M ClientAuthHelperImpl::GenAuthString Authentication Type: Currently impersonated user Client:(null) ==> AuthToken:SGBD012333.wsatkins.com llt4*****
40076 13:43:43.843  [9760] (w3wp) <8896> EV:L CAuthHelper::Reset Cancel registration? True CancelId: 12146096
40078 13:43:43.858  [9760] (w3wp) <5604> EV:L CBaseDirectoryServiceWrapper::CreateDirectoryService() - Entry [m_nNumTries = 40]
40079 13:43:43.858  [9760] (w3wp) <5604> EV:L CBaseDirectoryServiceWrapper::CreateDirectoryService() - Successfully communicated with an EV Directory Service on the local machine
40080 13:43:43.858  [9760] (w3wp) <8896> EV:L {CAutoJournalAccessor::GetSyncSlot} (Exit) Status: [Success]
40081 13:43:43.858  [9760] (w3wp) <8896> EV-L {Slot.GetSyncSlot} Slot.GetSyncSlot - VEID:1721F921FB15CC04088661D9666CAC94D1110000EVMAILSITE, SyncSlot:4e93aad4-1f13-4cfe-8bc8-d365ea754809, TimeOut:300
40082 13:43:43.858  [9760] (w3wp) <5604> EV:M CDirectoryVaultObject::GetAttributeListFromType Attribute list for type ArchiveFolderView : RootIdentity,FolderName,FolderPath,ParentFolderRootIdentity,FolderIcon,CustomId,CustomQual,SID,VaultEntryId,Type,Hidden,ContainerRootIdentity,OwningTrusteeIdentity,AutoSecurityDesc,ManualSecurityDesc,ArchiveVEID,ParentFolderVEID
40084 13:43:43.858  [9760] (w3wp) <5604> EV:L CBaseDirectoryServiceWrapper::CreateDirectoryService() - Entry [m_nNumTries = 40]
40085 13:43:43.858  [9760] (w3wp) <5604> EV:L CBaseDirectoryServiceWrapper::CreateDirectoryService() - Successfully communicated with an EV Directory Service on the local machine
40086 13:43:43.874  [9760] (w3wp) <5604> EV:M CDirectoryVaultObject::GetAttributeListFromType Attribute list for type ArchiveView : RootIdentity,VaultStoreEntryId,ArchiveName,ArchiveDescription,ArchiveAdminNote,ArchiveStatus,ArchiveLimitSize,ArchiveLimitOperation,ArchiveLimitStatus,ArchiveNotifyOperation,ArchiveNotifySize,DeleteExpiredItems,DeleteProtected,Structured,ArchiveFull,IndexingLevel,LocaleId,SID,VaultEntryId,Type,Hidden,ContainerRootIdentity,OwningTrusteeIdentity,AutoSecurityDesc,ManualSecurityDesc,IndexItemsMode,ConvertedContentMode,IgnoreIndexingPolicy,RetainArchiveHistory,HasHistory
40087 13:43:43.874  [9760] (w3wp) <5604> EV~W Event ID: 6287 Unable to fetch item from "EVMAIL05". |Reason: Access denied      (0xc0041801) |Saveset Id: 201001250000000~200712211517270000~Z~00D8D09296FCF68307CEB8A8FB78CA51 |Archive Name: All Public Folders|Archive Folder Path: �GBEMC - Epsom, The Wells�Water�Projects�Water & Environment (096)�5059933-STW PR09 |Reference: [GOAFS] |
40088 13:43:43.874  [9760] (w3wp) <5604> EV:H {CAutoStorageOnline::GetOnlineAttachmentFileSize} (Exit) Status: [Access denied      (0xc0041801)]

This is affecting a lot of our users and we would really like to know what has gone on to cause this?

We are currently rerunning the public folder archiving and shortcut processing on the servers to try and fix. Any help would be appreciated

Thanks

[Jeff_Shotton · Accepted solution]

So the fact that your clients are trying to access their local storage server for the item is normal. With Enterprise Vault 8 the storage service determined which server the requests for the recall went to, rather than the site alias, which was what was used in EV2007.

Now for your issue, a few things could have happened, but the most obvious one that springs to mind is that the public folder archive/archive folder permissions have been blanked - which could happen after an issue during the synch process (the permissions are wiped and re-written per folder each archiving run)

To determine if this is your issue, load up permissionsbrowser.exe, which will be found in the root Enterprise Vault installation folder (C:\program files\enterprise vault\  by default), select the public folder archive and browse through the folder structure looking to see if permissions have been synched, or are simply blank.

As this isn't affecting client item recall (at least, you haven't complained of this) i doubt that there is anything specifically wrong with IIS and auth and the way it is behaving. I also doubt it is a SPN (service principal name) issue.

If it is a synch problem you are already taking the right course of action and likely you will post here that it magically fixed itself. Fingers crossed :)

Regards,

Jeff

[Jeff_Shotton]

Oh and as a root cause, I've seen this happen when GC lookups were not happening in time, necessitating increasing the number of TCP connections via the maxuserports reg key

Regards,

Jeff

[Tifosa]

Hi Jeff. Your logic is correct.

Things moved on slightly since my original post. When I checked the permissionbrowser.exe and selected the All Public Folders vault I can see that the majority of the folders on one of our public folder servers was blank. Since re-running the task they are (hopefully) repopulating with the permissions and access is very slowly returning, but with over 256K folders it is taking it's time.

This has happened in the past and is getting more frequent, so this fix is well known to us. My real annoying question is why do they lose the permissions?

Surely once they've synced they should stay? and if there's a problem it should simply not update them and not blanket them!

[Tifosa]

Thanks Jeff, ignoe my preivous comment - posted after yours, :)

Do you know where the maxuserports reg key is set and what a good number would be?

[Jeff_Shotton]

Its all in the best practice technote: http://www.symantec.com/docs/TECH56172

However, I dont really condone setting the value to such a high level on windows 2008, since the functionality is very different to windows 2003 - originally it was a range of ports and now it is a maximum number of concurrent ports.

You may also find your GC's are being overloaded, so you might need some tuning there.

There is a bit on how to tune SQL regarding this setting - it will work just as well for EV :

http://support.microsoft.com/kb/328476

Regards,

Jeff