0xe0008488 - Access is denied

[Dave_Rutz]

Running BackupExec 11d ver 6235. (upgraded from 10d). All BackupExec and Windows updates are installed. Exchange backup is configured to backup individual mailboxes as well as do an IS backup (reported as �snapshot full�).

BackupExec is installed on a Windows 2003 Standard server in one subnet (xxx.xxx.11.6) in Domain A. Exchange is installed on a Windows 2003 standard server (RAID5) as a domain controller in another subnet (xxx.xxx.10.6) in Domain B (DMZ). An account on each domain is created with the appropriate rights to perform the backups.

The Exchange server is running McAfee VirusScan Enterprise 8.5.0i. Exclusions are set so as to not scan the IS or log files. The server is also running Sunbelt Messaging Ninja.

Email communications and DNS work perfectly. Ever since the upgrade to 11d, I am having issues backing up Exchange 2003.

The job log shows

Job ended: Tuesday, January 30, 2007 at 7:50:13 AM
Completed status: Failed
Final error: 0xe0008488 - Access is denied.
Final error category: Security Errors

For additional information regarding this error refer to link V-79-57344-33928

Under that, there are many entries similar to:

Backup- \\MAIL\Microsoft Exchange Mailboxes
V-79-57344-33928 - Access is denied.
Access denied to file User1 Top of Information Store.Inbox.FW: email1.
V-79-57344-33928 - Access is denied.
Access denied to file User1 Top of Information Store.Inbox.Re: email2.
V-79-57344-33928 - Access is denied.
Access denied to file User1 Top of Information Store.Inbox.Re: email3.
V-79-57344-33928 - Access is denied.
Access denied to file User1 Top of Information Store.Inbox.RE: email1.
V-79-57344-33928 - Access is denied.
Access denied to file User1 Top of Information Store.Sent Items.RE: email2.
V-79-57344-33928 - Access is denied.
Access denied to file User1 Top of Information Store.Sent Items.RE: email3.

Backup- \\MAIL\Microsoft Exchange Mailboxes
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User1 Top of Information Store.Inbox.FW: email1" is a corrupt file.
This file cannot verify.
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User1 Top of Information Store.Inbox. Re: email2" is a corrupt file.
This file cannot verify.
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User1 Top of Information Store.Inbox. Re: email3" is a corrupt file.
This file cannot verify.
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User2 Top of Information Store.Inbox. RE: email1" is a corrupt file.
This file cannot verify.
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User2 Top of Information Store. Sent Items.RE: email2" is a corrupt file.
This file cannot verify.
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User2 Top of Information Store. Sent Items.RE: email3" is a corrupt file.
This file cannot verify.

Using System Manager, I have moved the Information Store, Public folders and Log files from one partition to another. The backup errors still occur.

I have run the ESEUTIL and ISINTEG tools as recommended by Microsoft and, after correcting a few errors that were found, the results come back without errors. The backup errors still occur but on different mailboxes.

This problem has been going on for some time now meaning that all previous backups for the last month or so include the same errors.

[Bradley_Willads]

Dave,

Are you using the new GRT technology to backup your Exchange Mailboxes, or are you still using the legacy method?

If you're using the new GRT technology, I believe you are experiencing the issue outlined in this document:

Performing an Exchange Information Store Backup with the "Enable the restore of individual mail messages and folders from Information Store backups" selected, AND the backup job is directed to TAPE, results in a job status of failed and Access is Denied is reported in the job log.
http://support.veritas.com/docs/286144


If you're still using the legacy method for backups, have you logged into outlook as the same account you have specified for the backup job, and attempted to open those pieces of mail, in that users mailbox? Typically this will result in the same error, which usually results in the items being corrupted somehow, or the user restricing access on their own.

Brad Willadsen
Symantec

Backup Exec for Windows Servers Free E-mail Support
http://seer.entsupport.symantec.com/email_forms/intro_emailsup.asp

[Dave_Rutz]

Using the legacy method... No CPS, no GRT. Backup time is not an issue with only about 40 users and 12GB of email.
The "Enable the restore of individual mail messages and folders from Information Store backups" option is not checked.
Backups are directed to tape.

After logging into Outlook with the backup account and trying to open another user's Inbox folder, I get the error "The operation failed" when clicking on the "Names..." button.

In the Application event log, there was the following warning:
OALGen could not generate full details for some entries in the offline address list for address list '\Global Address List'. To see which entries are affected, event logging for the offline address list generator must be set to at least medium.
- Default Offline Address List

I set the logging to medium as suggested. I closed Outlook and reopened it, logging in with the backup account again. I proceeded to open a user's Inbox and was able to open the Global Address Box to select one of the concerned users. I can see the contents of their Inbox and I can open one of the emails reported as corrupt.

[Bradley_Willads]

Dave,

Are you running any Antivirus within Exchange itself, which scans email that is incoming/outgoing? This has been known to cause problems like this as well, with the legacy method for backing up mailboxes.

[Dave_Rutz]

McAfee VirusScan Enterprise 8.5 is configured to not scan email files and folders. We're also running Sunbelt Messaging Ninja that has its own incorporated anti-virus. Sunbelt have supplied 2 scripts to run before and after backing up with BackupExec
(http://support.sunbelt-software.com/scripts/rightnow.cfg/php.exe/enduser/std_adp.php?p_sid=Rux*Tdti&p_lva=&p_faqid=1494&p_created=1155238883&p_sp=cF9zcmNoPTEmcF9ncmlkc29ydD0mcF9yb3dfY250PTI5JnBfc2VhcmNoX3RleHQ9JnBfc2VhcmNoX3R5cGU9MyZwX3Byb2RfbHZsMT02NiZwX3Byb2RfbHZsMj1_YW55fiZwX3NvcnRfYnk9ZGZsdCZwX3BhZ2U9MQ**&p_li=).
These will disable then re-enable Sunbelt AV scanning. I ran the "disable" script before the backup started last night but the results are the same this morning (first lines of error log):

Backup- \\MAIL\Microsoft Exchange Mailboxes
V-79-57344-33928 - Access is denied.
Access denied to file Susan Murray .Top of Information Store.Inbox.Re: test1.

Backup- \\MAIL\Microsoft Exchange Mailboxes
WARNING: "\\MAIL\Microsoft Exchange Mailboxes\User1 .Top of Information Store.Inbox.Re: test1" is a corrupt file.
This file cannot verify.

There are no warnings or errors in the event logs on either servers pertaining to the backup job except the one saying that the job failed.

[Bradley_Willads]

Dave,

I found this Technote article that has the same symptoms your talking about here. Can you please follow it, and see if it resolves your issue?

http://support.veritas.com/docs/267692

If it doesn't help, can you check your Exchange servers' Application log to see if there are any errors occurring during the mailbox backups?

[Wes_Stewart]

I have had the same problem since going from 10 to 11. I have done the MaxObjsPerMapiSession registry entries with no success. I do know the problem starts as soon as the BackupExec Clients starts crashing on the Exchange server:

Event Type:Error
Event Source:Application Error
Event Category:(100)
Event ID:1000
Date:2/1/2007
Time:10:24:58 PM
User:N/A
Computer:TRINITY
Description:
Faulting application beremote.exe, version 11.0.6235.0, faulting module BEDSMBox.dll, version 11.0.6235.1, fault address 0x0001f659.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74 Applicat
0008: 69 6f 6e 20 46 61 69 6c ion Fail
0010: 75 72 65 20 20 62 65 72 ure ber
0018: 65 6d 6f 74 65 2e 65 78 emote.ex
0020: 65 20 31 31 2e 30 2e 36 e 11.0.6
0028: 32 33 35 2e 30 20 69 6e 235.0 in
0030: 20 42 45 44 53 4d 42 6f BEDSMBo
0038: 78 2e 64 6c 6c 20 31 31 x.dll 11
0040: 2e 30 2e 36 32 33 35 2e .0.6235.
0048: 31 20 61 74 20 6f 66 66 1 at off
0050: 73 65 74 20 30 30 30 31 set 0001
0058: 66 36 35 39 f659

[Bradley_Willads]

Wes,

This issue appears to be a little different than Dave's. Can you start your own thread, and I will assist you on that thread?

[Dave_Rutz]

Since I had already noticed an event ID 9646 on Thursday, I added the MaxObjsPerMapiSession key and the objtMessage object giving it a value of 500. During Thursday's backup (mailboxes), another event ID 9646 was logged, this time complaining about objtStream. I added that object to the registry key and give it a value of 500 too.

Another error that is logged is:
Ninja.VSAPI failed to complete Ninja Store Scanner request to retrieve message envelope and data. In order to resolve this problem please try to restart both Microsoft Exchange Store and Ninja Store Scanner services.

I rebooted the Exchange server (no errors in the event logs) and left things the way they were. Friday's backup job completed successfully with no errors in the log files! I ran another backup this morning to make sure that it wasn't just a fluke... job completed successfully. I guess there's no reason other than speed to run the Sunbelt AV scripts. I have put the scripts in the pre/post commands. We'll see...

Getting rid of the 9646 errors seems to have solved this problem since the Exchange server had been rebooted several times before.

Thanks for your assistance.

[Bradley_Willads]

No problem Dave. Glad we could help you.

[Andrew_Lewis]

Hi all

Getting the same problem as Dave here since the upgrade to version 11d from 9.1when doing to bricks level backup run.

Get 9646 errors to which Symantec support's answer is to increase registry settings.

I have tried updating the objtMessage settings in the registry on the Exchange server. We then get each time an error on another 9646 event

Mapi session "/o=Company Ltd/ou=site/cn=Recipients/cn=ServiceAccount" exceeded the maximum of 350 objects of type "objtMessage".

Mapi session "/o=Company Ltd/ou=site/cn=Recipients/cn=ServiceAccount" exceeded the maximum of 500 objects of type "objtStream".

Mapi session "/o=Company Ltd/ou=site/cn=Recipients/cn=ServiceAccount" exceeded the maximum of 500 objects of type "objtAttachment".

Now the settings have been tripled to each its original value as below but I still get the 9646 error even with the new values. How much I am meant to keep increasing the values to I don't know. I have read some reports that badly written software is the blame for these errors and that adjusting the figures by too much can make EXCHANGE unstable.


objtMessage 750

objtAttachment 750

objtStream 750

Main errror
Error category : Security ErrorsError : e0008488 - Access is denied.
For additional information regarding this error refer to link V-79-57344-33928


We also like Dave get the 100's of errors in BE job log reporting
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsAccepted: SMQ3 - Ops to help meeting" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsAccepted: Updated: SMQ3 - Ops to help meeting" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsRE: Phones" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsPhone List" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsRE: tonight" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsRE: tonight" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsFW: snr managment" is a corrupt file.
This file cannot verify.
WARNING: "\\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsRE: Help!" is a corrupt file.
This file cannot verify.

and in job history 100's of.........
Information StoreSent ItemsDevserver01 rebootFile is corrupt : \\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsDomains for SEATFile is corrupt : \\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsDriving myself: Thai EveningFile is corrupt : \\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsDriving myself: Thai EveningFile is corrupt : \\exchange01.domain.com\Microsoft Exchange Mailboxes\Andrew Lewis Top of Information StoreSent ItemsEmailing: hdclone.3.2.4.se.en.zip

Do we think that there is a genuine bug in 11d or has anyone got any ideas? Nothing has changed apart from the 11d upgrade!

Many thanks

Andrew

[ClarkR]

I am also having this same issue.  After upgrading to 11d, i'm experiencing the event ID 9646 on the exchange server the same errors in the job history log.  This is very frustating.  In the meantime, I'm taking a full backup of our info store.  The funny thing is that the differential mailbox backups work without recieving the error. 

[ClarkR]

I've since contacted Symantec one more time about this issue.  I was told that the way MAPI works in 11d is different than 10d.  I was advised to be aware of this, and that it was a microsoft issue.  Basically, the issue is unresolvable by Symantec.  I tend to disagree with this.  If the way MAPI works was changed, it can be changed again to supply a resolution to this problem. 

[Andrew_Lewis]

Guess what .....nearly 4 months living with this problem and still being fobbed off my Symantec support that there is no known fix. How can a support case go on so long without any attention?

I need bricks level backups working as I cannot use GRT as I do not have enough disk space on the media server to stage a 20GB information store!

Any body else managed to get bricks level working properly?

regards

Andrew

[rud3_80y]

Having the same problem here... GRT D2D2T doesn't work so they say to do two backups, one to disk and one to tape. Then you can't see GRT selcections when on tape and who has enough disk space to keep backups for indefinite periods?!

Or they say switch to legacy backup methods, but they just S the bed too. Was this product even tested on Exchange servers before it was released?! Why is it so different from 10.1? Why doesn't it work?

This is insane. This is costing my organization time and money.

[Rob_Gray]

Same error etc.  Only for the public folder store.
Have tried setting objtMessage to 750 & streams to 500 & attachments to 750.
This got the backup further along and no longer attachments error.
Got the streams error before the messages error 9646.
Now trying objtMessage = 1200 , Streams = 1000



All this because Sym changed the way they use MAPI. For once, not a  Microsoft change causing errors.
Cant believe they never tested brick backup on a realistic exchange system.

Wish I had never upgraded from 10.0 to 11d . Should have just done 10 to 10d as needed some update to backup sharepoint. So so I was told....

[Rob_Gray]

Brick backup works fine with the HUGE increase in object limits.

Backed up 128131 mail messages in 8338 folders in 0 mailbox(es)
Processed 21,014,293,761 bytes in  3 hours,  22 minutes, and  47 seconds.
Throughput rate: 98.8 MB/min