2270 and 2227 errors when restoring .p7s attachments

[EDU]

EV for Exchange 7.5 SP3

Win 2003 Ent

Centera

We've been having a problem when our users try to restore certain messages that are encrypted or signed. The messages do not restore, and may or may not produce an error on the client.

The EV event logs contain a 2270 and 2227 error for each of the items in question. Sometimes there is a 6391 error as well.

I've noticed that with many, if not all, of these messages, there is an attachment with a .p7s extension (smime.p7s, for example). These are the certificates, and I am guessing that they are somehow the cause of the problem.

On a possibly related topic, we get a number of 6592 errors during the nightly archiving runs. They don't have much information and say that an "Abnormal error occurred". Running dtrace showed that the errors were produced when trying to archive an item with an invalid or expired certificate. Further examination seemed to show that the items were being successfully archived, and it was just the conversion that was failing. The items were restorable in all of our tests.

It seems to me like these two things are related somehow. I've got an open case with tech support, but would appreciate any help that anyone can provide.

Many of our users encrypt a substantial amount of email and losing access to it causes big problems.

Regards,

Ethan

Message Edited by EDU on 01-06-2009 10:09 AM

[EDU]

Well, it turns out this problem is a lot more complicated than I thought. We have a number of users who are unable to restore certain SMIME messages directly from EV to their Exchange mailbox.

Here's a hypthetical example inspired by true life: A user has a folder in the vault containing numerous messages, including some SMIME signed emails.

1. They attempt to restore the entire 83 messages in the archive folder using Archive Explorer. The progress bar chugs merrily along and tells you at the end that "Items have been moved."

2. The user checks the target folder in Outlook and sees only 81 of the 83 messages.

3. The user goes back into Archive Explorer and (if they know how to search/refresh the view) they see that the last two messages remain in the vault. These messages can be previewed, they can be saved locally and opened, but they cannot be restored to Exchange directly. If they are exported to a .pst on the server by an administrator, they can be opened. Attempting to restore them to Exchange results in 2227 and 2270 errors on the server, just saying that it failed. Dtrace doesn't really give any additional information. Just that it failed for an unknown reason.

4. After working on this for several weeks, I made the connection between this problem and an issue relating to 6592 errors in the EV server logs. 6592 errors (Abnormal error occurred) occur when EV is trying to archive an item and it's unable to do so. While checking into these, I realized that they were caused by SMIME signed messages in Exchange. These messages cannot be opened. Turns out that the SMIME.p7m signature file attachment had been stripped out and EV wouldn't archive them. Apparently all of the ones that are IN the vault already were put there by a version prior to 2007 SP3 (which is what we're using.)

So, my question is: Has anyone see problems archiving corrupt messages, such as ones of the type IPM.Note.SMIME.multipart that do not have the SMIME.p7m file attachment?

Something seems to be corrupting messages and making it impossible to vault/unvault these messages. If I edit the message type to be IPM.Note using MFCMAPI, I am able to open the message with Outlook. I can also archive the message, but I may still be unable to restore it.

Symantec support has no record of this type of problem occurring before. Microsoft support says that it's clear that something has removed the file attachment, but I'm not able to determine what or how.

[EDU]

This seems to have been caused by EV vaulting SMIME messages that had their SMIME attachment stripped out while in Exchange.

It appears as if previous versions (7.0 and earlier) would archive messages of the class IPM.Note.SMIME.* even if the required SMIME.* file attachment was missing. We also upgraded from Exchange 2007 from 2003 around the same time, so it could be related to that also.

Once these messages are in the vault, they can be previewed, but not restored back to Exchange (which apparently rejects the EV MAPI connection when it tries to restore an SMIME-class message that doesn't have an SMIME attachment.

EV doesn't handle the error well, so the restore appears to have been successful, but no restored message appears.

So we have something in our environment that is occasionally stripping SMIME attachments while the messages are sitting in the Exchange mailbox. Version 7.5 SP3 is now skipping over these messages (6592 event ID: Abnormal error occurred) instead of vaulting them.

I guess that's good, but now I need to figure out how many of these non-restorable corrupt messages are sitting in our archive. How do I search for them? 14 TB, 70 million messages...

-Ethan

[Joseph_Rodgers]

Ethan,

FYI: Thanks for your own follow-up.  S/MIME archiving just came up and I appreciate the finding.

-Joe

[EDU]

It looks like the actual problem is that the s/mime file attachment is getting corrupted, possibly during the archiving process.

It seems to be happening with some encrypted/signed messages that contain more than one attachment. In these cases, the attached documents are encoded and stored as nested attachments within a single file along with the digital signature.

The problem is that the smime.p7s file attachment that contains the "real" attachments gets messed up somehow. When you attempt to restore the message from EV, there seems to be data missing from the smime.p7s attachment. One of the referenced attached documents is an empty file. A good analogy might be to imagine that the smime.p7s file is a container and the top gets ripped off and the contents are scattered about and damaged.

The other attachments and the message body are okay.

I suspect that this happens with messages created by certain versions of Outlook Express, (as evidenced by the headers of the "bad" messages - X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.xxxx) but I haven't been able to reproduce the problem.

These are the versions of Outlook Express listed in the corrupt messages within EV:

V6.00.3790.2663
V6.00.3790.2757
V6.00.3790.2826
V6.00.3790.2929
V6.00.3790.2992
V6.00.3790.4133

I'm still not any closer to a solution for this problem than I was 6 months ago.

There is apparently no way to search EV for messages with this problem. Symantec support originally told me that Discovery Accelerator would do the trick. I got an evaluation license to try it out, but it didn't work. I opened another support case and this time was told that Discovery Accelerator doesn't index the content of S/MIME attachments (encrypted or not) and so there was no way to search for affected messages.

Ethan