Footer stamping of messages larger than 10 MB

[pvdberg00]

We had some mail processing and memory consumption issues

on our C650's. It appears with large outgoing messages and her is the advice of Support:

"

these work queue issues (and several related application faults) were due to the injection of large messages, which led to mail processing and memory consumption issues.

One of the messages (MID 33566083) was approximately 72 MB in size (subject

"RE: Arrangement fee letter"), and another (MID 33519221) was 60 MB in size (subject "RE: xxx REDI CAL stukken / interim controle E&Y").  For all of our customers (regardless of platform and AsyncOS version), we do not advise the footer stamping of messages larger than 10 MB in size for this very reason.

It appears as though this issue has occurred several times since (the most recent being on Friday at approximately 6:51 PM), so it would be recommended that you update each of your disclaimer-related Outgoing Content Filters to include the following condition:

Message Size [is less than] 10485760 bytes

"

Our customer wants to stamp every outgoing message with a disclaimer (even the large messages). Is there anybody familair with this issue??

Peter.

[Christopher Smith]

Hello Peter,

I completely understand your frustration. I know this can be a difficult issue to deal with when you have senders that want to send large messages. Hopefully I can provide you with a bit more insight into this issue. One of the most vital resources for the IronPort appliance is memory.  For every message that the appliance processes, memory is used. This is basically like a memory object. The larger the message the larger the object. Larger message not only take up more memory, they can also consume other resources like processor time in that we take longer to perform other operations like anti-virus and anti-spam scanning as well as regex pattern matching for filters. If we extend too far into memory usage and exhaust all of our physical RAM we utilize swap space (as most operating systems do). The problem here is this can slow things down and compound the problem if there is a sudden burst of messages , especially large messages. The appliance is very intelligent in that it can detect increases in resource utilization and throughput and throttle back on services and processing using resource conservation mode. This should typically keep the appliance out of trouble and allow it to continue once the storm is over , so to speak.  This does come with limits however.  Depending the size of the appliance, the configuration settings, the volume and size of the traffic one could extend into a situation where we see application faults due to memory starvation.  In your case this is actually multiplied by a factor of 2. 

  Footer stamping requires making an in-memory copy of the message and can essentially double the RAM usage need by the core processes if applied to all messages.  So for example the 72MB message that was referenced now becomes a 144MB object. Now if there are a lot of messages of this size things can add up very quickly.  The recommended workaround will help but it obviously comes at a price (not being able to footer stamp all large messages). Unfortunately there is not a perfect answer for this other than limiting the footer stamping to the smaller messages or limiting the overall message size. In the past most would suggest ftp or scp for transfer of larger files but with the advent of greater bandwidth its sometimes a hard argument to make. I think in its original conception , no one expected an MTA to accomodate even a 5mb message much less 144mb but we have come a long way with bandwidth since then.

Christopher C Smith
CSE

Cisco IronPort Customer Support 

[pvdberg00]

Thanks for the answer.

Can you give me a list of limitations of the appliance ?

Peter

[Christopher Smith]

HI Peter,

As you know we have multiple models each with different levels of capacity.  From the C160 which would be the lowest capacity to the X1060 which would be the highest for example.  When an SE (Sales Engineer) performs and assessment through capacity planning many factors have to be considered.  Typically the starting point is a measurement of raw throughput in message per hour with all services turned off. By services I mean anti-spam, anti-virus, and LDAP for example. These of course would be base line figures and many other factors have to be considered including message size. Additionally processing messages using complex filters or filters with large dictionary lookups has to be considered. These are typically things that can get expensive in the way of resources.

Not trying to give you a canned answer here but I think the best answer is to the question what are the limitations of my system in its current configuration in my environment. The reason I say this is that it is not likely to see a system running at he baseline figures since the appliance would only be used for throughput and little or no processing. Each customers needs are different and not every customer processes the same types of messages all the time. I would recommend having a SE or a CSE ( through an SR)  take a look at your configuration and the statistics on your appliance so that we can tell you where the limits are and how close , or far , you are from those limits.  We would also be able to give you recommendations on any configuration changes to help improve things if needed.

Christopher C Smith

CSE
Cisco IronPort Customer Support 

[exMSW4319]

I note that some of the messages you are sending are very large indeed. Above 50Mb I would have thought that some recipients would be rejecting the messages on simple message size, or is that view now out of date? Does anyone else have a figure they generally work to as a rule-of-thumb?

I'm guessing wildly based on the subjects posted, but a simple letter shouldn't require an attachment of that size. You might need to supply a bit more desktop support (for example, take their scanner away until they promise to be good) in order to cut down on file sizes, if all of those attachments are hitting your file servers and internal mail system.

Finally, do make sure that all the large attachments are justifiable under whatever acceptable usage policy applies in your organisation. It's not one large message that sinks an appliance, it's a sustained sequence of them. I always use content control rules to stop all of the multimedia attachment types (not just the ones in the built-in media class) though your commercial requirements may differ. Don't let your budget pick up the bill for "Friday Fun"!