DVTI with QoS, how can I setup seperate SA's for each DSCP

[Andy Volp]

Hi, I'm testing a point to point IPSec connection using a SVTI configuration with QoS applied to the physical interface. The physical is an STM-1 link with AAL5-SANP. The QoS policy is applied to the PVC vbr-nrt.

Under load testing (generating 8 x IP streams with specific DSCP markings and a combined throughput of 130Mbps) I notice  the receive router IPSec anti-replay window is dropping traffic. Disabling the anti-replay window feature fixes the issue but this is discouraged. The production implementation will be across a private network but the customer has requested all traffic to be encrypted over the link. Dynamic routing is required as an alternate encrypted path is available, routing is used to determine best path selection.

I have read in the Cisco documentation that DVTI can support multiple SA but I'm unsure if I can use this as a fix for my QoS issue. Does anyone have a working example of how a point to point DVTI is configured to provide a separate SA for each traffic stream based on DSCP markings.

The main issue I'm trying to resolve is the limited window size of the anti-replay window (1024 max). using a single SA the anti-replay window size cannot cope with the QoS policy re-ordering of traffic so I'm hoping to configure multiple SA's to provide a separate anti replay window for each traffic type (DSCP).

The Cisco QoS design guide for VPN's mentions tuning the queue-depth setting for each class but this is very hit and miss and will require ongoing tuning to get it right, separate SA's I believe will be a better solution but I'm unsure how to implement this and can’t find a good example of how to do this.

Note. I have tried removing the QoS config, this stops the anti-replay drops but I no longer have control over dropped traffic during periods of interface congestion, QoS is a requirement.

Any help would be greatly appreciated. see example config attached.

[Marcin Latosiewicz]

Unless something changed pretty recently IPsec traffic selectors allow you to only choose IP address, IP mask, protocol number, port number (to chose source and destination). 

A couple of thoughts I'd like to share 

- You can try to queuing before this device. Obviously sub-optimal, but it's an idea. 

- Disable anti-replay, use shorter IPsec SA lifetimes instead (not exactly the same but also a possibility).

- QoS on tunnel interface instead of physical.

- If you're using... you can can have multiple GRE over IPsec tunnels between same source and destination peers. You can split traffic based on DHCP with PBR AFAIR. 

[Andy Volp]

Hi Marcin,

Thanks for the response and your thoughts.

In response to your ideas, I need to ensure priority to real-time traffic so QoS is a must, queuing before this router won't guarantee LLQ.

I have disabled anti-replay already as i have mentioned in my original post, this fixes the issue but the IPSec implementation is no longer RFC compliant although in my opinion this is not a show stopper considering the traffic is over a private network under our control. Reducing the lifetime may improve security making it more difficult for any would be attacker but that isn't the main issue.

I have tried applying the QoS policy to the SVTI (tunnel interface), this didn't improve the situation, I'm assuming QoS queuing is occurring after encryption because this didn't fix the problem. I also applied a shape average parent policy to ensure the child QoS policy was conforming to the underlying physical interface bandwidth (taking the L2 overhead in to consideration).

Multiple tunnels sounds interesting, I have thought about that but was hoping for a more practical solution through traffic selectors to create individual SA's... after all, this issue isn't new so would have thought there would be an IEEE (or similar group) working on a solution. I would be interested to hear from someone within Cisco for their take.

Thanks again Marcin.

[Marcin Latosiewicz]

Andy,

Just to be clear, disable anti-replay does not break "compliance" to RFC. 

rfc4303 mentions only that anti-replay is to be turned on by default. 

Agreed on multiple tunnels it's not very scalable in terms of deployment and maintenance. 

Multi-SA DVTI could allow you to terminate multiple IPsec SA(which are not "any any") but still with same traffic selectors. 

Regarding processing, indeed queing is done after encryption. For IEEE, I'm not sure if implementation specific aspects would be of interest, but it's an open community :-)

Not sure if anyone monitoring the forums will know anything about what we're planning, maybe punt this up to your SE?

M.

[Andy Volp]

Hi Marcin, good to know that disabling the anti-replay window doesn't make the IPSec implementation non rfc compliant. I was quoting from an older Cisco Enterprise QoS Solution Reference Network Design Guide referencing rfc2401 (which is now obsoleted by rfc4301)  specifically "As of IOS 12.3(14)T another potential remedy to QoS/IPSec Anti-Replay issues became available with the ability to expand or disable the IPSec Anti-Replay window. However it should be kept in mind that the IETF IPSec standards define the Anti-Replay window-sizes to be 64 packets and as such, this would not be a standards-compliant solution. On the other hand, the presented solution of tuning of the QoS queue-limits is fully standards-compliant."

rfc4301 has an interesting discussion point regarding the use of DSCP per SA.

"If different classes of traffic (distinguished by Differentiated

Services Code Point (DSCP) bits [NiBlBaBL98], [Gro02]) are sent on

the same SA, and if the receiver is employing the optional

anti-replay feature available in both AH and ESP, this could result

in inappropriate discarding of lower priority packets due to the

windowing mechanism used by this feature. Therefore, a sender SHOULD

put traffic of different classes, but with the same selector values,

on different SAs to support Quality of Service (QoS) appropriately.

To permit this, the IPsec implementation MUST permit establishment

and maintenance of multiple SAs between a given sender and receiver,

Kent & Seo Standards Track [Page 13]

RFC 4301 Security Architecture for IP December 2005

with the same selectors. Distribution of traffic among these

parallel SAs to support QoS is locally determined by the sender and

is not negotiated by IKE. The receiver MUST process the packets from

the different SAs without prejudice. These requirements apply to

both transport and tunnel mode SAs. In the case of tunnel mode SAs,

the DSCP values in question appear in the inner IP header. In

transport mode, the DSCP value might change en route, but this should

not cause problems with respect to IPsec processing since the value

is not employed for SA selection and MUST NOT be checked as part of

SA/packet validation. However, if significant re-ordering of packets

occurs in an SA, e.g., as a result of changes to DSCP values en

route, this may trigger packet discarding by a receiver due to

application of the anti-replay mechanism.

DISCUSSION: Although the DSCP [NiBlBaBL98, Gro02] and Explicit

Congestion Notification (ECN) [RaFlBl01] fields are not "selectors",

as that term in used in this architecture, the sender will need a

mechanism to direct packets with a given (set of) DSCP values to the

appropriate SA. This mechanism might be termed a "classifier". "

The use of the DSCP field is not used for the IPSec traffic selector, as you pointed out, but this rfc does make the point that this could be implemented (I assume as a proprietary capability) outside of the standard.

Would be a useful addition if Cisco included this for Cisco to Cisco IPSec deployments.

I'll forward this support forum discussion to our local Cisco SE and see what he can find out.

Thanks again for the reply, much appreciated.