QoS Scheduling before or after IPSec enryption

[JUANNN]

Hello,

I am a little confused about the following statement on the document Security and VPN Configuration Guide, Cisco IOS XE 17.x - IPsec Anti-Replay Window Expanding and Disabling [Cisco IOS XE 17] - Cisco

"Moving QoS scheduling before the encryption may solve the anti-replay issue but would render the QoS functionality useless. "

Why would placing the shceduler before the crypto engine would eliminate QoS functionality? I was thinking that packets get queued and then the egress packets, already ordered by priority thanks to the queues and the scheduler, just get the IPSec sequence number assigned and get out of the device. If the answer is that encrypting the packet obscures the QoS relevant headers so downstream routers cannot aplly QoS, wouldn't that happen also if the scheduler is placed after the crypto engine? 

[Ramblin Tech]

I have no specific knowledge of how IOS-XE implements IPSEC, which almost certainly has Platform Dependencies based on whether there is hardware available for forwarding and/or crypto. That said, QoS getting FUBARed if the crypto takes place after the QoS scheduler does make sense to me.

The QoS scheduler serializes the output from multiple, parallel queues by deciding the transmission order based on configured (or default) QoS policies. If after being serialized by the scheduler, a packet then needs to be sent to the crypto engine for encryption, that packet will be subject to additional latency due to the crypto process itself. In my limited IPSEC experience, I have found that crypto engines tended to have less throughput than the router's forwarding engine, which means that packets could be placed in output transmission queues for the scheduler to serialize faster than the crypto engine could work its magic. This could lead to additional queueing at the crypto engine as packets needing crypto service are serialized faster than the crypto engine can provide that service. This would add even more and variable latency on top of that needed for the crypto process itself. This extra latency is not accounted for in the configured QoS policies.

And what of the packets not needing crypto service originally serialized behind a packet needing crypto? Should those packets be blocked, HoL-style, waiting for the preceding packet to be encrypted and transmitted, or... should these succeeding packets be transmitted ahead of the previously prioritized packets needing crypto service? Either all packets are taxed by additional latency or the scheduler's priority order is mangled; one of the two will happen. Again, this unpredictable treatment is not accounted for in the configured QoS policies. 

If the QoS policies are enforced post-crypto, encrypted packets will have already incurred extra latency when placed in the output queue, but policies will work as expected.

[JUANNN]

Hello,

Thanks for the answer. I think that the main reason why packets are sent first to the crypto engine and then to the scheduler is because doing it the opposite way will not help with the anti-replay windows problem. Nothing guarantees that intermediate routers may reorder packets arbitrarily, so I thinks is not good to assume that packets will arrive in the same order they were transmitted. If packets get reordered on an intermediate hop, then the anti-replay windows issue comes again, no matter if packets go to the crypto engine of the first router before or after the scheduler. So to solve the issue, Cisco implements Multi-SNS:

Security and VPN Configuration Guide, Cisco IOS XE 17.x - IPsec Anti-Replay Window Expanding and Disabling [Cisco IOS XE 17] - Cisco

Note how even Cisco says that is still possible that "reordering within a number space could happen elsewhere in the network", so definitely reordering of packets that were on different queues is even more possible.

[ahollifield]

Why do you need QoS at all? Do you actually have a real bandwidth problem?

[JUANNN]

We use QoS in my organization to prioritize some traffic over other for critical situations and yes, we sometimes have bandwith problems depending on the transport we are using