MacSec Switch to Switch via Radios

[simoesmarco8626982]

Hi everyone, 

hope you can give me your opinion about this.

Have a customer that is being adamant that he wants MacSec enabled between the switches (Catalyst 9300), now the switches are connected to each via two Siklu Radios 80Ghz that provide 10Gbps.

I configured the switches with the following config that I learned from my training course from CBT Nuggets:

cts manual

sap pmk "KEY" 

Cisco states that we shouldn't have anything in the middle of a MacSec link, but curiously enough everything is working as it should. OSPF is working normally, bfd is working correctly, everything seems to be good. 

Now I'm having a weird issue where I'm using two EXFO testers, and when doing a bandwidth test when we start reaching the limit of the interface I start to have Out of Sequence Packets. If I disable macsec everything is good, but having it enabled creates out of sequence packets when we start reaching the limit at around 9Gbps.

Now I have to say, I did the exact same thing in other links that are configured in the same way and have the same hardware and was able to pass the full throughput without any issues, no packets loss and no out of sequence. But in one of the links I can only get rid of the out of sequence if I disable MacSec. 

Could the others tests have been a red haring that everything passed correctly? What is your opinion in a system configured with MacSec like this? Would you install or tell the customer that it shouldn't (this was already told).

Thank you for your help

[pman]

Hi,

The first thing that comes to my mind is Replay Protection, 

When frames are transmitted through the network, there is a possibility that frames get out of the ordered sequence. MACsec provides a configurable window that accepts a specified number of out-of-sequence frames.

Example:

interface te1/1/1
   macsec replay-protection windows-size 100

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/216849-troubleshoot-macsec-on-catalyst-9000.html#toc-hId-1810419820

[simoesmarco8626982]

Hi @pman 

Thank you for the reply.

For what I see the replay protection will allow the link to accept out of sequence packets, but the weird is I'm having out of sequence packets in the testers but no packet loss. Replay protection automatically drops the out of sequence frames and my links are configured for 0 where no out of sequence is accepted. In fact I can't have any out of sequence whatsoever, the links have to be flawless even at full speed. If this is not the case my customer will not accept the job.