Expressroute Direct and Macsec

[trondaker]

Hi,

Anyone ever get Macsec towards Azure up and running? We have a IOS-XE-switch, and followed the configuration guide for Macsec with PSK here: https://www.cisco.com/c/en/us/td/docs/iosxr/ncs5500/security/62x/b-system-security-cg-ncs5500-62x/b-system-security-cg-ncs5500-62x_chapter_0101.html#concept_gjz_ysl_vcb

The second we enable "macsec network-link" toward azure, the line-protocol goes down, and show mka summary says the link is in Init-mode. Nothing happens after that, and can see no packets from the other end with debugs. Problem is Microsoft Azure-guys have had a look, and everything looks good on their end. Ideas?

[Amplify6326]

I am having a similar issue but upon having Microsoft enable SCI on their end the MKA is established and the status is showed as "secured"

sh macsec mka summary
Wed Jul 28 15:38:02.371 CDT

NODE: node0_4_CPU0
========================================================================================
Interface-Name Status Cipher-Suite KeyChain PSK/EAP CKN
========================================================================================
Te0/4/0/2 Secured GCM-AES-XPN-256 KC-MACSEC-AZURE PRIMARY 1234AB

However, even though MACsec appears to be "up" at this point we aren't able to send any traffic over the link. No ARP, ping, BGP or anything. Is there something else that needs to be configured or set up?

[trondaker]

What does 

sh macsec interface x/x/x 

say? Any invalid packets? We were seeing a lot under Receive SA stats.

[Amplify6326]

Thanks for the reply trondaker. It turns out that while Microsoft Azure support had said that they had enabled SCI they actually had not. Getting them to admit to this was very difficult. Once SCI was enabled on their equipment everything started working perfectly.

Hopefully Cisco will update their software soon and SCI can be disabled on their equipment.

[Amplify6326]

Also we were trying to use the cipher GCM-AES-XPN-256 for MACsec. We have tried to bring up Expressroute Direct in a second location and we were having the same trouble even after Microsoft Support has said that SCI is enabled. After further investigation with Microsoft Support they are saying that they don't support this cipher even though their MACsec documentation says that they do. Changing the cipher to GCM-AES-256 has made these circuits function properly. It's very strange though that MACsec in our first location is working properly with the XPN cipher.

We are using 10 Gigabit circuits and the documents from IEEE on MACsec indicate that the XPN algorithms are really only meant for circuits 40 Gigabits and higher. So maybe that's the issue? The Microsoft documents don't indicate any such caveats however.

[SIMMN]

Sorry to reply an old post...But trying to figure out if MACSec can work with expressroute going through megaport or similiar provider? Or it would be strictly for Expressroute direct?

[trondaker]

AFAIK macsec is still just a hop-by-hop layer 2 protocol, so it would need to be done on each hop through the megaport link. Maybe you could do it if its a strict layer 2 transport through the megaport, but i think there still would be thing that would break it. I dont think Azure offers Macsec on non direct-circuits either, as they configure it on their MSEE, which isnt involved in non-direct.

[SIMMN]

After posting, I found link below from Megaport.

 https://docs.megaport.com/cloud/megaport/microsoft/

“You must use untagged VLANs if you are creating a MACsec connection.”

but I am just not sure if they refer to expressroute direct via megaport OR regular provider expressroute via megaport, due to my lack of knowledge…

[trondaker]

Looks like youre right, it is supported through various L2-transports, and if they mention it for non-direct circuits you should be good to go. Ask them for your specific use case and equipment though, as there are many pitfalls.