MACsec for industrial context ....

[sylvain.benoit]

Hi all, 

I'm trying to configure and use MACsec but the only things I can have are REBOOT !

I've read different document from CISCO (I try to understand, I deduce but I'm never sure about what I do ...), even the good articles from @Tim Glen but my use case is different and I success only to reboot every time I enable MACsec on an interface.

I use 4 IE3400 switches, in a RSTP loop.

After my investigation I think the issue could be in the key exchange/synchro between the switches ... but I haven't the solution.

My setting looks like :

Define a MKA  

>enable Enter privileged mode

#configure terminal

(config)#mka policy epas

(config-mka-policy)#send-secure-announcements

(config-mka-policy)#key-server priority 255

(config-mka-policy)#include-icv-indicator

(config-mka-policy)#macsec-cipher-suite gcm-aes-128

(config-mka-policy)#confidentiality-offset 30

(config-mka-policy)#end  

Define a key chain  

>enable Enter privileged mode

#configure terminal

(config)#key chain epas_keys macsec

(config-keychain)#key 1000

(config-keychain-key)#cryptographic-algorithm gcm-aes-128

(config-keychain-key)#key-string 12345678901234567890123456789012

(config-keychain-key)#lifetime local 12:12:00 July 28 2016 infinite

(config-keychain-key)#end  

Define MACsec with MKA and KeyChain  

>enable Enter privileged mode

#configure terminal

(config)#interface GigabitEthernet1/3

(config-if)#mka policy epas

(config-if)#mka pre-shared-key key-chain epas_keys

(config-if)#macsec replay-protection window-size 30  

(config-if)#macsec network-link  

(config-if)#authentication linksec policy must-secure

(config-if)#macsec ==> start macsec mechanism

The output :

S2(config)#interface GigabitEthernet1/3
S2(config-if)#macsec


Exception to IOS Thread:
Frame pointer 0x556AA651F8, PC = 0x556263B344

UNIX-EXT-SIGNAL: Segmentation fault(11), Process = Exec
-Traceback= 1#617e2f1027e37110b3ee03b07208884b :55598EB000+3B0DFE8 :55598EB000+3B0D834 iosd_crb_petra_unix:7F8177A000+6D588 linux-vdso:7F851C9000+5F4 :55598EB000+8D50344 :55598EB000+42013BC :55598EB000+41FABC0 :55598EB000+5D68FD8 :55598EB000+5D65630 :55598EB000+4FFA8DC :55598EB000+5D90F64 :55598EB000+2CA1D98

Fastpath Thread backtrace:
-Traceback= 1#617e2f1027e37110b3ee03b07208884b iosd_crb_petra_unix:7F8177A000+6D6C0 linux-vdso:7F851C9000+5F4 c:7F70AF9000+D102C iosd_crb_petra_unix:7F8177A000+3A120 prelib:7F81758000+7980 pthread:7F708CA000+7DD4 c:7F70AF9000+D0E9C

Packet Thread backtrace:
-Traceback= 1#617e2f1027e37110b3ee03b07208884b iosd_crb_petra_unix:7F8177A000+6D6C0 linux-vdso:7F851C9000+5F4 c:7F70AF9000+D102C iosd_crb_petra_unix:7F8177A000+65D78 prelib:7F81758000+7980 pthread:7F708CA000+7DD4 c:7F70AF9000+D0E9C

Auxiliary Thread backtrace:
-Traceback= 1#617e2f1027e37110b3ee03b07208884b iosd_crb_petra_unix:7F8177A000+6D6C0 linux-vdso:7F851C9000+5F4 pthread:7F708CA000+E1C0 iosd_crb_petra_unix:7F8177A000+73ADC prelib:7F81758000+7980 pthread:7F708CA000+7DD4 c:7F70AF9000+D0E9C

Worker Thread 1 backtrace:
-Traceback= 1#617e2f1027e37110b3ee03b07208884b iosd_crb_petra_unix:7F8177A000+6D6C0 linux-vdso:7F851C9000+5F4 pthread:7F708CA000+E1C0 thpool:7F7BFEA000+1DF0 prelib:7F81758000+7980 pthread:7F708CA000+7DD4 c:7F70AF9000+D0E9C

Worker Thread 2 backtrace:
-Traceback= 1#617e2f1027e37110b3ee03b07208884b iosd_crb_petra_unix:7F8177A000+6D6C0 linux-vdso:7F851C9000+5F4 pthread:7F708CA000+E1C0 thpool:7F7BFEA000+1DF0 prelib:7F81758000+7980 pthread:7F708CA000+7DD4 c:7F70AF9000+D0E9C

R0 = 00000000 R1 = 80005562AFDA07 R2 = 00000004 R3 = 00000000
R4 = 7F6A685118 R5 = 7F6A685010 R6 = 00000000 R7 = 00000000
R8 = 00000000 R9 = 00000000
R10 = 13000000 R11 = 01000000 R12 = 00000003 R13 = 00000000
R14 = 7F70C1A0C0 R15 = 00000000 R16 = 7F7CDB90E8 R17 = 7F70BBD280
R18 = 00000000 R19 = 00000000 R20 = 55682E66E0 R21 = 55682E66D8
R22 = 556AA651F8 R23 = 7F47D35C30 R24 = 00000001 R25 = 7F47D37018
R26 = 00000002 R27 = 12000048 R28 = 00000001 R29 = 556AA651F8
R30 = 556263B314 SP = 7F4C85E0F0
LR = 556263B314 PC = 556263B344

all my IE3400 run the firmware v17.10.01

I really appreciate if someone can help me. 

Thanks,

Sylvain

[Tim Glen]

Hi @sylvain.benoit 

  Having the same key-server priority on all devices isn't good. I also see you've enabled a lot non-default settings, send secure announcement, replay protections, confidentiality offset, etc, I'd suggest removing most of those and enabling if \when each is required. I'd try removing the following from your MKA policy.

send-secure-announcements
key-server priority 255
include-icv-indicator
confidentiality-offset 30

I'd remove this from your Key Chain,  in Cat9K IOS XE the key wont expire, by using the infinite keyword here you are doing the same thing so this is redundant. 

lifetime local 12:12:00 July 28 2016 infinite

I'd remove this from your interface config, must secure is enabled by default and lets simplify the config by removing replay protection which isn't mandatory. 

macsec replay-protection window-size 30  
authentication linksec policy must-secure

Hope this helps, let me know how you make out after adjusting your config. 

Tim

[sylvain.benoit]

thank you for this answer and I will test it. I understand and agree with the idea of keep it simple. But at the end I will need to add some security options.

I've an extra question. The four switches are in a RSTP loop. I configure the first switch and after the second .... the fourth. But I can't enable the 4 switches at the same time. What about the RSTP loop (and STP frames) when the first switch has MACsec enabled and the second not yet ?

thank a lot

Sylvain

[elliot perrignon]

Hi Tim Glen, I've tested this configuration on cisco industrial IE3400 rugged series switch and they always crash and reboot when I activate the macsec command on an interface. Also, I don't have the "macsec network-link", "authentication linksec policy must-secure", "macsec access-control" settings. Here's a screenshot of your configuration and mine:

Can you tell me what my problem is? Please note that the IOS version of the switch is 17.10.1. Thank you in advance.

Elliot Perrignon

[Tim Glen]

HI @elliot perrignon 

I'll be upfront, I've never been able to set hands on an IE switch for MACsec testing...

I'm not sure if it matters, but are you using an IE3400 Heavy Duty or an IE3400 Rugged?

I've found and reviewed the Security Config Guide for IE3400 but it's for 17.3, I wasn't able to find one for 17.10. 
https://www.cisco.com/c/en/us/td/docs/switches/lan/cisco_ie3X00/software/17_3/b_security_17-3_iot_switch_cg/m-macsec-protocol.html#id_117513

I very quickly reviewed the bug database but didn't see anything. I'd suggest you look as well as the IE shouldn't be rebooting.  I'd maybe try it with 17.6(5) to see if that eliminates the crashing after entering the MACsec interface command. 

Let me know how you make out. 

Tim

[sylvain.benoit]

Hi Tim,

I work with Elliot and we use Rugged serie.

We will update all of our IE3400 to the v17.12 to avoid any problem with official support. I've read documents (from CISCO) about MACsec and very often it mentionned the a licence is required. In my understanding even with "network essential" we can use MACsec-128, the only option is "Advanced network" to be able to use AES256. I don't know if this could pose a problem for us, if so a restart due to lack of license (regulatory or commercial) is a bit violent in terms of error management. But I don't think so.

Regards,

Sylvain

[Tim Glen]

Hi @sylvain.benoit  & @elliot perrignon 

Yes you are correct.

MACsec can encrypt data plane traffic with both GCM-AES-128 & GCM-AES-256.

In order to use GCM-AES-256 the Network Advantage license is required.

Using GCM-AES-128 is possible with both the Network Essentials or the Network Advantage license.

Hope this helps!

Tim

[sylvain.benoit · Accepted solution]

Hi all,

To close this thread I would like to add my final (and verified) information. On industrial switch offer from CISCO (IE3xxx) MACsec 128 doesn't work until you install the v17.13. Official support team has (finally) recognized they have a lot of issues in the firmware. The new version (17.13.1) is available since December, we have upgraded our 4 devices (IE3400) and MACsec works fine now.

regards,

Sylvain