We are having problems connecting our Nexus 2k to HP server ports in LACP mode (channel-group mode active).
The below is snippet from logs, before the port goes into suspended mode.
2013 Jun 7 17:14:21 PBR-core-1-PoC %LACP-FEX104-3-SYN_COLL_DIS_EN: WARNING: Potential Interop issue on [Ethernet104/1/7(0x1f670180)]: SYNC, COLLECT and DISTRIBUTE flags enabled too early by partner
ERROR: Cannot set/reset lacp suspend-individual for port-channel305 that is admin up
PBR-core-1-PoC(config-if)# 2013 Jun 7 17:14:41 PBR-core-1-PoC %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet104/1/7: Ethernet104/1/7 is suspended
2013 Jun 7 17:14:40 PBR-core-1-PoC %LACP-FEX104-5-LACP_SUSPEND_INDIVIDUAL: LACP port Ethernet104/1/7(0x1f670180) of port-channel port-channel305(0x16000130) not receiving any LACP BPDUs suspending (individual) port
Cisco: Nexus 7000 6.1.4 with N2K-C2248TP-1GE 6.1.4
HP Ethernet 1Gb 4-port 331FLR Adapter running Windows 2008 R2 sp1, Broadcom Driver Version : 220.127.116.11
Has anyone had success with LACP and this setup?
HP Ethernet 1Gb 4-port 331FLR Adapter
I've not come across the first message before, but the second message (%LACP-FEX104-5-LACP_SUSPEND_INDIVIDUAL) is because the switch is not seeing any LACPDU (Link Aggregation Control Protocol Data Units) on the link from the server.
What teaming mode options do you have on the server? You need something like 802.3ad Dynamic, which means it will be running an IEEE 802.3ad Link Aggregate with LACP as the control protocol. Note that you might see references to 802.1ax instead of 802.3ad as that's the correct name for the link aggregation standard these days.
Yes, the server teaming is set to Dynamic. We systematically tried every single combination, the only one which works is static LAG (channel-group mode on for Cisco, switch-load balancing for HP).
I'm not sure if you already found this, but the first error is documented, for the Nexus 5000 series at least in the System Messages and Recovery Procedures for the Cisco Nexus 5000 Family:
Error Message: LACP-3-SYN_COLL_DIS_EN: [chars] [[chars]]: SYNC, COLLECT and DISTRIBUTE flags enabled too early by partner
Explanation: Potential interop issue. Partner system seems to have enabled sync as well collecting or distributing flags too early even before actor has selected an aggregator
Recommended Action: No action is required.
I just love the Recommended Action
The message is indicating that the LACPDU from the server has the Synchronise, Collecting and Distributing flags all set on the first LACPDU it receives. When the link is first established the LACPDU is only supposed to have the Synchronise flag set, with the Collecting and Distributing flags set once the switch and host are sync'd. There's a prety good diagram of this at Networking Bodges in the Bringing Links Up section.
On the basis of this it would seem this is possibly a bug in the Broadcom driver. I see on the Broadcom website that version 18.104.22.168 of the driver is available. Are you able to try that driver on this server?
Thanks Steve, I did see that error message, however didn't understand the full implications. You have given me some good leads to follow. Will try some different drivers.
Even with Broadcom driver 22.214.171.124 we still see these interop messages.The vPC comes up ok and this is only an interop warning messge so I am not sure if it actually affects any end user experience?
I have this same issue with a DL360 G8. I put two servers in at the same time same server same nic etc and only have this issue on one of the two servers. Will keep combing the forums on this and respond if I get anything on it,
The LACP Timeout on the switch and the server needs to be configured with the same value. Short Timeout (1s) or Long Timeout (30s).
You can do show lacp interface ethernet1xx/y/z
The results will show the switch (Local) LACP_Timeout and the server (Neighbour)
Cisco only supports ISSU when LACP neighbours are using long timeouts, so would recommend all neighbours are using Long Timeouts only for that reason.