9800-CL - SWPORT-4-MAC_CONFLICT Issues

[Jegan Rajappa]

I am not sure how many of you have seen problem like my environment. Almost in all my 9800-CL HA SSO & standalone deployment I have started seeing SWPORT-4-MAC_CONFLICT Issues.

---------

Dec 15 09:47:59.848 UTC: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:001 TS:00000000699009892497 %SWPORT-4-MAC_CONFLICT: Dynamic mac 001E.1420.DBFF from GigabitEthernet2 conflict with SVI, please check the network topology and make sure there is no loop.

---------

wlc1#show int gigabitEthernet 2 | inc address
Hardware is CSR vNIC, address is 0050.5691.861b (bia 0050.5691.861b)
wlc1#show int vlan 224 | inc address
Hardware is Ethernet SVI, address is 001e.1420.dbff (bia 001e.1420.dbff)
Internet address is 10.160.224.41/24
wlc1#

---------

The packet went out from WLC1 comes back to WLC1, this can be seeing by capturing tcpdump in 9800-CL appliance side and vSwitch side, this is because of setting vSwitch to accept Promiscuous mode.

Following VMware KB article https://kb.vmware.com/s/article/59235 has a fix, once this change is applied in ESXi host, VMs (9800-CL) appliances to be reloaded, this problem would disappear once 9800-CL appliances are reloaded.

Resolution

To prevent this issue, you must enable the /Net/ReversePathFwdCheckPromisc setting:
esxcli system settings advanced set -o /Net/ReversePathFwdCheckPromisc -i 1

To check that the setting is enabled, run the following command:
esxcli system settings advanced list -o /Net/ReversePathFwdCheckPromisc
Path: /Net/ReversePathFwdCheckPromisc
Type: integer
Int Value: 1
Default Int Value: 0
Min Value: 0
Max Value: 1
String Value:
Default String Value:
Valid Characters:
Description: Block duplicate packet in a teamed environment when the virtual switch is set to Promiscuous mode

Note:

KB article says 'Duplicate Multicast or Broadcast Packets are Received by a Virtual Machine When the Interface is Operating in Promiscuous Mode', based on my observation unicast traffic is also received and making changes as per vmware KB article fixes the issues.

[marce1000]

         - (possibly) -> https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvt96686

  M.

[Feds]

Thanks marce1000, we also fall into that scenario (old 5500 pair + new 9800-CL pair in parallel) but currently there's no roaming between the two as 9800-CL has no APs. Conflict is between Wlclient and Gi2 and MACs are those of wireless clients on 9800-CL (getting to it via mobility peer with yet another 9800). Took a packet capture for ARP traffic on 9800-CL and I see a lot of duplicated packets originating on a wireless client on 9800-CL coming back into Gi2.
Cheers
Fed S.

[Cheah Lit Thor]

Hi,

May i know will this symptoms cause Flexconnect wireless users packet drop? my controllers have this exactly same error messages and i'm not sure my wireless client intermittently packet drop are due on this issue, but I'm observed that most of the time my wireless client had request time out the WLC9800CL will generate this message with the laptop mac address.

Please advise Thanks

[chandlerbr]

Thank you for pointing out the VMware KB article.  Worked like a charm with zero negative affects noted over two weeks of monitoring.  Chased this issue since deploying 9800-CLs staring with version 16.x and it is still not "fixed" in 17.6.1 as CSCvt96686 claims.

Good stuff Jegan.  Thank you!

[th3r1dd1ck]

Confirmed to work on the following:

Hyperflex Chasis All Flash version 4.0.2a-35199

Vmware Hypervisor 6.7

C9800-CL Code 17.4.3a

We were experiencing the same issue. the KB article from vmware removed this error.

thanks for the late night work. really helped us out.

[Temur Kalandia]

can you please share configs at vswitch and port group level ? how did you configured promiscious modes on both levels ?

[Scott Fella]

Take a look at the 9800-CL guide as that will show you what you need to do on the hypervisor depending on what hypervisor you are using.

https://www.cisco.com/c/en/us/products/collateral/wireless/catalyst-9800-cl-wireless-controller-cloud/nb-06-cat9800-cl-wirel-cloud-dep-guide-cte-en.html

[Temur Kalandia]

hi, 

i have read this doc, there are two parts which confuses me:

"By default, a hypervisor vSwitch is configured to reject promiscuous mode. If the 9800-CL is using tagged traffic (for a management VLAN, AP VLAN, etc.) via the management port, promiscuous mode needs to be set to accept in order for the vSwitch to carry tagged traffic" this is quote from the above doc, it says that promiscous mode should be accepted at vSwitch , not at port group level. this setup will couse many unnesessary traffic visible from virtual machines. 

further reading the same document , there another thread : "Both Promiscuous mode and Forged Transmits need to be set to Accept on the port group where the 9800-CL is connected" , based on both information should i reject promiscious mode at vSWitch level and accept it at port group level ? for me this looks like more correct way.

[Feds]

Hi Jegan and all,

Thanks for this post and providing the solution.
The KB article states "guest OS reboot" is sufficient. Given you have fixed the issue, did you have to just reboot the VM or actually power-cycling it from vSphere?
And if it has to be power cycled, can it be done via vSphere, if you know? I don't see a "halt" or "shutdown" command  (ISE has halt for example) nor I can find this quesiton answered anywhere.
We manage a HA pair of 9800-CL, 3rd party said they changed the hosts setting and I performed a couple of "redundancy force-switchovers" (failover/failback) which included reload of ex-Active WLC however MAC conflicts are still there. 3rd party is telling me VMware actually meant "power off VM" and not "reboot"...

Thanks heaps.
Fed S.

[Feds]

Replying to my own question re safely halt/shutdown a 9800-CL, as found in Cisco doco somewhere - issue "reload pause" and then VM can be safely shut or power cycled.

[Scott Fella]

Shut it down from the VM host or if you want to reboot, you can either do it from the host or from the 9800 CLI. No big deal, I have force shut it with no issues.

[Temur Kalandia]

Hello, 

i have an issue when WLC logs SWPORT-4-MAC_CONFLICT messages for interface gig2 (which is trunk created for client data). we have promiscious mode enable at vswitch level and portgroup level.

maybe we have to set promiscious mode to reject at vswitch level and set it accepted at port group level ? does it prevents duplicated packets ?