We have seen considerable number of Intermittent packet drop if the destination is sitting behind of VPC. The most common cause of this problem is that multiple domains associated with EPG contains overlapped VLAN block. Each vlan-pool has a dedicated range of vxlan-id pre-allocated by APIC, that is why same VLAN from different pool would end up with different vxlan-id. Here are two scenarios we usually see from the field.
1. EPGs deployed on VPC links contains two domains that associate with overlapped VLAN-pool
Because of both domains contain same access-encap vlan-100 (like below), however the allocated vxlan-id on leaf101 is vxlan-8292 but vxlan-8293 on leaf102. This will result of endpoint manager (EPM) process (a NXOS process running from leaf to sync the endpoint behind of VPC to peer leaf and etc) remove the endpoint info (MAC and IP) from hardware so leaf has no idea to forward the packet. This removal is based on the logic that same access-encap vlan deployed on VPC link must have the same vxlan-id.
2. EPGs deployed on individual links contains twodomains that associate with overlappedVLAN-pool
Because of both domains contain same access-encap vlan-100 (like below), however the allocated vxlan-id on leaf101 is vxlan-8292 but vxlan-8293 on leaf102. This will result of BPDU packet received on leaf101 VLAN-100 will be dropped on leaf102 because BPDU frame is flooded strictly within the receiving VLAN by encap with vxlan-8292 but leaf102 does not use vxlan-8292 for vlan-100 but vxlan-8293.
1. How to check the vxlan-id consistency
Here is the quickest way to verify if the vxlan-id is matching between two leaf switches. Issue the command below from both leaf101 and leaf102 and compare the fabric-encap.
2.How to confirm the EP info is removed by mismatched vxlan-id
leaf101# less /var/log/dme/log/epmc-trace.txt | grep -A 15 "Unknown FD" [2017 Nov 4 23:32:37.280637631:295753369:epm_mcec_pre_process_ep_req:807:E] Unknown FD vlan/vxlan 8997 bd_vnid 14909413 ... ignoring EP req; ep_flags local|vPC|MAC|sclass|
[2017 Nov 4 23:32:37.280638877:295753370:epm_send_ep_del_ack_to_peer:1174:t] EP req for EP for which FD/BD/VRF/Tun doesn't exist, deleting EP from EP Db, if it exists
[2017 Nov 4 23:32:37.280640484:295753371:epm_process_ep_del:2300:t] Delete req rcvd for EP: [2017 Nov 4 23:32:37.280642648:295753372:epm_debug_dump_epm_ep:398:t] log_collect_ep_event
mac = 0000.1111.2222; num_ips = 0
vlan = 21; epg_vnid = 8599; bd_vnid = 14909414; vrf_vnid = 2195457
ifindex = 0x16000000; tun_ifindex = 0; vtep_tun_ifindex = 0
sclass = 32779; ref_cnt = 4
flags = local|vPC|MAC|sclass|timer|
create_ts = 11/04/2017 15:32:59.046167
upd_ts = 11/04/2017 15:32:59.046167
VLAN pool is a bucket for VLAN ID and VxLAN IDs.
Each EPG/AEP could associate multiple domains, but each domain must associate with a vlan-pool containing unique vlan-block that is not overlapped with any other vlan-pool. This is to ensure the global consistent vlan-to-vxlan mapping.
If the design is for port-local VLAN use case, that is a different story.
Hi all, We are implementing Cisco ACI to the network, we are on management, monitoring part and i have a few concerns/questions.We are configuring NTP, Syslog, SNMP, Inband management, but the needs according current network is that we need dif...
ACI Fabric start learning external remote IP’s locally on the Leaf switches . Due to this reason, traffic is locally dropped on Leaf Sw03 instead of going outside the fabric. This issue basically interrupted most of the External & Internal services be...
We upgraded ACI from 2.2(2k) to 2.2(4p) and upgraded one group of leaf switches after APIC/spine upgrade.APIC showed those leaf switches were upgraded successfully, but actual leaf switches were not reloaded and not upgraded.Have you seen such a thing?&nb...
Transforming Day 2 Ops with Cisco Data Center Network Assurance and Insights
Live webinar: Network Insider Series
Wednesday, July 15, 202010:00 AM Pacific Standard Time (San Francisco, GTM -08:00) Driven by the growth of applications, deluge of data and t...