Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
1737
Views
0
Helpful
2
Replies

Packet flow over the ACI fabric

bruno.fernandes
Level 1
Level 1

‎03-06-2016 06:14 AM - edited ‎03-01-2019 04:55 AM

Hello guys,

While looking at the following document 

http://www.cisco.com/c/en/us/support/docs/switches/nexus-9336pq-aci-spine-switch/118930-technote-aci-00.html

That explains the packet flow over the Fabric, I have the following doubts that I simply don't understand !!!

Scenario in my lab:

1x spine

2x leafs

Fabric ACI version 1.2(1i)

#1 One end-host (physical) in each leaf node, same BD and same EPG ... using the same encap_id on both enpoints/leafs

leaf101# show mac address-table
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 29 0025.645b.7298 dynamic - F F eth1/39
   29 0002.0002.0002 dynamic - F F eth1/1                             ---This is on the other leaf ... shouldn't this be pointing to a tunnel ?

leaf102# show mac address-table
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
  43 0025.645b.7298 dynamic - F F eth1/39                            ---This is on the other leaf ... shouldn't this be pointing to a tunnel ?
* 43 0002.0002.0002 dynamic - F F eth1/1

I have checked the GST table and apparently it's OK, ex: from leaf101 to reach end-host with mac 0002.0002.0002 you need to go to the TEP on the other Leaf

#2 One end-host (physical) in each leaf node, same BD and same EPG ... using different encap_id on both enpoints/leafs (on leaf101 I use encap_vlan_id 3000 and on leaf102 I use encap_vlan_id 3001) ..... In this case I only see the local entrances

leaf101# show mac address-table
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 29 0025.645b.7298 dynamic - F F eth1/39

leaf102# show mac address-table
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 44 0002.0002.0002 dynamic - F F eth1/1

Now if I check on the GST table of either leaf I don't see the remote endpoint (ex: I'm assuming that from leaf101 GST I should see the mac-address from the other end-host with mac-address 0002.0002.0002 )

Here is the result:

from leaf101 .... 0002.0002.0002 .... it's not there !!!!

module-1# show platform internal ns forwarding lst-l2

======================================================================================================================================================
TABLE INSTANCE : 0
======================================================================================================================================================
------------------------------------------------------------------------------------------------------------------------------------------------------
MO SRC P M S B C P P D S
POS O VNID Address V DE MD/PT CLSS T PTR L T PTH N P A I L P
------------------------------------------------------------------------------------------------------------------------------------------------------
1472 0 f3ffd1 00:22:bd:f8:19:ff 1 0 00/00 1 A 0 0 1 1 0 0 0 1 0 0
2852 0 f3ffd1 00:01:00:01:00:01 1 0 00/02 4002 A 0 0 0 1 0 0 0 1 0 0
3384 0 f3ffd1 00:25:64:5b:72:98 1 0 00/05 4002 A 0 0 0 1 0 0 0 1 0 0

======================================================================================================================================================
TABLE INSTANCE : 1
======================================================================================================================================================
------------------------------------------------------------------------------------------------------------------------------------------------------
MO SRC P M S B C P P D S
POS O VNID Address V DE MD/PT CLSS T PTR L T PTH N P A I L P
------------------------------------------------------------------------------------------------------------------------------------------------------
2852 0 f3ffd1 00:01:00:01:00:01 1 0 00/02 4002 A b 0 0 1 0 0 0 0 1 0
3384 0 f3ffd1 00:25:64:5b:72:98 1 0 00/05 4002 A 5 0 0 1 0 0 0 0 1 0

module-1# show platform internal ns forwarding gst-l2

======================================================================================================================================================
TABLE INSTANCE : 0
======================================================================================================================================================
------------------------------------------------------------------------------------------------------------------------------------------------------
MO SRC P M S B C P P D S
POS O VNID Address V DE MD/PT CLSS T PTR L T PTH N P A I L P
------------------------------------------------------------------------------------------------------------------------------------------------------
716 0 f3ffd1 00:01:00:01:00:01 1 0 00/00 4002 A b 0 0 1 0 0 0 0 1 0
4253 0 f3ffd1 00:25:64:5b:72:98 1 0 00/00 4002 A 5 0 0 1 0 0 0 0 1 0

Is this normal behaviour ? Where is the remote mac-address ?

Since both endpoints are on the same EPG .... I was expecting to see on leaf101 a link pointing to the TEP address of leaf102 on the GST table !!!!

I have the checked the COOP database on the spine .... and it's there .....

Argghhh What I am missing here ?

Thanks,

Bruno Fernandes

Labels:
0 Helpful
2 Replies 2

lpember
Level 1
Level 1

‎03-06-2016 02:19 PM

Hi Bruno,

Can you share the output of "show endpoint"? Also, in the first scenario are these links in a vPC or are they individual links going to each leaf from the same server using different static paths?

0 Helpful

bruno.fernandes
Level 1
Level 1
In response to lpember

‎03-06-2016 04:11 PM

Hello Ipember,

This is simple stuff, no vPCs nothing exotic I'm using static path mapping, and for each scenario I'm allways using the same two different end-host

end-host/0025.645b.7298 -- is a PC

end-host/0002.0002.0002 -- is a router 

From scenario #1 to #2 I only change the static-path encap_vlan_id from the router....from 3000 to 3001

#1 One end-host (physical) in each leaf node, same BD and same EPG ... using the same encap_id on both enpoints/leafs

leaf101# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 38 0025.645b.7298 dynamic - F F eth1/39
  38 0002.0002.0002 dynamic - F F eth1/1

leaf101# show endpoint
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
38 vlan-3000 0025.645b.7298 L eth1/39
T-A:VRF-A vlan-3000 192.168.1.200 L
38 vlan-3000 0002.0002.0002 O eth1/1
T-A:VRF-A vlan-3000 192.168.1.110 O
overlay-1 10.0.8.95 L
overlay-1 10.0.208.67 L

leaf102# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
  10 0025.645b.7298 dynamic - F F eth1/39
* 10 0002.0002.0002 dynamic - F F eth1/1

leaf102# show endpoint
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
10 vlan-3000 0025.645b.7298 O eth1/39
T-A:VRF-A vlan-3000 192.168.1.200 O
10 vlan-3000 0002.0002.0002 L eth1/1
T-A:VRF-A vlan-3000 192.168.1.110 L
overlay-1 10.0.8.93 L
overlay-1 10.0.208.67 L
7/overlay-1 vxlan-16777209 58ac.789f.7a15 L eth1/47
7/overlay-1 vxlan-16777209 58ac.78f2.1163 L eth1/46

#2 One end-host (physical) in each leaf node, same BD and same EPG ... using different encap_id on both endpoints/leafs (on leaf101 I use encap_vlan_id 3000 and on leaf102 I use encap_vlan_id 3001) ..... In this case I only see the local entrances

leaf101# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 38 0025.645b.7298 dynamic - F F eth1/39

leaf101# show endpoint
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
38 vlan-3000 0025.645b.7298 L eth1/39
T-A:VRF-A vlan-3000 192.168.1.200 L
overlay-1 10.0.8.95 L
overlay-1 10.0.208.67 L

leaf102# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 12 0002.0002.0002 dynamic - F F eth1/1
* 7 58ac.789f.7a15 dynamic - F F eth1/47
* 7 58ac.78f2.1163 dynamic - F F eth1/46
leaf102# show endpoint
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
12 vlan-3001 0002.0002.0002 L eth1/1
T-A:VRF-A vlan-3001 192.168.1.110 L
overlay-1 10.0.8.93 L
overlay-1 10.0.208.67 L
7/overlay-1 vxlan-16777209 58ac.789f.7a15 L eth1/47
7/overlay-1 vxlan-16777209 58ac.78f2.1163 L eth1/46

Thanks,

Bruno

0 Helpful
Customers Also Viewed These Support Documents

Save 25% on Day-2 Operations Add-On License