cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
383
Views
1
Helpful
5
Replies

Explanation about interns Vlans and IPs ACI APIC

willytech007
Level 1
Level 1

Hi community

I'm studying about the certification 300-620 ACI exam and topics.

I got some terms and concepts about 3 different IPs and I got confused, they are: Infraestructure VLAN- IP VLAN, PTEP IP and FTEP IP, they are 3 differents IPs, my questiosn are: what's the function each one? some area the same in all switches? some examples about their use?

 

Thanks in advance

5 Replies 5

RedNectar
VIP Alumni
VIP Alumni

Hi @willytech007 ,

Let's start with the easy one - the PTEP or physical TEP

PTEP Addresses

As each LEAF switch boots up, it requests a DHCP address from the primary APIC. It assigns this address to lo0 within the overlay-1 VRF, and is pretty much used to identify that leaf for everything. For instance, all the following commands will show the VTEP address:

apic1# show switch
 ID    Pod   Address       In-Band IPv4  OOB IPv4     Version        Flags   Name
 ----  ----  ------------  ------------  ------------ -------------- -----   ---------
 1101  1     10.1.112.65   10.10.2.8     172.16.11.8  n9000-15.3(2a) asiv    Spine1101
 1201  1     10.1.112.64   10.10.2.5     172.16.11.5  n9000-15.3(2a) aliv    Leaf1201
 1202  1     10.1.112.66   10.10.2.6     172.16.11.6  n9000-15.3(2a) aliv    Leaf1202

and

apic1# fabric 1201-1202 show ip interface lo0 vrf overlay-1
----------------------------------------------------------------
 Node 1201 (Leaf1201)
----------------------------------------------------------------
IP Interface Status for VRF "overlay-1"
lo0, Interface status: protocol-up/link-up/admin-up, iod: 4, mode: ptep
  IP address: 10.1.112.64, IP subnet: 10.1.112.64/32
  IP broadcast address: 255.255.255.255
  IP primary address route-preference: 0, tag: 0


----------------------------------------------------------------
 Node 1202 (Leaf1202)
----------------------------------------------------------------
IP Interface Status for VRF "overlay-1"
lo0, Interface status: protocol-up/link-up/admin-up, iod: 4, mode: ptep
  IP address: 10.1.112.66, IP subnet: 10.1.112.66/32
  IP broadcast address: 255.255.255.255
  IP primary address route-preference: 0, tag: 0

and

apic1# fabric 1201-1202 show isis dteps vrf overlay-1
----------------------------------------------------------------
 Node 1201 (Leaf1201)
----------------------------------------------------------------

IS-IS Dynamic Tunnel End Point (DTEP) database:
DTEP-Address       Role    Encapsulation   Type
10.1.8.65          SPINE   N/A             PHYSICAL,PROXY-ACAST-MAC
10.1.8.67          SPINE   N/A             PHYSICAL,PROXY-ACAST-V4
10.1.112.65        SPINE   N/A             PHYSICAL #VTEP of Spine
10.1.8.66          SPINE   N/A             PHYSICAL,PROXY-ACAST-V6
10.1.8.64          LEAF    N/A             PHYSICAL
10.1.112.66        LEAF    N/A             PHYSICAL #VTEP of OTHER leaf


----------------------------------------------------------------
 Node 1202 (Leaf1202)
----------------------------------------------------------------

IS-IS Dynamic Tunnel End Point (DTEP) database:
DTEP-Address       Role    Encapsulation   Type
10.1.112.65        SPINE   N/A             PHYSICAL #VTEP of Spine
10.1.8.66          SPINE   N/A             PHYSICAL,PROXY-ACAST-V6
10.1.8.65          SPINE   N/A             PHYSICAL,PROXY-ACAST-MAC
10.1.8.67          SPINE   N/A             PHYSICAL,PROXY-ACAST-V4
10.1.112.64        LEAF    N/A             PHYSICAL
10.1.8.64          LEAF    N/A             PHYSICAL #VTEP of OTHER leaf

and

apic1# fabric 1201-1202 show ip interface brief vrf overlay-1
----------------------------------------------------------------
 Node 1201 (Leaf1201)
----------------------------------------------------------------
IP Interface Status for VRF "overlay-1"(4)
Interface            Address              Interface Status
eth1/49              unassigned           protocol-down/link-down/admin-up
eth1/50              unassigned           protocol-down/link-down/admin-up
eth1/51              unassigned           protocol-up/link-up/admin-up
eth1/51.8            unnumbered           protocol-up/link-up/admin-up
                     (lo0)
eth1/52              unassigned           protocol-down/link-down/admin-up
eth1/53              unassigned           protocol-down/link-down/admin-up
eth1/54              unassigned           protocol-down/link-down/admin-up
vlan7                10.1.0.30/27         protocol-up/link-up/admin-up
lo0                  10.1.112.64/32       protocol-up/link-up/admin-up #VTEP Address
lo1                  10.1.8.64/32         protocol-up/link-up/admin-up
lo1023               10.1.0.32/32         protocol-up/link-up/admin-up. #FTEP Address


----------------------------------------------------------------
 Node 1202 (Leaf1202)
----------------------------------------------------------------
IP Interface Status for VRF "overlay-1"(4)
Interface            Address              Interface Status
eth1/49              unassigned           protocol-down/link-down/admin-up
eth1/50              unassigned           protocol-down/link-down/admin-up
eth1/51              unassigned           protocol-up/link-up/admin-up
eth1/51.10           unnumbered           protocol-up/link-up/admin-up
                     (lo0)
eth1/52              unassigned           protocol-down/link-down/admin-up
eth1/53              unassigned           protocol-down/link-down/admin-up
eth1/54              unassigned           protocol-down/link-down/admin-up
vlan7                10.1.0.30/27         protocol-up/link-up/admin-up
lo0                  10.1.112.66/32       protocol-up/link-up/admin-up #VTEP Address
lo1                  10.1.8.64/32         protocol-up/link-up/admin-up
lo1023               10.1.0.32/32         protocol-up/link-up/admin-up #FTEP Address

FTEP Address

Now - if you look at this last example, you'll see another loopback address lo1023 . This is the FTEP address, (Fabric TEP address I THINK) that is the same on all switches [edit: all leaf switches] in the fabric - note in the example above, it is 10.1.0.32/32  on both switches. It is used if the switch has a vSwitch attached that is using VXLAN encapsulation between vSwitches (i.e. in certain VMM environments)

 "Infraestructure VLAN- IP VLAN"

Now I'm a bit confused about this one - I'm not sure what you are referring to with "Infraestructure VLAN- IP VLAN"

The Infrastructure VLAN is the VLAN used between leaf switches and the APIC. The only place you'll see an IP on the Infrastructure VLAN is on the APIC. On my APIC, the infrastructure VLAN is VLAN 3961, so...

apic1# ifconfig bond0.3961
bond0.3961: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.1.0.1  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::36ed:1bff:fe8b:5a2b  prefixlen 64  scopeid 0x20
        ether 34:ed:1b:8b:5a:2b  txqueuelen 1000  (Ethernet)
        RX packets 90334557  bytes 60247196551 (60.2 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 64888897  bytes 33932495663 (33.9 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

You'll see that each leaf has a tunnel between this APIC IP and it's PTEP - on my Leaf1202 it happens to be on Tunnel1so you'll see it with this command:

apic1# fabric 1202 show interface tunnel 1
----------------------------------------------------------------
 Node 1202 (Leaf1202)
----------------------------------------------------------------
Tunnel1 is up
    MTU 9000 bytes, BW 0 Kbit
    Transport protocol is in VRF "overlay-1"
    Tunnel protocol/transport is ivxlan
    Tunnel source 10.1.112.66/32 (lo0)  #This switch's PTEP
    Tunnel destination 10.1.0.1         #APIC infrastructure VLAN IP address
    Last clearing of "show interface" counters never
    Tx
    0 packets output, 1 minute output rate 0 packets/sec
    Rx
    0 packets input, 1 minute input rate 0 packets/sec

so I'm assuming that this answers your question - if not, give me a bit more detail about this one.

RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

Hi dear

I have a question about the FTEP when you said all switches , they include spines?

and about the Infraestructure VLAN, in your image of leaf switch displays vlan7 with IP, is it right? or what's does means that number?

and what function lo1 do?

In this image extract from book, show vlan8

willytech007_0-1748071153652.png

The explanation of PTEP is  for encapsulation and decap VXLAN dataplane

Infra Vlan is for communication between apic and switches control-mgmt traffic

What do you think about these sentences?

Thanks in advance

 

Hi @willytech007 ,


I have a question about the FTEP when you said all switches , they include spines?

No. The FTEP IP only exists on leaf switches. I've edited my original reply.

and about the Infraestructure VLAN, in your image of leaf switch displays vlan7 with IP, is it right? or what's does means that number?

No. in my example VLAN 7 just happens to be the VLAN that holds the IP address of the subnet associated with the default Bridge Domain of the infra tenant. You can see the IP address if you navigate to Tenants > infra >> Networking > Bridge Domains > Subnets

However, that IP will be implemented on each switch independently of each other - so there is no guarantee that it will be VLAN 7 on any given switch.

VLAN 7 has no relationship with the Infrastructure VLAN

[BTW - the spelling is Infrastructure not Infraestructure] 

and what function lo1 do?

Interface lo1 is the IP address that was assigned to these two switches when a VPC Explicit Protection Group was created for these two switches. You can see the IP address if you navigate to Fabric > Access Policies >> Policies > Switch > Virtual Port Channel default

In this image extract from book, show vlan8

willytech007_0-1748071153652.png

In the book example VLAN 8 is the VLAN associated with with the default Bridge Domain of the infra tenant, proving my point above that there is no relationship between the VLAN number (recall that in my example it was VLAN 7) and the IP address.

You can also see from the output of the show vlan extended command that on the switch LEAF101, the infrastructure VLAN is VLAN 3600

The explanation of PTEP is  for encapsulation and decap VXLAN dataplane

Correct. FTEP addresses are for the encapsulation and decapsulation of VXLAN packets when VXLANs are used to identify EPGs (rather than VLANs used to identify EPGs). This will only occur if there is a vSwitch attached to that leaf that terminates the encapsulated VXLAN that represents the EPG.

Infra Vlan is for communication between apic and switches control-mgmt traffic

Correct. As I said before "The Infrastructure VLAN is the VLAN used between leaf switches and the APIC. The only place you'll see an IP on the Infrastructure VLAN is on the APIC." And indeed, it is used for control-mgmt traffic.  All traffic leaving the APIC destined to some leaf will leave the APIC encapsulated in this Infrastructure VLAN - and as you can see from your book example, gets remapped to the relevant VLAN for the infra:default Bridge Domain.

I hope this helps.


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem.


 

RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

Hi there

In the book said that SVI vlan 8 is Infraestructure vlan on leafs, here the text 

willytech007_0-1748277850923.png

I understand that only ACI create SVI > vlan infraestructure(defined initial config console)

But the book makes me think also leaf switches have SVI referenced Infraestructure VLAN.

Thanks for you help

Regards

 

 

HI @willytech007 ,

I understand that only ACI create SVI > vlan infraestructure(defined initial config console)

Half correct. During the initial config console, the user allocates a VLAN ID to be used as the Infrastructure VLAN.

BUT this VLAN ID has NOTHING to do with the VLAN ID of the SVI VLAN ID assigned to the switches. It will be assigned to the bond0 sub-interface on the APIC, BUT NOT to the SVI VLAN ID on the switches.

I know this is NOT how normal L3 switches work, but normal L3 switches don't work in an environment where hundreds of thousands of VLANs (hence the same ID used in multiple places) have to work in unison across a whole fabric.

But the book makes me think also leaf switches have SVI referenced Infraestructure VLAN.

Correct. But this is the internal mapping of the infrastructure VLAN to the switch's local internal VLAN. In the example in the book, it just happened to be VLAN 8.  In my example it just happened to be VLAN 7. Next time it may be something different. There is nothing in ACI that says it has to be VLAN 7 or 8 or any other number. And it doesn't have to be the same number on every leaf switch.

Keep in mind that this is an INTERNAL VLAN - potentially different on every switch. So when packets arrive from the APIC tagged with the Infrastructure VLAN ID (3961 in my example, 3600 in the book example) they will be remapped to VLAN 7/8 (7 in my example, 8 in the book example) so that they can reach that IP address that is on the SVI of VLAN 7/8

RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

Review Cisco Networking for a $25 gift card

Save 25% on Day-2 Operations Add-On License