Per Port VLAN

dpita · ‎09-26-2016

Introduction

Customer needs to use the same VLAN for two different EPGs? Encap already in use fault? PPV is for you!

Prerequisites

Requirements

Seperate VLAN pool for VLANs to be duplicated (namespace)
EPGs need to have unique BD (can be same VRF or different)
Interface Policy Group needs to have L2 Interface policy for Port Local scope

Configure

Configure Unique(different) VLAN pools with the same encap inside

Enable port local scope for the Interface Policy Group for the existing interface already using the VLAN

Under the Tenant, create a new BD and EPG for the VLAN.

Verify

The output below from ELTMC shows two sets of BD/EPG. the FD_VLAN for 373 is shown twice but its important to note the Fabric_encap is unique. This fabric_encap is generated based on the VLAN pool/namespace. Thats why its required to have a unique VLAN pool, so that the fabric_encap VXLAN/vnid is unique.

module-1# show system internal eltmc info vlan brief
VLAN-Info
VlanId  HW_VlanId Type            Access_enc Access_enc Fabric_enc Fabric_enc BDVlan  
                                  Type                 Type                      
==================================================================================
      1        1    BD_CTRL_VLAN    802.1q      4094     VXLAN  16777209       0
      4       13     BD_EXT_VLAN    802.1q        99     VXLAN  15499165       4
      5        2         BD_VLAN   Unknown         0     VXLAN  15761386       5
      8        3         BD_VLAN   Unknown         0     VXLAN  15531930       8
      9       16         FD_VLAN    802.1q      2265     VXLAN      9402       8
     10        4         BD_VLAN   Unknown         0     VXLAN  15105997      10
     11       17         FD_VLAN    802.1q      2261     VXLAN      9398      10
     12        5         BD_VLAN   Unknown         0     VXLAN  16351141      12
     13       18         FD_VLAN    802.1q      2259     VXLAN      9396      12
     14       14     BD_EXT_VLAN    802.1q      2198     VXLAN  15695749      14
     15       19         FD_VLAN    802.1q      2262     VXLAN      9399       8
     16        6         BD_VLAN   Unknown         0     VXLAN  16351138      16
     17       20         FD_VLAN    802.1q      2255     VXLAN      9392      16
     18        7         BD_VLAN   Unknown         0     VXLAN  15925209      18
     19       21         FD_VLAN    802.1q      2260     VXLAN      9397      18
     20        8         BD_VLAN   Unknown         0     VXLAN  16056263      20
     21       22         FD_VLAN    802.1q      2263     VXLAN      9400      20
     22       15     BD_EXT_VLAN    802.1q      2104     VXLAN  14811122      22
     25        9         BD_VLAN   Unknown         0     VXLAN  16056264      25
     26       10         FD_VLAN    802.1q       375     VXLAN      9811      25
     27       23         BD_VLAN   Unknown         0     VXLAN  16416668      27
     28       24         FD_VLAN    802.1q       373     VXLAN      9809      27
     29       11         BD_VLAN   Unknown         0     VXLAN  16121791      29
     30       25         FD_VLAN    802.1q       374     VXLAN      9810      29
     31       12         BD_VLAN   Unknown         0     VXLAN  16187318      31
     32       26         FD_VLAN    802.1q       390     VXLAN      9826      31
     35       31         FD_VLAN    802.1q      1100     VXLAN      8392       5
     42       32         BD_VLAN   Unknown         0     VXLAN  14942179      42
     43       33         FD_VLAN    802.1q      2195     VXLAN      8592      42
     45       34         BD_VLAN   Unknown         0     VXLAN  16416669      45
     46       35         FD_VLAN    802.1q       373     VXLAN     10592      45
module-1#

Note: BD1/EPG1 has encap vlan-373 and uniquely identified in the fabric as BD-16416668/EPG-9809. BD2/EPG2 has encap vlan-373 again BUT the fabric_encap for the BD/EPG is different than before BD-16416668/EPG-10592

Also interesting to note is the HW column. This shows the front panel ASIC VLAN and how it is translated uniquely.

module-1# show system internal eltmc info interface e1/25
            IfInfo: 
           interface:   Ethernet1/25   :::         ifindex:      436305920
                 iod:             54   :::           state:             up
            External:          FALSE

      NorthStar Info:
                 Mod:              0   :::            Port:             25
          port_layer:             L2   :::     fabric_port:              0
           port_mode:          trunk   :::  native_vlan_id:              0
         switchingSt:        enabled   :::           speed:          10000

     Storm Ctrl Info:
                Type:        Percent
            Stm_rate:     100.000000   :::       Stm_burst:     100.000000
      Stm_rate(Mbps):   10000.000000   ::: Stm_burst(Mbps):   10000.000000
      Stm_rate(toks):           6250   ::: Stm_burst(toks):          65535
       Stm_Pol_Apply:              0

xlate_l2_classid_unset:              0
            vlan_bmp:          25-32
      vlan_bmp_count:              8
        acc_vlan_bmp:    373-375,390
  acc_vlan_bmp_count:              4
     scope(0:G, 1:L):              1   :::       class_id::              4
   mac_limit_reached:              0   :::       mac_limit:              0
port_sec_feature_set:              0   ::: mac_limit_action:              0

      NorthStar Info:
          pc_mbr_idx:             11   ::: dest_learn_port:             12
      dest_encap_idx:             56

            BCM Info:

[SDB INFO]:
                 iod:             54
         pc_if_index:              0
        fab_if_index:              0
               sv_if:              0
                 svp:              0
          bcm_l3_eif:              0
       internal_vlan:              0
          encap_vlan:              0
                 mod:              0
                port:             25
         non_byp_mod:              0
        non_byp_port:             25
         ns_lrn_port:             12
           v6_tbl_id:              0
           v4_tbl_id:              0
          router_mac:00.00.00.00.00.00
          unnumbered:              0
        bcm_trunk_id:              0
        tunnel_mp st:     1096941571
           tep_ip st:     1096941571
          ip_if_mode:              0
          bcm_vrf_id:              0
         Overlay idx:              0
            External:          FALSE

FP Entries
    ifp_port_mask_m0:            666
::::
module-1#

With the output above we queried ELTMC again but this time for information on how the interface is programmed. Highlighted we see the scope field is set to local. This allows the front panel ASIC to have extra translations as well as have ACI classify traffic with (vlan, port)

The moquery below for the concrete vlan "vlanCktEp" and filtered by "encap==vlan-373" shows two objects on that particular leaf. Highlighted are the duplicated encap vlan and unique DN and EPG DN as well

fab1-p1-leaf1# moquery -c vlanCktEp -f 'vlan.CktEp.encap=="vlan-373"'
Total Objects shown: 2

# vlan.CktEp
encap                : vlan-373
adminSt              : active
allowUsegUnsupported : 0
childAction          : 
classPrefOperSt      : encap
createTs             : 2016-09-06T08:45:52.000-04:00
ctrl                 : policy-enforced
dn                   : sys/ctx-[vxlan-2326529]/bd-[vxlan-16416668]/vlan-[vlan-373]
enfPref              : hw
epUpSeqNum           : 0
epgDn                : uni/tn-dpita-tenant/ap-dpita-AP/epg-dpita-EPG1
excessiveTcnFlushCnt : 0
fabEncap             : vxlan-9809
fwdCtrl              : mdst-flood
hwId                 : 24
id                   : 28
lcOwn                : local
modTs                : 2016-09-06T08:45:54.308-04:00
mode                 : CE
monPolDn             : uni/tn-common/monepg-default
name                 : dpita-tenant:dpita-AP:dpita-EPG1
operSt               : up
operStQual           : unspecified
operState            : 0
pcTag                : 16391
proxyArpUnsupported  : 0
qosPrio              : unspecified
qosmCfgFailedBmp     : 
qosmCfgFailedTs      : 00:00:00:00.000
qosmCfgState         : 0
rn                   : vlan-[vlan-373]
status               : 
type                 : ckt-vlan
vlanmgrCfgFailedBmp  : 
vlanmgrCfgFailedTs   : 00:00:00:00.000
vlanmgrCfgState      : 0

# vlan.CktEp
encap                : vlan-373
adminSt              : active
allowUsegUnsupported : 0
childAction          : 
classPrefOperSt      : encap
createTs             : 2016-09-06T08:46:18.000-04:00
ctrl                 : policy-enforced
dn                   : sys/ctx-[vxlan-2326529]/bd-[vxlan-16416669]/vlan-[vlan-373]
enfPref              : hw
epUpSeqNum           : 0
epgDn                : uni/tn-dpita-tenant/ap-dpita-AP/epg-test-ppv
excessiveTcnFlushCnt : 0
fabEncap             : vxlan-10592
fwdCtrl              : mdst-flood
hwId                 : 35
id                   : 46
lcOwn                : local
modTs                : 2016-09-06T08:46:19.964-04:00
mode                 : CE
monPolDn             : uni/tn-common/monepg-default
name                 : dpita-tenant:dpita-AP:test-ppv
operSt               : up
operStQual           : unspecified
operState            : 0
pcTag                : 49155
proxyArpUnsupported  : 0
qosPrio              : unspecified
qosmCfgFailedBmp     : 
qosmCfgFailedTs      : 00:00:00:00.000
qosmCfgState         : 0
rn                   : vlan-[vlan-373]
status               : 
type                 : ckt-vlan
vlanmgrCfgFailedBmp  : 
vlanmgrCfgFailedTs   : 00:00:00:00.000
vlanmgrCfgState      : 0

fab1-p1-leaf1#

Troubleshoot

RedNectar · ‎11-30-2016

Hi Dpita,

How did you manage to issue the

show system internal eltmc info vlan brief

command? When I try, the system only lets me type

show system internal eltm

I'm running version 2.1.1(h) on the APIC, 12.1.1(h) on the 9Ks

Regards

RedNectar

Jason Williams · ‎11-30-2016

This is a line card command. Use vsh_lc to enter this mode.

leaf1#

leaf1# vsh_lc
vsh_lc
module-1# show sys int eltmc info vlan br

RedNectar · ‎11-30-2016

Of course - I should have looked at the module-1# prompt!

Thanks

guillerm · ‎03-02-2017

Hello Daniel,

we have got the message "Encap Already Used in Another EPG" you mentionned at the beginning of this port; without apparent reason

I understand this might be solved by the PPV option you described above;

what I don't understand is why this message appears ;

we had a 1st UCS (UCS1) connected via VPC on 2 LEAFs 201 & 202, and using 1 static path encap vlan 11 on EPG11 : no problem

when adding a 2nd UCS (UCS2) also connected via another VPC on the 2 same LEAFs 201 & 202, and adding 1 static path encap for the same VLAN 11 on the same EPG1, we got this "Encap Already Used in Another EPG" message

any idea why such a message appears ?

thanks

hriskulk · ‎03-14-2017

HI,

Both VPC cannot have same encap unless the VPC port scope is local.Atleast 1 VPC has to be local in this case.Also,The two epgs should be in different BD's.Please configure as above and assign same vlan,you will not see the problem.

-Hrishi

aertural1 · ‎11-13-2017

Hi all,

Is there any disadvantages or extra limitations of PPV configuration or can we use it for all ports without concern?

Thanks. Regards.

SandevChopra07800 · ‎08-04-2020

You cannot have same vlan encap on 2 different EPGs if you are adding the same ports in each. Even with PPV turned on, this does not work. The static ports that you assign to both the EPGs have to be different. Verified this in the lab.