ACI L3 out OSPF

[nexus13213]

hi,

need help to configure L3 out OSPF.

I got an error when configuring external L3.

what is the meaning of oper.state missing ?

thanks

[Joseph Young]

Are you seeing any faults for the l3out?

[nexus13213]

hi yoseph,

yes, there is an error,

Where can i change the configuration of port as l2 or l3 ?

any suggest ?

thanks

[Joseph Young]

That fault means that the same port is being deployed somewhere else as an L2 switchport so you are unable to deploy it also as a routed port in the l3out.

From the apic you should be able to do 'show running-config leaf 101 interface ethernet 1/16' to see what encap vlans are actually deployed. Should allow you to hunt down what epg the port is actually a part of.

[nexus13213]

hi yoseph,

i have tried to move to another port, but still got the same error.

apic1(config)# fabric 101 show interface eth 1/16 
----------------------------------------------------------------
 Node 101 (leaf01)
----------------------------------------------------------------
Ethernet1/16 is up
admin state is up, Dedicated Interface
 Hardware: 1000/10000 Ethernet, address: 00f6.634f.2cb4 (bia 00f6.634f.2cb4)
 MTU 9000 bytes, BW 1000000 Kbit, DLY 1 usec
 reliability 255/255, txload 1/255, rxload 1/255
 Encapsulation ARPA, medium is broadcast
 Port mode is trunk
 full-duplex, 1000 Mb/s, media type is 1G
 FEC (forward-error-correction) : disable-fec
 Beacon is turned off
 Auto-Negotiation is turned on
 Input flow-control is off, output flow-control is off
 Auto-mdix is turned off
 Rate mode is dedicated
 Switchport monitor is off
 EtherType is 0x8100
 EEE (efficient-ethernet) : n/a
 Last link flapped 21:13:06
 Last clearing of "show interface" counters never
 8 interface resets
 30 seconds input rate 0 bits/sec, 0 packets/sec
 30 seconds output rate 1336 bits/sec, 2 packets/sec
 Load-Interval #2: 5 minute (300 seconds)
 input rate 48 bps, 0 pps; output rate 1232 bps, 2 pps
 RX
 0 unicast packets 1279 multicast packets 1 broadcast packets
 1280 input packets 562824 bytes
 0 jumbo packets 0 storm suppression bytes
 0 runts 0 giants 0 CRC 0 no buffer
 0 input error 0 short frame 0 overrun 0 underrun 0 ignored
 0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
 0 input with dribble 0 input discard
 0 Rx pause
 TX
 0 unicast packets 84870 multicast packets 89173 broadcast packets
 174043 output packets 13410671 bytes
 0 jumbo packets
 0 output error 0 collision 0 deferred 0 late collision
 0 lost carrier 0 no carrier 0 babble 0 output discard
 0 Tx pause

port 1/16 i have never used before, and why Port mode is trunk ? while I configured it as routed port ?

thanks

[Markus Sandmeier]

Hi,

Infrastructure vlan enabled under your AEP?

br,

markus

[nexus13213]

hi,

its disable, should I enable it ?

thanks,

[Markus Sandmeier]

no disabled it's fine.

[Jason Williams]

Could you log into the leaf node and check the 'show interface eth x/y switchport' output? You will be able to see the PI (platform independent) VLANs allowed on that interface. 

leaf# show interface eth1/27 switchport
Name: Ethernet1/27
Switchport: Enabled
Switchport Monitor: not-a-span-dest
Operational Mode: trunk
Access Mode Vlan: unknown (default)
Trunking Native Mode VLAN: unknown (default)
Trunking VLANs Allowed: 4-6
FabricPath Topology List Allowed: 0
Administrative private-vlan primary host-association: none
Administrative private-vlan secondary host-association: none
Administrative private-vlan primary mapping: none
Administrative private-vlan secondary mapping: none
Administrative private-vlan trunk native VLAN: none
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk private VLANs: none
Operational private-vlan: none

Convert the PI VLANs to encap VLANs using 'show vlan id x extended'

leaf# show vlan id 4-6 extended

VLAN Name                             Status    Ports
---- -------------------------------- --------- ---------------
4    jw1:jw2                          active    Eth1/27
5    jw1:jw-ap:jw2                    active    Eth1/27
6    jw1:jw1:l3out-jw-                active    Eth1/27
     static:vlan-1500

VLAN Type  Vlan-mode  Encap
---- ----- ---------- -------------------------------
4    enet  CE         vxlan-16187318
5    enet  CE         vlan-2000
6    enet  CE         vxlan-14942177, vlan-1500

[nexus13213]

Hi Jasonw2,

here the output

apic1# fabric 101 show interface ethernet 1/16 switchport 
----------------------------------------------------------------
 Node 101 (leaf01)
----------------------------------------------------------------
Name: Ethernet1/16
 Switchport: Enabled
 Switchport Monitor: not-a-span-dest
 Operational Mode: trunk
 Access Mode Vlan: 2 (default)
 Trunking Native Mode VLAN: unknown (default)
 Trunking VLANs Allowed: 2
 FabricPath Topology List Allowed: 0
 Administrative private-vlan primary host-association: none
 Administrative private-vlan secondary host-association: none
 Administrative private-vlan primary mapping: none
 Administrative private-vlan secondary mapping: none
 Administrative private-vlan trunk native VLAN: none
 Administrative private-vlan trunk encapsulation: dot1q
 Administrative private-vlan trunk normal VLANs: none
 Administrative private-vlan trunk private VLANs: none
 Operational private-vlan: none
apic1# 

apic1# fabric 101 show vlan id 2 extended
----------------------------------------------------------------
Node 101 (leaf01)
----------------------------------------------------------------

VLAN Name                            Status     Ports
---- -------------------------------- --------- -------------------------------
2          infra:default                 active     Eth1/16, Eth1/48

VLAN  Type         Vlan-mode         Encap
---- ----- ---------- -------------------------------
2           enet         CE                       vxlan-16777209, vlan-3967
apic1#

any suggest ?

thanks,

[aleccham]

Hello nexus13213,

If we have already ruled out a configuration of a static path in infra tenant access AP default epg, then I'm curious if this port was ever used for fabric device communication perhaps APICs? Has any fabric device been plugged into this port?

One thing I might try is to cleanly reload the leaf. Depending on the switch model we can use setup-clean-confih.sh for the Gen 1, and acidiag touch clean on our gen 2 (EX HW). A reload is needed after either command.

[nexus13213]

hi aleccham,

i have never used that port before. may be tomorrow i will try to factory default the leaf.

thanks

[Jason Williams]

Thanks, 

The infrastructure VLAN is programmed on the interface, but we need to understand why. Below are possible causes of infra VLAN being allowed: 

1. Interface is connected to an APIC. In this scenario, the APIC and leaf will talk over the interface. APIC will tell the switch to provision the infra VLAN on that interface. This is not user configurable. If I understand correctly, Eth1/16 is not connected to an APIC so this should not be the issue. 

2. User enables infra VLAN on the interface via access policies configuration. This should only be used for AVS configuration.

Questions for you:

1. What is Eth1/16 connected to?

2. I understand that you mentioned Infra is not enabled on the AEP, but could you please run through your access policy configuration again and verify?

It is very important to check everything below and how they are mapped to Leaf 101 Eth1/16.

- Switch Profile

- Interface Profile

- Interface Policy Group

- Attached Entity Profile

After you've verified how the policies are mapped, it would help if you could upload some screen captures of the AEP and the usage of the AEP. See my examples attached to this post. 

 

When posting the screen cap, could you point out which policy group is in use for the interface?

[nexus13213]

Hi Jasonw2,

- What is Eth1/16 connected to? its connected to Catalyst 3750

- Switch Profile

- Interface Profile

- Interface Policy Group

- Attached Entity Profile

currently only one policy group is in use for the interface.

thanks

[Jason Williams]

Could you try going to the switch profile and removing the interface profile from switch profile? 

After removing the interface from the switch profile, check the show interface eth x/y switchport command again to see if the VLAN was removed. 

If the VLAN is still allowed on the interface then still keep the interface profile deleted and try another step. Go to Fabric -> Inventory -> Leaf (101) -> Interfaces -> Physical Interfaces. Disable Eth1/16 for 2 minutes, then enable the interface. Check the output of show interface eth x/y switchport to see if the VLAN was removed. 

If either of the 2 worked, then try adding the interface profile to the switch profile again.