01-12-2021 12:53 PM - edited 01-12-2021 08:46 PM
Hi,
We have an issue where it looks like a VPC is not working very well with a port-channel on an ASR.
An ASR1001X connected to two Nexus switches, with a port-channel, is trying to reach a firewall which is connected to those switches.
The gateway of the ASR is one of the Nexus switches, which can reach the firewall again.
But as long as the interface on the ASR to the other Nexus (the one that is not the gateway) is up we cannot reach the firewall from the ASR.
Schematic our setup looks as following:
/------ eth1/3 (po5) ---- Nexus 1 ---- eth1/6 (po4) ----\ t0/0/0 (po1) | | port34 (LAG0) | eth1/1 (po1) eth1/2 (po1) | | | V | | ASR1001X | P | Firewall | | C | | | eth1/1 (po1) eth1/2 (po1) | t0/0/1 (po1) | | port33 (LAG0) \------ eth1/3 (po5) ----- Nexus 2 ---- eth1/6 (po4) ----/
On the ASR we have the following (relevant) configuration:
interface Port-channel1 no ip address ! interface Port-channel1.147 encapsulation dot1Q 147 ip address 10.250.1.171 255.255.255.248 ! interface TenGigabitEthernet0/0/0 description Nexus1-eth1/3 no ip address cdp enable channel-group 1 mode active ! interface TenGigabitEthernet0/0/1 description Nexus2-eth1/3 no ip address cdp enable channel-group 1 mode active !
The gateway for the firewall on the ASR is 10.250.1.169 (which is an VLAN interface on Nexus2, which is connected to t0/0/1). The specific VLAN is configured to be allowed over the VPC peerlink on the Nexus switches.
When we try to ping the firewall we get a timeout, but when we shut port t0/0/0 it suddenly works. Therefor I assume VLAN 147 is not passed through from Nexus1 to Nexus2 over the VPC peerlink?
It seems we are doing something fundamentally wrong but we cannot figure out what it is.
This is the (relevant) configuration on Nexus1:
vpc domain 2 peer-keepalive destination 10.251.101.14 interface port-channel1 switchport mode trunk switchport trunk allowed vlan 102-103,107,109,145,147,1100-1101,1103 spanning-tree port type network vpc peer-link interface port-channel5 description ASR1001 switchport mode trunk switchport trunk allowed vlan 102,109,145,147,1100-1101,1103 vpc 5 interface Ethernet1/1 description VPC switchport mode trunk switchport trunk allowed vlan 102-103,107,109,145,147,1100-1101,1103 channel-group 1 mode active interface Ethernet1/2 description VPC switchport mode trunk switchport trunk allowed vlan 102-103,107,109,145,147,1100-1101,1103 channel-group 1 mode active interface Ethernet1/3 description ASR1001-T0/0/0 switchport mode trunk switchport trunk allowed vlan 102,109,145,147,1100-1101,1103 channel-group 5 mode active
And this is the (relevant) configuration on Nexus2, which is almost identical except for an extra VLAN interface:
vpc domain 2 peer-keepalive destination 10.251.101.13 interface Vlan147 no shutdown mtu 9000 ip address 10.250.1.169/29 interface port-channel1 switchport mode trunk switchport trunk allowed vlan 102-103,107,109,145,147,1100-1101,1103 spanning-tree port type network vpc peer-link interface port-channel5 description ASR1001 switchport mode trunk switchport trunk allowed vlan 102,109,145,147,1100-1101,1103 vpc 5 interface Ethernet1/1 description VPC switchport mode trunk switchport trunk allowed vlan 102-103,107,109,145,147,1100-1101,1103 channel-group 1 mode active interface Ethernet1/2 description VPC switchport mode trunk switchport trunk allowed vlan 102-103,107,109,145,147,1100-1101,1103 channel-group 1 mode active interface Ethernet1/3 description ASR1001-T0/0/1 switchport mode trunk switchport trunk allowed vlan 102,109,145,147,1100-1101,1103 channel-group 5 mode active
On both port-channels VLAN 147 is allowed so it should be reachable to reach Nexus2 over Nexus1 from ASR1001?
The VPC seems working fine as well:
Nexus1# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 2 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : failed Type-2 inconsistency reason : SVI type-2 configuration incompatible vPC role : primary, operational secondary Number of vPCs configured : 4 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po1 up 102-103,107,109,145,147,1100-1101,1103 vPC status ---------------------------------------------------------------------------- Id Port Status Consistency Reason Active vlans -- ------------ ------ ----------- ------ --------------- 5 Po5 up success success 102,109,145,14 7,1100-1101,11 03
Can anyone point out what we are doing wrong here?
Thanks in advance !
/ Richard
Solved! Go to Solution.
01-13-2021 02:36 AM
If you Loking some Layer 3, you can not use peerlink, you need consider having Layer 3 as per vPC design concern for the BGP (is this iBGP or eBGP)
01-12-2021 01:59 PM - edited 01-12-2021 02:11 PM
Hi,
What is the reason for not having a vlan interface (SVI) on Nexus-1?
interface Vlan147 no shutdown mtu 9000 ip address 10.250.1.x/29
I also see this in the output of sh vpc:
Type-2 consistency status : failed Type-2 inconsistency reason : SVI type-2 configuration incompatible
HTH
01-12-2021 08:38 PM
Hi Reza,
Thank you for your answer.
We used to have a SVI, in the same VLAN, on Nexus1 as well but while reading this forum we read somewhere it would be better to have two seperate VLANs.
On Nexus1 we have VLAN145 now and on Nexus2 we have VLAN147. There is a BGP session between the ASR and both switches over those VLANs. As long as Nexus1 is the preferred path everything works fine. If the BGP session with Nexus1 goes down the preferred path will become Nexus2. But if, for whatever reason, the link stays up with Nexus1 we have the problem illustrated in my first post.
When we had both switches in the same VLAN we had the exact same problem.
The warning about the SVI type-2 configuration incompatible is because of we don't have the same SVI's on both switches. Which is as far as I read only a warning and not an error.
Nexus1# sh vpc consistency-parameters global Legend: Type 1 : vPC will be suspended in case of mismatch Name Type Local Value Peer Value ------------- ---- ---------------------- ----------------------- STP MST Simulate PVST 1 Enabled Enabled STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled, BPDUFilter, Edge BPDUGuard Disabled Disabled STP MST Region Name 1 "" "" STP Disabled 1 VLANs 1-3967 VLANs 1-3967 STP Mode 1 Rapid-PVST Rapid-PVST STP Bridge Assurance 1 Enabled Enabled STP Loopguard 1 Disabled Disabled STP MST Region Instance to 1 VLAN Mapping STP MST Region Revision 1 0 0 Interface-vlan admin up 2 102-103,107,145 103,107,109,147 Interface-vlan routing 2 102-103,107,145 103,107,109,147 capability Xconnect Vlans 1 QoS (Cos) 2 ([0-7], [], [], [], ([0-7], [], [], [], [], []) [], []) Network QoS (MTU) 2 (9000, 9000, 9000, (9000, 9000, 9000, 9000, 9000, 9000) 9000, 9000, 9000) Network Qos (Pause: 2 (F, F, F, F, F, F) (F, F, F, F, F, F) T->Enabled, F->Disabled) Input Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0) Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F) Priority: T->Enabled, F->Disabled) Output Queuing (Bandwidth 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0) Remaining) Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F) Priority: T->Enabled, F->Disabled) HW profile Forwarding Mode 1 normal normal Allowed VLANs - 102-103,107,109,145,14 102-103,107,109,145,14 7,1100-1101,1103 7,1100-1101,1103 Local suspended VLANs - - -
The vlan 107 and 109 difference is correct as certain devices are only connected to nexus1 (vlan107) and other devices are only connected to nexus2 (vlan109).
01-12-2021 05:12 PM - edited 01-12-2021 05:12 PM
If you configure only on one nexus - that becomes an orphan port, vPC need to be active/active, why not considering HSRP config here for vlan147
interface Vlan147 no shutdown mtu 9000 ip address 10.250.1.169/29
01-12-2021 08:42 PM
Hello Balaji,
The reason there is no HSRP (/VRRP) is because the interface is used for BGP peering. If we use HSRP the IP-address will be moved to the other switch, in case of a failover, but the BGP session would become invalid and needs to be re-initialized again.
We used to have the same SVI on both switches but it came with the exact same problem.
01-13-2021 02:36 AM
If you Loking some Layer 3, you can not use peerlink, you need consider having Layer 3 as per vPC design concern for the BGP (is this iBGP or eBGP)
01-14-2021 01:16 AM
It is eBGP and we have indeed reconsidered our design .
We have removed the port-channel on the ASR and on the Nexus switches.
Instead we are using individual VLANs to each switch and not forwarding this VLAN over the VPC:
/--- eth1/3 - vlan145 ---- Nexus 1 ---- eth1/6 (po4) ----\ t0/0/0.145 | | port34 (LAG0) | eth1/1 (po1) eth1/2 (po1) | | | V | | ASR1001X | P | Firewall | | C | | | eth1/1 (po1) eth1/2 (po1) | t0/0/1.147 | | port33 (LAG0) \--- eth1/3 - vlan147 ---- Nexus 2 ---- eth1/6 (po4) ----/
BGP will automatically change the route if one of the Nexus goes down.
01-15-2021 12:08 AM
yes, that is also a good approach BGP and BFD combination also if you prefer to do.
01-12-2021 05:55 PM - edited 01-14-2021 09:24 AM
...
01-14-2021 06:05 PM
Hi friend
please what is model of nexus you use it is 3... ?
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: