05-28-2024 08:40 AM
We are encountering an issue with setting up OSPF routing on two Nexus9000 switches we recently put into service.
We have several branch locations in our network. Each has its own branch router or L3 device. These devices use OSPF to form neighbor relationships.
When we first setup the two Nexus9000 devices, we attempted to set up the OSPF routing on a VLAN interface. This interface was set up on both devices with HSRP applied.
The config was very much like this:
Nexus A
interface Vlan990
description CharterMPLS
no shutdown
no ip redirects
ip address 172.16.99.237/24
no ipv6 redirects
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 3 keygoeshere
no ip ospf passive-interface
ip router ospf 10 area 0.0.0.0
hsrp 90
priority 150
timers 1 3
ip 172.16.99.236
Nexus B
interface Vlan990
description CharterMPLS
no shutdown
no ip redirects
ip address 172.16.99.238/24
no ipv6 redirects
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 3 keygoeshere
no ip ospf passive-interface
ip router ospf 10 area 0.0.0.0
hsrp 90
priority 150
timers 1 3
ip 172.16.99.236
Ethernet interface for connection:
interface Ethernet1/6
switchport
switchport mode trunk
switchport trunk allowed vlan 990
Under this configuration the Nexus devices would not form neighbor relationships with the other devices in the network.
However, if we configured an ethernet interface directly with the OSPF info, neighboring came right up and works perfectly.
Config on the interface:
interface Ethernet1/45
description CHARTER_MPLS
no ip redirects
ip address 172.16.99.236/24
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 3 keygoeshere
no ip ospf passive-interface
ip router ospf 10 area 0.0.0.0
no shutdown
Do we have errors in our config somewhere such that setting OSPF on the VLAN interface causes the neighbor relationships to not be built?
Thanks!
05-28-2024 09:03 AM - edited 05-28-2024 09:05 AM
Hello @mbrown-revitycu ,
>> Under this configuration the Nexus devices would not form neighbor relationships with the other devices in the network.
>> However, if we configured an ethernet interface directly with the OSPF info, neighboring came right up and works perfectly.
Check the MTU when you use the SVI and when you use the physical interface it is the first thing I would look at.
show ip interface vlan 990 compared to show ip interface eth1/45
Hope to help
Giuseppe
05-28-2024 10:01 AM
You can use VLAN'
But Nexus not like other SW that auto add vlan to db
So what you need is only add vlan by
Vlan 990
And the ospf will be work
MHM
06-04-2024 09:28 AM
I'm sorry, I don't understand this reply.
"You can use VLAN'
But Nexus not like other SW that auto add vlan to db
So what you need is only add vlan by
Vlan 990
And the ospf will be work"
We do have this also in the config:
vlan 1,5,10,30,35,66,150,160,252-254,990,999-1000
[REDACTED]
vlan 990
name Charter_MPLS
vlan 999
name SilverpeakLAN
spanning-tree vlan 5,10,30,35,66,150,160,990,999 priority 4096
Is any of that what you mean? Or is there a separate place to define the VLAN?
Thanks!
06-04-2024 09:41 AM
show ip interface brief <<- share this check if VLAN 990 is UP/UP or not
MHM
06-04-2024 10:34 AM
It's not currently listed in a show ip int brief, because we don't actually have an IP address on the VLAN interface right now, only this is configured:
interface Vlan990
description Charter MPLS
no ip redirects
no ipv6 redirects
However, when we have attempted to use the VLAN for OSPF, the configuration we tried is something like this:
interface Vlan990
description Charter MPLS
no ip redirects
ip address 172.16.99.235/24
no ipv6 redirects
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 3 authkeygoeshere
no ip ospf passive-interface
no shutdown
06-04-2024 10:37 AM
With sure
Ip router ospf <> area <>
?
If Yes I will run lab and use VLAN between two NSK and share result here.
One more not' are both NSK is vPC?
Are vlan is vpc vlan or non-vpc vlan?
MHM
06-04-2024 10:47 AM
The OSPF config is:
router ospf 10
router-id 172.16.99.236
log-adjacency-changes
area 0.0.0.0 authentication
passive-interface default
We have some vPC VLANs, we have some non-vPC VLANs.
This one - when we tried to use it - was set up as non-vPC VLAN.
06-04-2024 10:54 AM
And this vlan use vpc peer-link not separate trunk between two SW?
MHM
06-04-2024 11:59 AM
We have one peer link set up between the two Nexus switches.
interface port-channel1000
description ***VPC PEER LINKS***
switchport
switchport mode trunk
spanning-tree port type network
vpc peer-link
interface Ethernet1/53
description ***VPC PEER LINKS***
switchport
switchport mode trunk
channel-group 1000 mode active
no shutdown
interface Ethernet1/54
description ***VPC PEER LINKS***
switchport
switchport mode trunk
channel-group 1000 mode active
no shutdown
06-04-2024 12:09 PM
So this trunk peer-link for both vpc vlan and non-vpc vlan
But all vlan allow so all vlan is vpc vlan
Anyway I will run lab with same topology you have abd check results
MHM
06-04-2024 12:12 PM
Thank you for all the suggestions and help.
I want to clarify something though, that I don't think I did in my earlier posts.
This pair we are trying to fix are the only Nexus switches we have.
All other branches we are connecting by OSPF are using Cisco ISR 4321 or 4321.
06-04-2024 11:32 PM
this is my lab
trunk allow all vlan between two NSK, this trunk is peer-link
the OSPF is OK between two SW after I sure that the vlan is UP and appear in show vpc brief and STP not BLK this VLAN
NOW, additional point about the ISR I dont get it? can you more elaborate
MHM
06-04-2024 08:26 PM
Hi Mbrown,
Can you clarify if you are also making changes to the router side configurations when you make the following changes to the Nexus?
"Under this configuration the Nexus devices would not form neighbor relationships with the other devices in the network.
However, if we configured an ethernet interface directly with the OSPF info, neighboring came right up and works perfectly."
The frame leaving a routed layer 3 port versus a layer 2 trunk port is very different. In the SVI configurations you provided, you are using VLAN 990. Naturally, you have also configured the layer 2 port as a trunk carrying VLAN 990. In this scenario the frames leaving the port to form the OSPF neighborship will be tagged with 802.1q for VLAN 990. When you are in this configuration is the router port configured to accept these VLAN tagged frames with something like a dot1.q encapsulation for vlan 990?
Conversely, when you have the Nexus port configured as a layer 3 port, the frames will not be sent with a dot1.q vlan tag. So the router will consume it differently.
Just wanted some clarification on this one point before we continue taking a look at other potential causes. If the router configuration is not changing between the above two scenarios, I would suggest moving the layer 2 port to an access port for vlan 990 instead of a trunk, or adding dot1q encapsulation to the router config.
Also, since it has not been discussed yet, please be sure you have your VPC domain with 'layer3 peer-router', as the OSPF neighborships may not be stable unless this is configured. This would not prevent the neighborships in your current scenario from forming, but could cause issues down the road or with with OSPF neighborships to just 1 of the peers. This configuration needs to be on both VPC peers.
Thanks!
Scott Hoppmann
Cisco HTTS - Data Center Route and Switch
RTP, NC
06-05-2024 08:31 AM
I'll do my best to respond to everyone coherently in one post here.
Our network consists of eight branch locations. Each of these locations has a L3 switch or a simpler dedicated ISR that does the OSPF routing.
Branch A = Nexus9000 pair
Branch B = Meraki MS350-48FP stack
Branch C = Meraki MS350-48FP stack
Branch D, E, F, G, H = Cisco ISR 4331
Branch J = Cisco ISR 4321
Each of these devices is performing the OSPF routing for its branch. There is not an additional router at Branch A - the Nexus is the router.
Each is in the OSPF network currently, neighboring up with all its peers as we want - this minus the SD-WAN device being included at Branch A. When we attempt to add the SD-WAN device to the Branch A network, OSPF breaks down at Branch A.
As it is, we have not been making any changes to any of the routers when attempting to add the SD-WAN device.
We set up a subnet specific to this traffic, that being 172.16.99.0/24. Each branch device has an address in that subnet that is being used to pass the OSPF traffic.
At the branches with the ISR 4321/4331, the config on the interface performing the OSPF routing does not use a dot1q subinterface. The interface is assigned an IP address in the subnet.
Example config shown below:
interface GigabitEthernet0/0/1
ip address 172.16.99.250 255.255.255.0
ip access-group CUI-IN in
ip access-group CUI-OUT out
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 7 011F570A501F5642331F5D1D
negotiation auto
At the branches with the Meraki core switches, an interface is set up on the stack with an address in the subnet, and using VLAN 990.
Interface Editor
Switch or switch stack
MAIN-CORE-S
Name
Charter MPLS
VLAN
990
Subnet
172.16.99.0/24
Interface IP
172.16.99.253
Multicast routing
Disabled
OSPF settings
Area
0: Backbone
Cost
1
Passive?
No
On the Nexus9000 stack, "layer3 peer-router" is configured as shown here.
Switch A
vpc domain 1
peer-switch
role priority 100
peer-keepalive destination 10.255.255.2 source 10.255.255.1 vrf keepalive
peer-gateway
layer3 peer-router
auto-recovery
Switch B
vpc domain 1
peer-switch
peer-keepalive destination 10.255.255.1 source 10.255.255.2 vrf keepalive
peer-gateway
layer3 peer-router
Routing thru OSPF is working on the Nexus as is, which is a direct connection from the Spectrum WAN router to the Eth 1/45 interface of one of the Nexus switches.
interface Ethernet1/45
description CHARTER_MPLS
no ip redirects
ip address 172.16.99.236/24
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 3 5fbc4a00c637609c98e546fb504bde1e
no ip ospf passive-interface
ip router ospf 10 area 0.0.0.0
no shutdown
At the risk of overwhelming with info that may not even be necessary, I'll end this post here.
Please let me know if I can provide anything else though. I do sincerely appreciate everyone taking the time on this.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide