Issue pinging between OSPF neighbors

[cactus92]

SETUP

Two physical C9300 switches with IP routing enabled.  The two are sitting side-by-side and are directly connected over Gi1/0/3.  Gi1/0/3 is broken up into subinterfaces.  Both switches use Vlan99 for management and an SVI created which have IPs in the same /24 subnet.  Autostate is disabled to keep the SVI in an up/up state since this setup is entirely L3.  The switches are OSPF neighbors.  

PROBLEM

The Vlan99 SVI on one switch is unable to ping the Vlan99 SVI on the other switch.

SW1 CONFIG (relevant bits only)

hostname SW1
!
ip routing
no ip gratuitous-arps
!
interface Loopback1
ip address 10.1.0.1 255.255.255.255
 ip ospf 1 area 0
!
interface GigabitEthernet1/0/3
 no switchport
 no ip address
!
interface GigabitEthernet1/0/3.311
 encapsulation dot1Q 311
 ip address 172.31.1.1 255.255.255.252
 ip ospf 1 area 0
!
interface Vlan99
 ip address 192.168.0.1 255.255.255.0
 ip ospf 1 area 0
 no autostate
!
router ospf 1
 router-id 10.1.0.1
 passive-interface default
 no passive-interface Vlan99
 no passive-interface GigabitEthernet1/0/3.311

SW2 CONFIG (relevant bits only)

hostname SW2
!
ip routing
no ip gratuitous-arps
!
interface Loopback1
 ip address 10.1.0.2 255.255.255.255
 ip ospf 1 area 0
!
interface GigabitEthernet1/0/3
 no switchport
 no ip address
!
interface GigabitEthernet1/0/3.311
 encapsulation dot1Q 311
 ip address 172.31.1.2 255.255.255.252
 ip ospf 1 area 0
!
interface Vlan99
 ip address 192.168.0.2 255.255.255.0
 ip ospf 1 area 0
 no autostate
!
router ospf 1
 router-id 10.1.0.2
 passive-interface default
 no passive-interface Vlan99
 no passive-interface GigabitEthernet1/0/3.311

SW1 Outputs

SW1#show ip route

Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 2 subnets
C        10.1.0.1 is directly connected, Loopback1
O        10.1.0.2 [110/2] via 172.31.1.2, 00:05:47, GigabitEthernet1/0/3.311
      172.31.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.31.1.0/30 is directly connected, GigabitEthernet1/0/3.311
L        172.31.1.1/32 is directly connected, GigabitEthernet1/0/3.311
      192.168.0.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.0.0/24 is directly connected, Vlan99
L        192.168.0.1/32 is directly connected, Vlan99

SW1#show ip ospf neigh

Neighbor ID     Pri   State           Dead Time   Address         Interface
10.1.0.3          1   FULL/DR         00:00:34    172.31.1.2      GigabitEthernet1/0/3.311

SW1#ping 192.168.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

SW1#show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  172.31.1.1              -   cc36.cf5e.81d8  ARPA   GigabitEthernet1/0/3.311
Internet  172.31.1.2             52   cc36.cf5e.7ed8  ARPA   GigabitEthernet1/0/3.311
Internet  192.168.0.1             -   cc36.cf5e.81dd  ARPA   Vlan99
Internet  192.168.0.2             0   Incomplete      ARPA

SW2 Outputs

SW2#show ip route

Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 2 subnets
O        10.1.0.1 [110/2] via 172.31.1.1, 00:46:10, GigabitEthernet1/0/3.311
C        10.1.0.2 is directly connected, Loopback1
      172.31.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.31.1.0/30 is directly connected, GigabitEthernet1/0/3.311
L        172.31.1.2/32 is directly connected, GigabitEthernet1/0/3.311
      192.168.0.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.0.0/24 is directly connected, Vlan99
L        192.168.0.2/32 is directly connected, Vlan99


SW2#show ip ospf neigh

Neighbor ID     Pri   State           Dead Time   Address         Interface
10.1.0.1          1   FULL/BDR        00:00:36    172.31.1.1      GigabitEthernet1/0/3.311

SW2#ping 192.168.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

SW2#show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  172.31.1.1             50   cc36.cf5e.81d8  ARPA   GigabitEthernet1/0/3.311
Internet  172.31.1.2              -   cc36.cf5e.7ed8  ARPA   GigabitEthernet1/0/3.311
Internet  192.168.0.1             0   Incomplete      ARPA
Internet  192.168.0.2             -   cc36.cf5e.7edd  ARPA   Vlan99

ADDITIONAL NOTES

When I create a static route to facilitate this traffic, it works fine.  For example:

SW1(config)#ip route 192.168.0.2 255.255.255.255 10.1.0.2
SW1(config)#do ping 192.168.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

I'm having trouble wrapping my head around why I can't directly ping between the Vlan99 SVIs on both these switches when they're OSPF neighbors and the SVIs are in the same subnet.  

Any help would be appreciated! 

[Royalty · Accepted solution]

Hi Cactus92,

So, you've configured a routed point-to-point link between the switches using subinterfaces, which is a valid Layer 3 design. However, may I say you're trying to use the same subnet on an SVI (interface vlan99) on two switches that do not share L2 connectivity. You now have overlapping subnets which is caused by having the 192.168.0.0/24 network split across a layer 3 boundary; the layer 3 routed link between the two switches has split the network, and now, the 192.168.0.0/24 subnet is 'living' in two different networks. If you want VLAN 99 to be configured on each switch with an IP address in the same subnet then it will require a switched access or trunk port between them with the respective needed VLANs. Just reiterating, the issue is because VLAN 99 is configured on both switches with IP addresses in the same subnet (192.168.0.0/24). They are in the same subnet but not part of the same L2 network, so the design would be invalid as it breaks subnetting and routing/switching principles

This network on VLAN 99 isn't directly connected since there is a layer 3 link that divides them. For example, SW1 would need routes to get to SW2. However, If you look at the output of `show ip route` on either switch, you will see that the route for 192.168.0.0/24 is directly connected. This is a hint. You should actually be receiving a route on a given switch to 192.168.0.0/24 via OSPF, which would be advertised from the adjacent switch. But, a directly connected route has a higher Administrative Distance than an OSPF route, so the OSPF route sits in the OSPF database and is not injected into the RIB. For example, you could check on SW1 by doing a 'show ip ospf database router <adjacent switch's RID>' and you would see a received Router LSA for the 192.168.0.0/24. This route which would point to the next-hop of the adjacent switch would be needed. I am going a bit off-topic, but just throwing some of the symptoms out there. The above is not necessarily the problem, it's just one of the symptoms caused by subnet overlapping. Other elements of connectivity are failing, like ARP is unable to resolve the MAC address of the adjacent host in the 192.168.0.0/24 network because ARPs do not traverse routed links. Even statically setting the MAC address will not fix it, because the routing table (and FIB) believe that the router 'owns' the network and that it doesn't exist in another part of the network.

You mentioned that pings were successful after adding a static to route to the host address of SW2's SVI for VLAN 99 - a /32 route is more specific than a /24. Longest match routing is considered first and before administrative distance in terms of path selection for IP routing. Check the ip routing table with the static route vs without and see the difference that the switch now knows where to send the traffic in order to deliver the ICMP echo requests to the correct next-hop.

If you want to keep MGMT addressing on each device and within the 192.168.0.0/24 subnet, you could use a loopback interface and assign the same IP addresses with a /32 mask and advertise into OSPF. Fun fact, loopbacks are advertised into OSPF as a /32 regardless of the subnet mask configured as they are treated as network type loopback and a stub host. Alternatively, add another link between the two switches in VLAN 99 (or a trunk link and prune the unnecessary VLANs). You could also change the subnets to use different /24s for the VLAN 99 SVI on each switch. You could also just make GigabitEthernet1/0/3 a trunk link and extend VLANs across them, but that has its drawbacks and advantages like all of these potential solutions mentioned. Those are just a few thoughts.

Please feel free to ask if anything is confusing with that

[cactus92]

Thank you so much for the detailed response!  This definitely answers my question.

[paul driver · Accepted solution]

Hello
The reason is you dont have any L2 connectivity for the Vlan 99 on the switches, either add a separate connection as a trunk or default interface gig1/0/3 and make that a trunk then recreate 172.31.1.0/30 as a L3 svi 311

[cactus92]

Thanks for the response!  Just for testing, I added a L2 trunk link between the switches and everything works as expected.  

[Martin L]

check SDM settings for routing - if switch supports SDM, it usually it is set for switching, not routing;

not sure if u need sub-interfaces, if so why?  Try removing IP from sub-interface - or shut it down

Regards, ML
**Have fun labbing!!!***
***Please Rate All Helpful Responses ***

[Joseph W. Doherty]

not sure if u need sub-interfaces, if so why? 

For what appears to be a p2p, definitely not necessary.

Also, you might configure OSPF as a p2p too.

check SDM settings for routing - if switch supports SDM, it usually it is set for switching, not routing;

As SDM settings apportion resource ratios, insufficient information posted to say which template ought to be used.

[Joseph W. Doherty]

As the others have described, especially @Royalty , you're treating V99 as it's the same VLAN across both your L3 switches, but it's not.  It has same name but it's two different VLANs.

So, as suggested, either you need to make it one VLAN, or assign each its own subnet.

Personally, for L3 switches being used as routers, I would recommend using loopbacks as also described by @Royalty .  (BTW, you can do the same on routers.)

[cactus92]

Thank you for the reply!  I'll experiment with replacing the SVI with a Loopback in the same subnet and see how that works.  

[Joseph W. Doherty]

. . . with a Loopback in the same subnet and see how that works.

BTW, as @Royalty already mentioned, each loopback will be in its own subnet, a host or /32 network.  In the past, have used a /24 address block, but it can be otherwise.  Remember, if working out of an address block, 1st and last IPs are available too