Nexus 5000 Series Core routing issue

[asochowski]

So where do I begin? This is probably an easy fix as i feel i must have missed something but anyeays let me start. I have created a new VLAN (Vlan177) on our core switches (5548tp) for a new phone system implementation. I work in a hospital and trying to make easy sense of our network with our outside offices are all dedicated dark fiber with L3 3850's routing each of those sites. Initially all equipment was going to be in the main hospital so they built everything out under vlan177 (172.17.0.0/16) ahead of time. Turns out one of the controllers was installed in one of the outside offices. they said it would take them too long in order to configure everything again for the one controller under a new vlan so i said well i can do a point to point /31 network (however i dont like to because it is messy but it is still doable.). So i set everything up I have my routing statements in there defining to go to the outside router for the .30 and .31 addresses. Below are all the tests i have done and their results as well as the configs of the core router. the issue is from inside that vlan i cannot ping successfully (host unreachable) the 172.17.0.31 (<-this i sthe controller in the outside office) from inside its own vlan. 172.17.0.26 is a VM in the hospital attached to the core where i initially did testing from. I cannot pint the 31 address or access it via web browser from this machine or any other machine in this vlan. But i can access all the devices from any machine not on that vlan. 

TESTS:

ping/browser test from VM/my own laptop on Vlan177(172.17.0.0/16) to 172.17.0.31 - Host Unreachable/no browser

ping/browser test from VM/my own laptop on Vlan177 (172.17.0.0/16) to any other address on any other vlan - Succesfull

ping/browser test from VM/my own laptop on any vlan but 177 to 177(both the 172.17.0.26 and .31) is succesfull

CONFIG: - This is only the config for what pertains to the vlan177

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

vrf context INSIDE

ip route 172.17.0.0/16 Null0
ip route 172.17.0.30/32 192.168.34.1
ip route 172.17.0.31/32 192.168.34.1

interface Vlan177
no shutdown
description voice-172.17.0.0/16
vrf member INSIDE
ip address 172.17.0.2/16
ip ospf passive-interface
ip router ospf 1 area 0.0.0.0
hsrp version 2
hsrp 177
preempt
ip 172.17.0.1
hsrp 1177
preempt
ip 172.17.0.30

router ospf 1

vrf INSIDE
area 10.20.0.0 stub no-summary
area 10.20.8.0 stub no-summary
area 10.20.16.0 stub no-summary
area 10.20.24.0 stub no-summary
area 10.20.32.0 stub no-summary
area 10.20.40.0 stub no-summary
area 10.20.48.0 stub no-summary
area 10.20.56.0 stub no-summary
area 10.20.58.0 stub no-summary
area 10.25.0.0 stub no-summary
area 10.30.0.0 nssa no-summary default-information-originate
area 10.56.7.0 nssa no-summary
area 10.69.8.0 stub no-summary
area 192.168.27.0 stub no-summary
area 192.168.34.0 nssa no-summary default-information-originate
area 192.168.43.0 nssa no-summary default-information-originate

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

I think I know what the issue is. simply put there is nothing stating for the internal routing of that vlan that if you need to access .31 go here first. I feel like it might have something to do with area 0.0.0.0 not being in the ospf 1 vrf inside table. but i am not sure as to what that might actually do and well i work at a hospital so i walk on egg shells 24/7 as there is no downtime normally. if someone knows how to fix this issue it is greatly appreciated. thank you all in advance.

[mlund]

Hi

For me it looks like a missing proxy arp. If a machine inside the network 172.17.0.0/16 tries to access another machine on the same network, this access is then done directly between these machines, without going to the default router. Starting with an arp request to the destination ip. Because it is local ip the router will not answer that arp. However, if the router is configured to do proxy arp, it will answer that arp request if it knows how to reach the destination.

/Mikael

[MHM Cisco World]

ping from laptop, i.e. from wifi so the traffic is encapsulate inside CAPWAP and send to controller, from there the traffic is routing. 
so take consider this case it may controller site issue not yours.

[Richard Burts]

I think that /Mikael is on the right track that this is an issue with arp. But I do not think that proxy arp is involved or would provide a solution. I believe that this is what is going on:

- if your laptop attempts ping to any address in 172.17.0.0 other than 172.17.0.31 your laptop sends an arp request as a broadcast in the local vlan, receives an arp response and ping is successful.

- if your laptop attempt ping to 172.17.0.31 your laptop sends an arp request as a broadcast in the local vlan, there is not anything to forward the broadcast to the remote small subnet, so your laptop does not receive an arp response and ping fails.

- if a laptop outside of vlan 177 attempts ping to 172.17.0.31 it does not arp for 172.17.0.31 but arps for the gateway to access vlan 177. laptop receives an arp response for the gateway, sends ping request to the gateway, the gateway has a route statement for the address and forwards the ping and it is successful.

To understand why proxy arp would not provide a solution we need to think about the behavior of the svi for vlan 177. Proxy arp is for responding when an arp request is received for a remote destination. When the vlan interface receives an arp request for 172.17.0.31 the vlan interface considers this to be a local address and does not respond. If you think about what does recognize that 172.17.0.31 being remote it is the L3 forwarding process and not the vlan interface.

[mlund]

Richard may be absolutely right, but I recall that I have done this, but it's long time since, and it was on classic old IOS. Maybe nexus don't support it, or, I may have remember wrong. Maybe worth trying, I myself don't have any possibility to try though.

/Mikael

[MHM Cisco World]

you can be sure by connect laptop via wired to SW port, this port is assign with VLAN same as AP and then check the L3 reachability. 
the issue from using Wifi. 

[Richard Burts]

I do not believe that this is an issue about using wifi. And I do not believe that this is an issue about IOS vs Nexus. I believe that it is architectural. If you have a large subnet, then you carve out of that address space a very small subnet, and that small subnet is remote then there is going to be an issue when devices in the larger subnet attempt to communicate with the smaller subnet if they need to arp for the remote small subnet.

Perhaps /Mikael has done something like this. The closest I can think of would be the Cisco IP Mobility feature in which a mobile device (very small remote subnet) is able to communicate with the larger subnet. But I do not think that is what we are talking about here. For information about IP Mobility feature see this link:

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/mob_ntwks/configuration/15-mt/mob_ntwks-15-mt-book/imo-mbl-ntwks.html

Perhaps one other way to think about this issue is that basically it creates a discontiguous network. And in discontiguous networks hosts in one part have problems communicating with hosts in the other part.

[asochowski]

thank you all for the input. i think i was thinking of this too much with a logical mind of them being in the same subnet. but like richard say while technically they are, they really arent with the /31 network. i was able to reason with mitel and them changing the IP address literally took less than five minutes on their side. i believe they were worried about things not always being able to communicate properly if it is on a different subnet (which i dont know where they have installed equipment before but that seems easier to do all the time instead of extending out a network through a /31 network). eitherway i do appreciate being able to get feedback on the issue as i thought i was loosing it and hearing the explanations made things a lot more clear as to why this wasnt working. which gave me leverage in just having them change the ip of the one controller on their side. 

[Richard Burts]

Thanks for the update. Glad that our explanations have been helpful.