Cisco IP routing on L3 switches

[BetronParks]

I'm trying to make sure I understand the full capability of using the "ip routing" command on a L3 switch. Enabling "ip routing" ONLY enable inter-vlan routing assuming the SVI of the switch is the gateway of the VLAN. If I have some switch down the line with with SVI for VLAN 1 and 2, traffic is still being routed all the way to the gateway of the VLAN before being routed to the other VLAN.

[Pawan Raut]

For all the traffic destined to other networks/vlans the traffic go to that vlan gateway first and then inter vlan routing happens.

Please rate for helpful post

[Richard Burts]

If I am understanding the question correctly it is about an environment where there are multiple switches connected on some vlan (perhaps vlan 2). Perhaps they are sw1, which connects to sw2, which connects to sw3 all of whom have ports in vlan 2 and where ip routing is enabled on at least sw1 and sw3. If my understanding is not correct then please provide clarification.

I believe that the question assumes some host is connected on sw1 in vlan 2 and that the default gateway on that host is sw3. If the host connected to sw1 sends a packet whose destination is in a different subnet the host will send the packet with sw3 as the default gateway (the destination mac address is the address of sw3). The question is whether the packet is forwarded all the way to sw3 or whether a routing action is performed on sw1. My experience is that the routing action will be made on sw1 (assuming that the routing table on sw1 has a viable path to the destination subnet) and not on the configured default gateway.

[Jon Marshall]

Rick 

Are you sure about that ? 

If the default gateway is on sw3 then how can sw1 do the routing between vlans because the packet has the destination mac address of the SVI on sw3. 

Jon

[Joseph W. Doherty]

My experience has been, if hosts have a default gateway, they send to it regardless if they pass through another L3 switch with a SVI on the same network.  Perhaps if hosts don't have a gateway defined, and gateways are acting as proxies, the "closest" switch might get the host's traffic, but I've not used proxy gateways so I'm unsure.

Also, of course, if another gateway has the better path (via routing) to the destination, and redirect not disabled, whatever gateway got the initial packet to that destination, will redirect to the better gateway.  This too, I believe, ignores where the gateways are, physically, relative to the sending host.

[Joseph W. Doherty]

Yes and no.  "IP routing" enables routing, but that also means the routing process needs to "know" about routes and interfaces.  For directly connected interfaces, with IPs, it has that information, so it can route between such route prefixes on that device.  These might be SVIs or they might be routed ports.

Regarding another switch, down the line (on shared VLAN[s]) it doesn't matter whether those shared VLANs have SVI on the same shared VLAN(s).  What matters is how hosts on the shared VLAN(s) find or "know" their gateway.

For example, given:

host1 (192.168.50.10/24) <VLAN50> L3SW1 (SVI 50 192.168.50.1/24, SVI 60 192.168.60.1/24) <trunk> L2SW2 <trunk> L3SW3 (SVI 50 192.168.50.2/24, SVI 60 192.168.60.2/24) <VLAN60> host2 (192.168.60.10/24)

host 1 can use either 192.168.50.1 or 192.168.50.2 at a gateway to get to host 2

and

host 2 can use either 192.168.60.1 or 192.168.60.2 at a gateway to get to host 1

however if you had:

host1 (192.168.50.10/24) <VLAN50> L3SW1 (SVI 50 192.168.50.1/24) <trunk> L2SW2 <trunk> L3SW3 (SVI 60 192.168.60.1/24) <VLAN60> host2 (192.168.60.10/24)

even if VLANs 50 and 60 are known to both L3SW1 and L3SW3, your would a static routing statements, or IGP for hosts 1 and 2 to communicate.

Lastly for something like:

host1 (192.168.50.10/24) <VLAN50> L3SW1 (SVI 50 192.168.50.1/24) <trunk> L2SW2 <trunk> L3SW3 (SVI 60 192.168.60.1/24) <VLAN50> host2 (192.168.50.11/24)

No routing is needed between hosts 1 and 2.

[Richard Burts]

Restating what I understand to be the essence of the question. If I have something wrong please provide clarification.

There are 3 switches, sw1 connects to sw2 and sw2 connects to sw3. vlan 2 is present on each of the switches and carried on the connection between switches. sw1 and sw3 have SVI for vlan 2 with appropriate IP addresses and mask (and probably does on sw2 but that is less important in this discussion). There are other vlans/other subnets in the network and these are known to all of the switches for which ip routing is enabled. There is a host connected to vlan 2 on sw1 and this host has an appropriate IP address and mask and its default gateway is the SVI address of sw3. The question is if that host on sw1 sends a packet with an IP destination address in a remote subnet is the forwarding done on sw1 or on sw3.

Here is what I believe is happening. If a switch is configured with ip routing enabled when a frame arrives on an SVI the switch will evaluate the destination mac address (in this case it would be the mac of sw3 in vlan 2)  and prepare for a layer 2 forwarding decision, and also the switch evaluates the destination IP address and prepare for a layer 3 forwarding decision. I believe that the layer 3 evaluation is performed first and if the switch finds that the destination IP is remote and if the switch has an appropriate entry in its routing table for the destination subnet then the switch does the layer 3 forwarding (not the layer 2 forwarding). The layer 2 process would have forwarded to sw3 but precedence is given to the layer 3 process.

[Jon Marshall]

Rick 

Genuinely interested in this because this is the first time I have ever heard this. 

My understanding is sw1 simply forwards the packet to sw3 based on the mac address because the mac address does not belong to any interface on sw1 but it does have an entry in it's mac address table pointing out of the interface connecting to sw2. 

In this case why would it need to evaluate the L3 address, it would only do this if the mac address matched one of it's own interfaces. 

I would be very interested to see any documentation for this behaviour because I have never come across this before. 

Jon

[Richard Burts]

In thinking about what I have just posted I believe that I could have described it in somewhat different terms which might have been helpful. So let me try again:

We are used to thinking of switches using a layer 2 process to forward traffic based on the destination mac address. (this forwarding is always within a certain vlan). When we enable ip routing we introduce a layer 3 process to forward traffic based on the destination IP address. The question is which process takes precedence. It is my belief that the layer 3 process takes precedence. So the IP forwarding decision would have been on sw1.

[Jon Marshall]

I can't see how it can because any L3 device only process packets that have a mac address matching one of it's interfaces. 

I guess a simple test would be to add a route for the remote subnet on sw1 with a next hop of null0 and have the route for the remote subnet on sw3 with the correct next hop IP which I may be able to do with a couple of switches at work. 

I have to say though I have never seen this behaviour in any documentation I have read. 

Jon

[Joseph W. Doherty]

Rick, sorry if it seems I too am piling on, but my understanding too is, unless you direct the packet to a L3 switch's SVI's MAC, a frame is handled as would a pure L2 switch.  So, if your frame has sw3's MAC, sw1 will not route the packet.

What I believe we're discussing the the 3rd example in my OP.

[Richard Burts]

I thought that I had seen this situation but it has been a long time and perhaps my memory is not correct. In responding I thought that the layer 3 logic took precedence over the layer 2 logic. But the more I think about it I believe that I was probably not correct and that layer 2 logic has precedence over layer 3 logic in a L3 switch. So probably the behavior is that sw1 forwards the frame and that routing is done by the configured gateway sw3. 

[Giuseppe Larosa]

Hello @Richard Burts ,

I agree with @Jon Marshall :

if the host has already resolved the MAC address of SW3 SVI the frame will be

L2 switched by SW1 to SW2 to SW3 and then SW3 will perform inter VLAN routing.

This happens in a classic scenario where each switch is a standalone device.

To be noted in the modern approach called SD Access ( managed by DNA C)  where IS-IS builds a fabric between member switches , LISP is used to locate the switch where a remote host is connected to ( either in the same VLAN or in another one) VXVLAN with UDP encapsulation is used at the data plane, L3 forwarding uses an Anycast Gateway approach so each member switch is able to perform the L3 switching between VLANs/subnets.

Something similar can happen on Nexus vPC where each device can process directly packets for the HSRP VIP regardless of the fact to be or not the active member for the group.

Another classic example near what Rick describes happens when the host has a wrong shorter subnet mask and ARPs directly for the destination address thinking it is directly connected. In this case if proxy ARP is enabled SW1 can answer to the ARP request with its own MAC address and then perform inter VLAN routing.

In the past when troubleshooting a Catalyst C6500 with FWSM , CSM or ACE load balancers we could see inside a single box how a packet could have the default gateway on the MSFC and then to be sent to  FWSM or the CSM or ACE to reach a virtual server and then a real server. Really difficult the first times .

Hope to help

Giuseppe