Problem routing between a router and a L3 switch

[SlipperyPete]

Hello,

I've run into an interesting problem trying to route VLAN traffic from a L3 switch on towards an ISP edge router. We're trying to share our connection to an ISP with another office by installing a 3750 LAN switch in the path between us and the ISP edge router. To give this other office access, we created a new VLAN on the 3750 and enabled IP routing on the switch. So the primary VLAN (I'll call it VLAN 1) on the switch is the edge network that gives us connectivity to the ISP, while this second VLAN (I'll call it VLAN 2) connects to the other office. We've had VLAN 1 in place for some time now, and we can connect via VLAN 1 to the ISP (and the WAN) and ping the VLAN 1 next hop on the ISP router, which is the Ethernet interface on their router. But when we wanted to add VLAN 2 and forward their traffic (via the existing VLAN 1 connection), the ISP's solution was to take a VLAN 2 IP address and put a seconday IP address on their router interface. From the ISP end it makes sense, because any traffic destined for either VLAN 1 or VLAN 2 will be routed accordingly by that Ethernet interface, since the router associates that interface with both subnets.

On our end, on the 3750 switch, we've enabled IP routing, and configured a default route pointing to the VLAN 1 IP address on the ISP's Ethernet interface. I think it should work like this: traffic coming from VLAN 2 (destined for the WAN) arrives at our L3 switch, the switch sees the default route and forwards it on to the ISP router, the ISP router receives it and routes it appropriately.

However, we've found that the VLAN 2 traffic dies when it hits the ISP router. We can ping between VLAN 1 and VLAN 2, so we're good right up until it hits that Ethernet interface. Also, the ISP cannot ping from that interface to VLAN 2. So VLAN 2 is simply not making it across that link between the router and the switch. I think it has to do with the router using a L3 routed interface, whereas we're using VLANs and switch ports. By configuring a secondary IP address on that interface, the router basically sees the link as two separate links - one for each subnet. But a standard Ethernet link requires both ends of the link to have an address is the same subnet. In our case, the VLAN 1 traffic is passed because the switch port on our end is a member of that VLAN. But VLAN 2 does not actually exist on that link, so the secondary IP address configured on the ISP router does not see its corresponding next hop for that subnet on the other end of the link.

So what can we do to make this work? I know it's a bit complicated to describe, so I've included a simple drawing that illustrates the issue.

Thanks.

[Jon Marshall]

Can't read visios but your description explains it all.

Basically this would work far better if the ISP simply added a route on their router for the vlan 2 subnet pointing to your switch rather than adding a secondary IP on their router. Can you ask if they can do this ?

Jon

[SlipperyPete]

Hi Jon,

Yes, I had the same thought. I'll ask them if they can do that. I think they wanted to use the secondary IP because that interface is participating in whatever routing protocol they're using (BGP, OSPF, etc.), so by having it as an active network on the interface, they don't have to redistribute any static routes.

Another option I've conisdered is configuring the switch port a router port. I've done that before in lab situations, so I know you can force a 3750 to change one of its switchports to a L3 routed port - essentially turning the switch into a router.

Thanks for the feedback.