L3 Switching / VLAN Isolation Q

[sfabel]

All,

I appreciate any input on this. There must be something I'm missing - I'm grateful for any hints or answers.

We have two 4948-10GEs running in HSRP mode for some dozen VLANs. They have their priorities adjusted so they alternate in being the 'primary' between the VLANs. The VLAN interfaces are the default router for the clients in each VLAN.

The issue was encountered when we tried connecting to the DHCP server on an interface external to the local net. In this case, the return packets were dropped at the local client NIC, since the answer came back from the local IP, not the 'far' IP.

I.e.:    

Client (Ubuntu) 172.17.5.100 -> 172.16.1.6 (DHCP Server, which also has an interface with address 172.17.5.2)

Client Routing table is very simple: default -> 172.17.5.1 (HSRP address) and 172.17.5.0/24 -> eth0

This led to:

172.17.5.100 -> 172.17.5.1 -> 172.16.1.6  and a return path of  172.17.5.2 -> 172.17.5.100, where the packet was dropped.

Adding an explicit device in the local routing table like so:

$ sudo route add -net 172.16.1.0/24 dev eth0

Makes for a direct connection; in fact, my local arp cache now fills up with the switches MAC address and the 172.16.1.0/24 addresses.

So, two questions here, because I was actually trying to separate out these VLANs and now it seems as if a simple route entry on a client can revert this entire scenario into a L2 switching environment:

1. Is this behavior normal? What am I missing here?
2. Can I somehow force a "L3 barrier" between those VLANs? I mean, isn't that the point of a L3-capable switch? Ultimately, I'd like to use access lists to further qualify this barrier, but for sure I'd like to separate out the broadcast domains?

Dazed and confused,

Stephan

[Jon Marshall]

Stephan

Do you have an "ip helper-address 172.16.1.6" command on the L3 vlan interface for the 172.17.5.x subnet ?

If so then the DHCP request from the client will go direct to the DHCP server as well as be forwarded by the L3 switch. You really want an either or solution ie. if you want the DHCP server to be on a remote subnet then don't also give it an interface in the 172.17.5.x subnet.

That should help with the DHCP server issue.

In terms of the static route you added to the client.

Do the 172.16.1.x IPs in your client arp cache have unique mac addresses per IP or is the mac address the same ?

Jon

[sfabel]

Jon,

first off thanks for your answer. I appreciate your recommendation with the ip helper-address; we're probably going to implement it this way and remove the trunk into the client VLANs from the DHCP server.

As far as the ARP cache goes; I see mostly the switch's MAC address, except for the DHCP server, where I see the MAC address of the 'far' NIC.

Thanks for your help,

Stephan

[Jon Marshall · Accepted solution]

Stephan

As far as the ARP cache goes; I see mostly the switch's MAC address, except for the DHCP server, where I see the MAC address of the 'far' NIC.

That makes sense. When you added that route to your client it basically then thought 172.16.1.x was local so it arps out for any 172.16.1.x IP address as opposed to simply sending traffic to 172.16.1.x IPs to it's own default gateway.

The reason you see the switch's mac address for the entries in the arp cache is probably because the L3 vlan interface for the 172.17.5.x vlan is running proxy arp. So when it sees the arp for 172.16.1.x come from the client it responds with it's own mac address.

The client has no idea this is not the actual 172.16.1.x mac address it sent an arp for and sends the packet to the switch mac address and the switch then forwards it on to the 172.16.1.x device.

Proxy arp can be useful in some limited scenarios but usually you simply configure your client with the right IP, subnet mask, and defautl gateway and there is no need for it.

Jon