We have a Cisco C3650-48 port in stacking and in L2 only mode (no ip routing) with default gateway set.
There were static routes entered (not sure why) and may be by mistake. Since then we are seeing stale static routes with AD 0 in the show ip route output. Find below the snap -
Gateway of last resort is 10.0.1x.50 to network 0.0.0.0
S* 0.0.0.0/0 [0/0] via 10.0.13.50
22.214.171.124/32 is subnetted, 1 subnets
S 126.96.36.199 [0/0] via 10.0.1y.x, Vlan1
188.8.131.52/32 is subnetted, 1 subnets
S 184.108.40.206 [0/0] via 10.0.1y.x, Vlan1
10.0.0.0/8 is variably subnetted, 8 subnets, 2 masks
S 10.0.x.10/32 [0/0] via 10.0.1y.x, Vlan1
S 10.0.x.23/32 [0/0] via 10.0.1y.x, Vlan1
S 10.0.x.102/32 [0/0] via 10.0.1y.x, Vlan1
C 10.0.1x.0/24 is directly connected, Vlan1
L 10.0.1x.150/32 is directly connected, Vlan1
S 10.0.24x.x0/32 [0/0] via 10.0.1y.x, Vlan1
S 10.x.0.y/32 [0/0] via 10.0.1y.x, Vlan1
When ip routing is enabled, the stale routes just vanish away.
Anyone seen such abnormal behaviour ? As the switch is in stack, we wont be able to reload the entire stack due to production network. Also there are no bugs reported as such.
Let me know if any hints or workaround possible.
First of all, I did not come across a similar behavior - I suppose that I would need to test this in a lab.
However, with no ip routing configured, the entire contents of show ip route are irrelevant - with the IP routing disabled, the contents of the routing table are not used; in fact, you should not even see the routing table printed out. I suspect that with no ip routing, the switch does not really maintain the contents of the routing table, as it is unused anyway. Only after configuring the ip routing back, the switch reinitialized its routing table and removed the stale entries.
Unless you have experienced any problems with the traffic, I would consider this to be a cosmetic issue. Once again, with no ip routing configured, the switch does not really maintain or use the routing table, so whatever is left there has very likely no impact whatsoever.
Thanks for the quick reply. The cocern was raised as customer has couple of servers in the /24 subnet which were impacted due to reverse traffic getting blackholed due to routing on L2 stack. The same /24 subnet is used on the L2 stack for switch management.
The other thing is, if we ping any host the /32 static route for that host gets installed (see the snapshot in the first post for 220.127.116.11 and 18.104.22.168). Though this is cosmetic customer has been stressing for the actual cause/bug and further investigations to stop this hebaviour completely. As it is a banking customer, they are not ready for the reload of the stack and what I feel is it will not be useful just to reload a single switch at a time to avoid complete downtime.
If its L2 can you not remove the static routes form the global config , turn off ip routing and save it.
If its an L2 device only you shoudnet even have routing on really just set the default-gateway
no ip route ......
We have tried all that. Currently we do not have static routes, ip routing is disabled (no ip routing) and default gateway set. But still we are seeing those static routes. The weired behaviour is, why any ping done from the switch installs a host route with AD 0.
Even Tac has not seen such behaviour and has no findings yet.