L3 Switch - Route Failover

[tolinrome tolinrome]

I have 5 L3 switches that are all connected to each other with routed links. Each of the 5 locations has its own L3 switch. All switches are using OSPF and see routes to each other. One of the switches physically went down. A couple of the switches were routing through the switch that went down in order to reach their destination.

In the attached diagram I drew, switch 4 went down and switch 2 default route is switch 3 and witch 3 default route is switch 4. But, as mentioned before switch 2 & 3 have routes in its routing table to Switch 1 & 5 but it wouldn't route there. The purpose of having all L3 routed switches was for the purpose of having a route the other way if 1 of the switches went down. How can I accomplish this?

Thanks.

[rais]

Are all the routes OSPF generated or you have static routes too?

[Richard Burts]

We do not have enough information to be able to identify the cause of your issue. Given the drawing and assuming correct configuration of OSPF, and assuming no other factors like static routes etc I would expect that if 4 went down that 2 and 3 should have stopped trying to route through 4 and should have routed the other way. 

A starting point to investigate the issue would be the output of these commands from each of the routers

show ip interface brief

show ip ospf neighbor

show ip route (when 4 was up and active)

show ip route (when 4 was down)

It would also be helpful if you would post configuration of the routers (especially 2 3 and 4)

[tolinrome tolinrome]

The only static route on each switch is the default route to the next hop switch. All other routes in the switches routing tables are generated by OSPF.

[Giuseppe Larosa]

Hello @tolinrome tolinrome ,

you should make the main building #5 to generate a default route in OSPF.

All other sites at that point would use the shorted path to the default route if you remove the static default route in each of them

Note: I am assuming you have a single exit point to the internet in the main building.

If you have multiple exit points you can still generate a default route in each ASBR to be injected in OSPF and you can choice if you have a primary / secondary exit points or they work at the same time.

Also you should use p2p L3 links otherwise next-hops stay alive in ARP table even if one link or device fails.

Hope to help

Giuseppe

[rais]

Default route with unreachable next-hop [switch 4 in this case] would only black-hole the traffic. 

You need to inject default-route via OSPF.

HTH.

[Joseph W. Doherty]

You're using individual static routes for a routes, including the default route, on your switches?  If yes, just one per switch, or do you have floating statics to cover the loss of a link?  Further, do you have SLAs for such statics to cover lost of reachability while link still up?

If using statics for default route, have you considered using OSPF dynamic default injection?

OSPF routes, should re-converge with loss of link or loss of reachability.

[tolinrome tolinrome]

Thank you for all your responses. I will research these and reply back.