Switch that SolarWinds Cannot Reach

[Patrick McHenry]

Hi,

I have a switch that solarwinds is reporting down and from the switch I also cannot ping SolarWinds from the managment address of the switch.

The switch is not currently routing but when I do an sh ip route:

sh ip route
Default gateway is not set

Host               Gateway           Last Use    Total Uses  Interface
172.20.8.211       10.3.1.254            0:00       6096264  Vlan103

I get this entry. It's not configured. The 172.20.8.211 is Solarwinds and the 10.3.1.254 is the default route so this is why the switch is not pinging out of the vlan but how do I get rid of this entry?

Thanks, Pat

[John Blakley]

Yes...I would disable it everywhere...

[Robert R]

I had a similar problem to this and it turns out the cause was ICMP redirects.  The SVI on the layer 3 switch that was doing the routing did not have the no ip redirects command configured, so it basically told the switch to use what was at the time an alternate route to the destination ip address.  To resolve, do the following...

On the switch that solarwinds can't reach.

clear ip redirect

then from the the device doing the layer 3 routing for your management vlan, under the svi or subinterface (whichever you use)

no ip redirect  (there are legitimate uses for this, so you may want to read up on this a bit more before implementing this command.

http://www.cisco.com/en/US/tech/tk365/technologies_tech_note09186a0080094702.shtml

try pinging again, all should work. 

Hope that helps.

Rob

[Patrick McHenry]

I'm trying to understand when you would need to use ip redirects or more accirately if we need this. We have an Internet connection that is accessed by going through a Firewall on the same LAN. The LAN gateway ip is 10.10.1.1 and the firewall address to reach the Internet is 10.10.1.254. There is a default route in the core switch that is 0.0.0.0 0.0.0.0 10.10.1.254. I don't see why we would need ip redirects as the default route would take care of things, correct?

Thanks, Pat.

[John Blakley]

Redirects are usually sent when it's routed back out the same interface. In general, if you expect all of your traffic to use the default gateway (which is what I believe should be happening) then you can turn ip redirects off.

[Patrick McHenry]

Not exactly - if someone is going to the Internet they would want to go to 10.10.1.254 (firewall). But, I'm thinking that the default route of 0.0.0.0 0.0.0.0.0 10.10.1.254 would take care of things -no? So, I'm assumming traffic comes from a device wanting to go to Google. It hits the core switch where the gateway is and the default route. The Core switch will send the traffic to the firewall using the default route. Is this correct?

Thanks, Pat.

[Patrick McHenry]

Actually, it might have to go into the Gateway VLAN interface and then come back out to the 10.10.1.254 firewall interface.

[John Blakley]

Yes, if it doesn't have a more specific route...look at it like this:

If you have 2 routers on the same switch, one router (R1) has a connection to the ISP with a default route, and the other router (R2) has a connection to the mpls cloud. Both routers internal address are 10.10.10.1 and 10.10.10.2 respectively. If you have a host trying to get to www.cisco.com, it will send the request to the default gateway. (10.10.10.1 in this case.) The router will then look up what the destination address is and realize that it doesn't have a route, so it sends the traffic to its default gateway.

Now, the same host wants to get to something internal at 172.20.20.50. So, it sends traffic to its default gateway at 10.10.10.1 (R1). R1 looks up the destination in the routing table and sees that to get to 172.20.20.0/24, send traffic to 10.10.10.2 (R2). In this case, R1 will forward the traffic back out the interface that it received it on to R2 and at the same time it will send a redirect to the host. The host will put it in its cache stating to get to 172.20.20.50 go to R2. It will not send traffic to R1 for this destination as long as the cache entry is in its table.

John

[Patrick McHenry]

John,

So, From what you say, in my case I need to leave ip redirect on the interface because the second example describes my situation.

Thanks, Pat.

[John Blakley]

That's completely up to you. In theory, it helps hosts know of a direct route, but in practice it can also cause issues because now the host doesn't query the router for a better path.

[Patrick McHenry]

So, the host will get to the right destination either way. With ip redirect you might have slightly better performance? Because the host will know of the route - but, like you said, it could cause problems like I experienced.

Thanks, Pat.

[Patrick McHenry]

Thanks for the explanatrion byt the way - it cleared it up.

[John Blakley]

Yes, Solarwinds in this case tried to get to a host that was out R1 for instance, and R1 decided the best way to get somewhere was over R2. In my case, the best route is to always go over the mpls circuit, but when it's down it needs to go over the cell card. So, my R1's default route changes from pointing out to the mpls cloud to now pointing to R2 because that's the new default route. It was sending a redirect to my hosts which caused my issue.

Glad I could be of some help