Routing Question

[jeff.cook]

I have a Cisco 3550-12G at the center of my network. It is acting as a router. All internal networks are attached directly to it via GBICs. I then have 1 static default route out to our firewall. This works great. However when a link goes down, IOS update, down fiber, whatever, all traffic goes to our firewall. Now this is exactly what I would expect to happen. Then when the link comes back up, most of the internal traffic still goes to the firewall. When I look at the devices that are having issues I see that the route table has been updated to show the firewall as the route to other internal networks. There are no routing protocols turned. To get traffic to flow correctly I need to remove this route from the table. This can take a long time and on some devices I don’t have access to the route table all I can do is reboot it.

So this is my question. Is there a way to have the Cisco router only route the traffic and not try to update the route table on the nodes?

Thank you for any help you can give.

[skarundi]

Sounds like the firewall is proxy ARPing. Its only a guess. the following document explains how proxy arp works.

URL: http://www.cisco.com/warp/public/105/5.html

[jeff.cook]

Sounds like that could be it, however it is not updating the ARP table only the route table.

I did forget to add one thing. This seams to only be happening to the servers that are on the same network as the firewall. It effects everything on the network, only beacuse all of the servers are in that one subnet.

[smif101]

Can you provide the config of this 3550. You say that you are not running any routing protocols yet you get dynamic routes entered into the routing table. Are there separate routers hanging off of this 3550?

[jeff.cook]

There are a number of 3550-48 SMI and 3548XL that are attached to the 3550-12G with GBICs. None of the other switches are routing. Only the 3550-12G. Then on one of the 3550-48 (the data center switch) is a firewall (not a PIX). There are no dynamic routes. I have directly connected routes and a static default route.

Whenever a link to a branch goes down the servers in the data center talk to the core. The route to that branch is not there because the link is down. This is because it is a directly connect route not a dynamic route. The core sends the packets to the default route (firewall). Now some how the server puts a static rout in it's route table, not the route table of the 3550. The route then needs to be cleared from each of the servers before it can talk to the branch when the link comes up. Now this didn't happen when we were using Bay ARN routers and T1 lines to the branches. None of the servers or the router is running any routing protocol. I have attached a basic network diagram and the config of the core.

Thank You

[Kanwar]

One thing you might want to try is putting a persistent route in the servers or devices. for your internal subnet you can point them to the switch. because it is static it will override anything that is learned dynamically.

Hope it helps

[baldy]

This sounds like perfectly normal behaviour. The Switch has lost a local subnet and so no longer has a route to it. Packets forwarded to it will be resolved to the default route. As the packets orginate on a connected network the switch (acting as a router) will do two things. Forward the packet to the known next-hop i.e the firewall and then send an ICMP redirect to the source (your server) telling it to use the Firewall as its next hop for that host-specific network. The servers will respond to this Redirect by placing a route in its route table.

What would normally happen when the link comes back up is the firewall on learning the network is now available via the switch would redirect the server back.

So the question is 1, is there a routing protocol between the Firewall and the switch and 2, is the firewall allowed to send redirects to the inside networks.

If its all set up statically you have a problem a the firewall will never redirect you back again.

A possible solution is to connect the firewall to the switch on its own subnet, the redirects would then not be an issue.

Pete

[jeff.cook]

Thank you Pete this really helped understand what is happening. I was kind of thinking the firewall was sending out the route change.

So our firewall is set to not route packets back inside. I think most firewalls would do this. Our firewall would of course just route those packets back to the switch, because that is what it's route table says. I'm guessing that the firewall is just dropping the packets. Is there a way to turn off ICMP redirect on the switch, and is that an OK idea? Or would it be better to allow the firewall to route internal traffic? Or can I get the switch to send out the ICMP redirect when the link comes up. Could I manually send out the ICMP redirects?

Thank You

[rwcrowe]

no ip redirects on 3550

[jeff.cook]

Will/could this cause any issues?

Thank You

[baldy]

It would certainly stop the redirecting under the circumstances you describe.

However what will happen is when any hosts local to your firewall subnet access anything via the firewall , all the traffic will flow via the 3550 as it is no longer capable of redirecting the host to the correct gateway.

So you need to work out how much traffic this is likely to be and if it will have an impact on your lan performance. If the only thing via the firewall is a low speed (in LAN terms) internet link then it shouldn't present a problem.

Alternatively if the only hosts in the same subnet as the firewall are a managable number of server you could configure persistant routes on the servers, internal networks to the switch, default route to the firewall.

There are lots of ways to skin this particular cat.

Pete

[jeff.cook]

Thank you for everyone’s help so for. The info you have been giving me really helps.

What if I change what the IP address was attached to? Right now I have the Cisco 3550 setup with each GBIC having an IP address. So when a link goes down so does the route. What if I change it so I create a VLAN for each link and assign the IP address to it. Will this work? Will it change the load on the device? Can you think of any reason why I should or should not do this?

Thank You