Core Network Routing Issue

[KENNEDY BIZUAYEHU TADEGE]

Dear All, We've a network which is composed of two 5525 ASA firewalls configured for fail over, two 4507 switches configured as the core/distribution, they are configured to be dhcp servers for separate scopes, static and ospf routing with their own loopback address, layer two etherchannel between them and 2960 switches at an access layer. the problem is when both 4507 switches are working together internet becomes terribly slow but if one of them is down the internet connection becomes fast and stable. some times we can't even ping the ip address of the Ephone (the service provider DSL device) which is connected to the outside interface of the ASAs.

please advise on the issue.

BR

[Jon Marshall]

Difficult to advise without knowing your routing, firewall setup etc.

How many vlans, are they all routed on the 4500s, what is the OPSF for ie. you mention statics and only the 4500s and firewalls so why OSPF.

Are all the access switches uplinked to both switches, are these L2 trunks or L3 links,  what STP are you running, are the firewall inside interfaces in their own dedicated vlan, do you point to the active firewall IP as the next hop for the default route etc etc.

Jon

[KENNEDY BIZUAYEHU TADEGE]

Thanks for your valuable response !

Actually OSPF is configured for future connectivity with other branch offices. currently we are using only the static routes

all the access switches uplinked to both switches, are L2 trunks.

I've attached the config files.

[Jon Marshall]

Your DHCP configuration on the 4500s makes no sense.

On ds1 you have defined all the DHCP pools with a default gateway of 10.6.x.1 for each pool but then for half of all the pools you exclude all IPs.

And on ds2 you have defined all the DHCP pools with a default gateway of 10.6.x.2 for each pool and then you exclude all the IPs for the pools that ds1 is responsible for.

You have two switches but absolutely no redundancy ie. if ds1 fails then all clients with a default gateway pointing to ds1 cannot use ds2 because it's vlan interface has a different IP.

This is not the way to use two switches as a pair.

You should be using something like HSRP between the two 4500s so if one fails then the gateway for clients can be moved to the other 4500.

Also you are running OSPF for all the interfaces.

Can you post a "sh ip ospf neigh" from one of the switches. 

Jon

[KENNEDY BIZUAYEHU TADEGE]

since it's a mid night I'm at home and I don't have access to the switches. But when one of the switches goes down all Computers get connection from the one that is active.

for redundancy we configured the below

spanning-tree mode rapid-pvst
spanning-tree extend system-id
spanning-tree vlan 2,16,20,24,28,32,36,40,44,48,52,56,60,64,68,72 priority 4096
spanning-tree vlan 76,80,84,88,92,96,100,104 priority 4096
spanning-tree vlan 108,112,116,120,124,128,132,136,140,144,148,152 priority 0
spanning-tree vlan 156,160,176,192,208,224,228,232,236,240,242-243 priority 0
spanning-tree vlan 252 priority 0

[Jon Marshall]

Okay, the above is STP not end client redundancy.

I need to go over them again but I can't see how it works because if a client is using ds1 as it's gateway then it is using the SVI IP on ds1.

If that switch fails then the SVI for the same vlan on ds2 does not have the same IP address so the clients default gateway is now wrong.

I'm not criticising just trying to work out how it is meant to work but you can see clearly that the SVIs for the same vlans use different IPs on each switch so depending on which switch the client is using if that switch goes down then the clients default gateway is no longer available.

Like I say i'll have a look again and perhaps I missed something but do you understand what I mean ?

Jon

[KENNEDY BIZUAYEHU TADEGE]

I understand what you are saying and what is your suggestion ? 

[Jon Marshall]

Well firstly I will go over the configs again as I may have missed something and I haven't looked at the firewall yet.

I don't really like making suggestions until I understand how the network works currently as I could make things worse and I don't really want to do that :-)

I can't understand how all clients continue to work when there is only one switch so I would like at least to understand that part.

The DHCP pool and clients configuration my be a part of the problem as may the OSPF side of things as there will likely be multiple peerings across the trunk interconnect between the switches.

I'll have a good luck at the configs tomorrow but I may need you to do some tests when you are at the switches.

Jon

[KENNEDY BIZUAYEHU TADEGE]

Hi Jon,

Sorry for the late response and please find below my answers.

I didn't exclude all IPs from DS2, instead I split the scope into two which is half for DS1 and the rest for DS2. our subnet is 255.255.252.0

If we shut DS1 down, clients get IP from DS2

Regarding the inside subnets on ASA, we have NAT configured

we've configured OSPF for branch connectivity on future. For current usage we are using static routes.

Connectivity between ASA and 4500 is through VLAN 252 (no IP)

If we go down to one switch it all works fine, which switch is that ? Both

Unless I am missing something your configuration needs changing so that both switches can act as the default gateway for clients if one of them is down.How ?

BR

Kennedy

[Jon Marshall]

Kennedy

I didn't exclude all IPs from DS2, instead I split the scope into two which is half for DS1 and the rest for DS2. our subnet is 255.255.252.0

That's not what the configs are showing, see my example from above. You are doing the right thing in principle but ds1 only has a few IPs excluded.

But yes my mistake with the mask so ds2 does have some IPs.

Are the 4500s and the ASA running OSPF between them ?

Jon

[KENNEDY BIZUAYEHU TADEGE]

No

[Jon Marshall]

So how do the ASAs know where to send return traffic back to your internal subnets ?

They don't have any static routes for those subnets.

Jon

[KENNEDY BIZUAYEHU TADEGE]

Actually we've configured the below ospf commands 

on the switches

router ospf 1
 network 10.6.0.0 0.0.255.255 area 4

on ASA

router ospf 1
 network 10.4.252.0 255.255.255.0 area 4
 network 192.168.1.0 255.255.255.0 area 4

and we've ospf learned routes on the switches

[Jon Marshall]

Okay, so the ASAs see two equal cost paths back to each vlan.

Where I am still confused is the DHCP setup and the SVI IPs.

You say that you can shut down either switch and it works fine but how do clients with the wrong default gateway work ?

If we could get to the bottom of that we may be able to see why with both switches your performance is poor.

Jon

[KENNEDY BIZUAYEHU TADEGE]

Regarding the default gateway

We've different SVIs configured on both switches and you can think both of the switches have independent configuration 

for instance

On DS1

ip dhcp pool VLAN20-Pool
 network 10.6.20.0 255.255.252.0
 default-router 10.6.20.1
 dns-server 8.8.8.8 

interface Vlan20
 ip address 10.6.20.1 255.255.252.0

On DS2

ip dhcp pool VLAN20-Pool
 network 10.6.20.0 255.255.252.0
 default-router 10.6.20.2
 dns-server 8.8.8.8 

interface Vlan20
 ip address 10.6.20.2 255.255.252.0

Note: the performance becomes poor only when both of the switches are on