cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1632
Views
10
Helpful
10
Replies

Master of stacked 3750 switches down will bring the network down

hxmengmetro
Level 1
Level 1

Hi All,

I have two equal cost eigrp core routers. Two 3750G stacked switch as a distribution switch connected to the two core routers. I have another router connected to the two core routers too as a remote site router. So I use stacked 3750 switches to avoid the messy STP stuff. I expect less than 500ms outage when the master switch of stacked 3750 distribution switch goes down. But every time I reload the master switch, I can't ping the switch connected to the stacked 3750 distribution switches until the previous master booted half way. I tried stack-mac persistent but it didn't work. Is there anything I'm missing to have slave switch take over all the traffic immediately without interrupting the traffic for too long time? Thanks guys. All the distribution switches are enhanced images.

Really appreicate your help!

Here is the lab configuration:

Current configuration : 2921 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname HQ_SW_DIST
!
!
no aaa new-model
switch 1 provision ws-c3750g-12s
switch 2 provision ws-c3750g-12s
stack-mac persistent timer 5
system mtu routing 1500
ip subnet-zero
ip routing
!
!
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
interface Port-channel1
no switchport
ip address 10.9.100.3 255.255.255.254
ip hello-interval eigrp 200 1
ip hold-time eigrp 200 3
ip summary-address eigrp 200 10.101.0.0 255.255.0.0 5
ip summary-address eigrp 200 10.1.0.0 255.255.0.0 5
load-interval 30
!
interface Port-channel2
no switchport
ip address 10.9.100.5 255.255.255.254
ip hello-interval eigrp 200 1
ip hold-time eigrp 200 3
ip summary-address eigrp 200 10.101.0.0 255.255.0.0 5
ip summary-address eigrp 200 10.1.0.0 255.255.0.0 5

load-interval 30
!
interface Port-channel3
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/1
description ETHCHAN To HQ_RTR_COR1
no switchport
no ip address
channel-group 1 mode active
!
interface GigabitEthernet1/0/2
description ETHCHAN To CF_RTR_COR2
no switchport
no ip address
channel-group 2 mode active
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!        
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
description ETHCHAN To ACCESS SWITCH
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 3 mode active
!
interface GigabitEthernet2/0/1
description ETHCHAN To HQ_RTR_COR1
no switchport
no ip address

description ETHCHAN To HQ_RTR_COR1
no switchport
no ip address
channel-group 1 mode active
!
interface GigabitEthernet2/0/2
description ETHCHAN To CF_RTR_COR2
no switchport
no ip address
channel-group 2 mode active
!
interface GigabitEthernet2/0/3
!
interface GigabitEthernet2/0/4
!
interface GigabitEthernet2/0/5
!
interface GigabitEthernet2/0/6
!
interface GigabitEthernet2/0/7
!
interface GigabitEthernet2/0/8
!
interface GigabitEthernet2/0/9
!
interface GigabitEthernet2/0/10
!        
interface GigabitEthernet2/0/11
!
interface GigabitEthernet2/0/12
description ETHCHAN To ACCESS SWITCH
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 3 mode active
!
interface Vlan1
ip address 10.1.1.254 255.255.0.0
!
interface Vlan2
ip address 10.101.1.254 255.255.0.0

!
router eigrp 200
network 10.1.0.0 0.0.255.255
network 10.9.100.0 0.0.0.255
network 10.101.0.0 0.0.255.255
no auto-summary
!
ip classless
ip http server
!
!
!
control-plane
!
!
line con 0
line vty 0 4
login
line vty 5 15
login
!
end

Lou

10 Replies 10

Reza Sharifi
Hall of Fame
Hall of Fame

Hi Lou,

I have tested the same exact scenario, but with OSPF.  The only thing that fixed the fail over issue from master to member switch was deploying stack-mac persistent.

Now, since this command is not helping you, I am wondering if there is any aditional command needed for EIGRP.

Reza

Thanks a lot Reza for this information. At least I know this works with OSPF. I checked the routes when the master reboot. The routes are ok during reboot. I don't think EIGRP got messed up at that time. It must be something with the stack configuration. I can't figure it out at this time. Do you see anything I'm missing for the stack configuration?

By the way, have deployed this setup in production enviornment? Is there any issue with it? Thanks.

Lou

I found reason why this failed. I have 2 3750G-12S-E switches stacked. Since I don't want to use fiber, I just use RJ45 SFP in 3750G-12S to connect one 3560's two rj45 ports. They are etherchannel. So when I powered down one member of stacked switch, it works great. The traffic is still flowing except one packet loss. But when I reboot one stack member, the problem happened. I see the link light in 3560 switch port is still green even the corresponding RJ45 SPF link light in 3750G-12S is off. When I pull this cable out of 3560G, the problem goes away, the traffic is flowing again. So The issue is when I reboot the stack switch with RJ45 SFP, the other end still thinks this link is up which confuse the etherchannel.

So the problem is why the 3560G port is lighted green while the other end of 3750-12S RJ45 SFP is lighted off. Is there anything I can configure in 3750G-12S to avoid this issue? Thanks a lot!

Lou

Hi Lou,

Is there any way for you to use fiber cable since the 3560 also comes with Gig SFP ports? I am curious to know if this issue is some how related to Copper SFP you are using.

Reza

Reza,

I don't have enough patch fiber on hands. Just ordered. Will be here next week. In addition, I put copper SFP in 3560G without using its embedded RJ45 ports. Still the same thing. The port in 3750G is off light while the port in 3560 is still green up when the 3750G reboot. I will be

Lou

I replaced the copper SFP with fiber ones. Reload serveral times and everything works during the reboot. The link light at the both ends were off when the switch was in reboot. So the copper SFP did cause the issue. But pay attention, the issue only exists between 3750G-12S and 3560G or between 3750G-12S and 3750G-24T when they have copper SFPs. It's weird issue and just be careful about copper SFP. I guess this maybe just software bug.

Lou

Hi Reza, I asked one question in another post about the down time you experience when the master goes down. Based on 3750 datasheet, the layer 2 traffic will not be affected but layer 3 will be interrupted. My simple test shows 3 seconds silence over the layer 3 IP call. Is there any optimization we can make to shorten the down time. Did you use NSF (non-stop-forwarding) feature in 3750?

Lou

Tested nsf. No difference. I also tested file transfer. The file transfer (windows copy paste between two computers) stopped when I powered off the master switch and the file transfer failed. But the reliable file transfer like FTP only paused a little bit and did continue the transfer. So end user would see a little bit interruption but not much.

Good testing Lue

In my case NSF did not make any difference either.  I even changed the OSPF timers, but those did not help either.

I only tested it with ping and if I remember correctly, I lost one ping.  I think, overall 1 to 3 seconds is reasonable.

Reza

Thanks Reza. Yes, you are right. Usually it lost one ping when I was doing the test. Good thing about the voip calls is it didn't drop the call. I found one link online saying the layer 3 traffic failover takes 3 - 5 seconds. I guess that's why I get about 3 - 8 seconds silence over the phone. Didn't test STP with HSRP. Not sure how long it will interrup the connections. But I won't go that route since it will waste one link per access switch.

Thanks again for your help here.

Lou

Review Cisco Networking for a $25 gift card