cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1104
Views
0
Helpful
5
Replies

3750 Stack: Lose VLAN SVI when Master goes down....Remaining switch does not route

Jay Matrona
Level 1
Level 1

Hi Everyone

 

The Problem:

I was on the Cisco 3750 switch stack(SW1/SW2) at our Datacenter via putty SSH using the IP of 10.224.14.103.  This IP is configured on an SVI. 

I needed to power off the master(SW2) and move its power cord into a different power strip in the server rack. The moment this happened, I lost my connection to the switch.  Not only did I lose my telnet session but all traffic ceased to pass across the remaining switch bringing down the entire environment.  Even after several minutes, which I figured would be enough time to allow the remaining switch to become the master, I still could not ping 10.224.14.103...not even from a directly connected firewall.

Even after the master(sw2) came back up, I still could not access it. As soon as I powered off SW1, the switch came back and I could reach 10.224.14.103. This issue creates a single point of failure in the datacenter.

 

 

The Cause:

I opened a ticket with Cisco TAC and went through this situation.  Here was the conclusion after TAC simulation:

the moment stack master was powered off the newly elected master did not have the same svi configuration to pass L3 traffic.  The svi would need to be manually added to this new master.

 

Possible Solution??:

I proposed the idea of making an interface on the secondary switch into an L3 interface on the same subnet as the SVI.  So, I would assign and address of 10.224.14.111 to g1/0/1.  I would need the interface to be up/up, so I would need to find a device at the datacenter on that vlan that I could connect to the interface.

Question:  Would this proposed solution fix the issue?  If the Master goes down and the SVI is lost, will this L3 interface allow for routing to continue over the remaining active switch?  Is there a different solution available?

 

Thanks

 

 

5 Replies 5

Jon Marshall
Hall of Fame
Hall of Fame

the moment stack master was powered off the newly elected master did not have the same svi configuration to pass L3 traffic.  The svi would need to be manually added to this new master.

That's not really a good answer.

The whole point of a stack is that if the master fails another switch can take over all functions, that's part of what you are paying for.

You shouldn't have to implement a workaround because the devices are not working properly.

Did they say anything else ?

Jon

I agree Jon, that is not really a good answer.  The tech was not very helpful and we went around in circles on the issue.   Anyways, my main point of contention was, what is the point of having a stack if it is this vulnerable to failure if the master dies??

Based on your comment, it sounds like the issue is at L2 and that the stack-mac persistent timer could possibly be a fix.  But again, it is a workaround.

 

Thanks

Jay

It's not actually a workaround as such because like I say the new stack master should send out gratuitous arps but some end devices don't accept them which isn't really a fault on the 3750s.

It is used quite often on switch stacks but it may not be the issue anyway.

Joe makes a good point about IOS versions, features sets.

The reason I asked about whether the new master never saw the SVI is because with your workaround if all switches are using the same configuration then it won't allow you to configure another interface in the same subnet on a member switch.

It could also be a bug in the IOS although I would expect TAC to be able to tell you that.

I appreciate it is difficult but I would be raising a complaint with TAC because the answer simply isn't good enough.

It's like having dual sups for redundancy in a 6500 and being told no you can't synchronise the configurations so all that money you spent to get near instantaneous failover is wasted because you have to manually log in and copy the configurations across.

Jon

 

Jon Marshall
Hall of Fame
Hall of Fame

Did they actually show that the SVI wasn't there or did you see that on the new master switch ?

When a new switch takes over the master role it's SVIs will have different mac addresses. As far as I know the switch should send gratuitous arps to notify of the new mac address.

However you can use the "stack-mac persistent timer" to keep the same mac address on the SVI.

Not saying that is the issue but it may be worth considering -

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750x_3560x/software/release/12-2_55_se/configuration/guide/3750xscg/swstack.html#68751

Jon

Joseph W. Doherty
Hall of Fame
Hall of Fame

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Besides what Jon posted, do your two stacked switches have the same feature set support?  It's possible to run a stack where lost of a master will also lose advanced features, such as routing.

Also in such a situation, when two such switches contend for stack master, the higher featured stack member has, by default, election priority, but if it later joins a running stack, there's no stack election and no stack master preemption.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco