Unexpected Network outage when Changing of Spanning-tree mode pvst to Spanningtree mode rapid-pvst

[DodsonRS_2]

I have caused a major network outage by issuing the following command on the Core VSS switch which houses the datacenter servers.

#spanning-tree mode rapid-pvst.

This command is suspected in causing Windows Network Load Balancing (NLB), although I am not quiet sure that this is the case, as none of the Catalyst 3750 switches attached by were affected.

The effect was to break the NLB connected virtural Webserver Farm, rendering the URL for the internal websites unobtainable.

Please could someone help to explain this.

[Jon Marshall]

Robert

How long did the outage last ? With RSTP it is very important to make sure all your links are correctly defined ie. P2P links for switch interconnections and spanning-tree portfast on edge devices.

To be quite honest, making a change like this without a planned outage is definitely not something i would have considered doing.

Jon

[tarun_cisco]

Hi,

I just want to know here plz tht why you said unexpected.

As much as I am aware if you will change Switch port mode or stp it reconverges which can be 10 second or 50 sec and thus will cause outage almost for sure.

Please correct me if wrong.

Sent from Cisco Technical Support iPhone App

[DodsonRS_2]

All connected servers are connected in to the vss 6500 in ether channels, would they have been affected by the spanning tree mode change?

As everything that is terminated in the vss switch 6500 dual terminated in a ether channel (including access switches and servers)what role does spanning tree play?

[Peter Paluch]

Hi Robert,

Your outage was most probably caused by the fact that after the STP mode change, the VSS began the Proposal/Agreement procedure in RSTP with its connected neighbors, attempting to negotiate the rapid convergence of connected links to the forwarding state. However, assuming that the servers were not connected to PortFast ports that are unaffected by Proposal/Agreement mechanism, and the remaining switches were still running legacy STP, none of the devices responded to the Proposals sent by your VSS, and the ports on the VSS had to go over the Learning/Forwarding states using timers that took 30 seconds.

Even in networks where there are no loops and all ports can be in the forwarding state, the STP/RSTP must first determine that it is so, and that process takes time. In STP, it is driven by timers. In RSTP, it is mutually negotiated between switches and if the negotiation does not work, RSTP also falls back on timers. Changing the STP mode is a major event for switched network, and doing it in production is strongly inadvisable as there will be transient connectivity dropout until the network stabilizes.

I would personally suggest learning more about the RSTP mechanics before just turning it on. A good document is here:

http://www.cisco.com/en/US/tech/tk389/tk621/technologies_white_paper09186a0080094cfa.shtml

Best regards,

Peter

[Peter Paluch]

Robert,

I concur with all the friends here. Changing the STP version will necessarily cause a network outage because the new STP version needs to reconverge with its neighboring switches. If the switches still run the legacy STP, absolutely no rapid convergence will be attained by activating the RSTP, as the RSTP is able to rapidly converge with RSTP peers only. Also, the RSTP (and MSTP) need to have the edge ports properly configured, otherwise they may get blocked for up to 30 seconds on each topology change in the network that causes the Proposal/Agreement mechanism to kick into action.

Changing the STP version is a major change in a switched network, and should be always performed during maintenance windows.

Best regards,

Peter