VSL 4500 Interface Waiting to be aggregated

[adrian.ciubotariu.lacatusu]

Hi,

i have 2 Chassis 4500 with 2 supervisor for chasssis.

After a reboot, i have this situation:

100 Po100(SU) - Te1/3/1(w) Te1/4/1(P)
101 Po101(SU) - Te2/3/1(w) Te2/4/1(P)

and the Supervisor:

show module
Switch Number: 1 Role: Virtual Switch Active

Chassis Type : WS-C4507R+E

Power consumed by backplane : 40 Watts

Mod Ports Card Type Model Serial No.
---+-----+--------------------------------------+------------------+-----------
1 48 10/100/1000BaseT (RJ45) WS-X4648-RJ45-E
2 48 10/100/1000BaseT (RJ45) WS-X4648-RJ45-E
3 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E
4 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP)
5 18 10GE (X2), 1000BaseX (SFP) WS-X4606-X2-E
6 18 10GE (X2), 1000BaseX (SFP) WS-X4606-X2-E

M MAC addresses Hw Fw Sw Status
--+--------------------------------+---+------------+----------------+---------
1 2.2 Ok
2 2.2 Ok
3 3.1 15.0(1r)SG10 03.08.01.E Ok
4 Provision
5 2.0 Ok
6 2.0 Ok

Mod Redundancy role Operating mode Redundancy status
----+-------------------+-------------------+----------------------------------
3 Active Supervisor SSO Active
4 In-Chassis Standby RPR Standby cold

Switch Number: 2 Role: Virtual Switch Standby

Chassis Type : WS-C4507R+E

Power consumed by backplane : 40 Watts

Mod Ports Card Type Model Serial No.
---+-----+--------------------------------------+------------------+-----------
1 48 10/100/1000BaseT (RJ45) WS-X4648-RJ45-E
2 48 10/100/1000BaseT (RJ45) WS-X4648-RJ45-E
3 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP)
4 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E
5 18 10GE (X2), 1000BaseX (SFP) WS-X4606-X2-E
6 18 10GE (X2), 1000BaseX (SFP) WS-X4606-X2-E

M MAC addresses Hw Fw Sw Status
--+--------------------------------+---+------------+----------------+---------
1 2.2 Ok
2 2.2 Ok
3 Provision
4 3.1 15.0(1r)SG10 03.08.01.E Ok
5 2.0 Ok
6 2.0 Ok

Mod Redundancy role Operating mode Redundancy status
----+-------------------+-------------------+----------------------------------
3 In-Chassis Standby RPR Standby cold
4 Standby Supervisor SSO Standby hot

[balaji.bandi]

what is the reason of reboot :

After a reboot, i have this situation:

100 Po100(SU) - Te1/3/1(w) Te1/4/1(P)
101 Po101(SU) - Te2/3/1(w) Te2/4/1(P)

Is the issue you reporting 1/3/1 and 2/3/1 (W) ? ( w - waiting to be aggregated) - have you checked the reseating the SFP ?

can you also post complete :

#show switch virtual redundancy

#show switch virtual link port-channel

[adrian.ciubotariu.lacatusu]

Hello,

Yes, I have already checked the SFPs, but they work properly. The problem I found on this client, who doesn't have a log storage system, so unfortunately I don't know what happened before. I found myself two active / active chassis.

show switch virtual redundancy
Executing the command on VSS member switch role = VSS Active, id = 1
My Switch Id = 1
Peer Switch Id = 2
Last switchover reason = none
Configured Redundancy Mode = Stateful Switchover
Operating Redundancy Mode = Stateful Switchover

Switch 1 Slot 3 Processor Information :
-----------------------------------------------
Current Software state = ACTIVE
Image Version = Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-UNIVERSAL-M), Version 15.2(4)E1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 29-Jan-16 00:40 by prod_rel_team
BOOT = bootflash:cat4500e-universal.SPA.03.08.01.E.152-4.E1.bin,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2101
Fabric State = ACTIVE
Control Plane State = ACTIVE

Switch 2 Slot 4 Processor Information :
-----------------------------------------------
Current Software state = STANDBY HOT (switchover target)
Image Version = Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-UNIVERSAL-M), Version 15.2(4)E1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 29-Jan-16 00:40 by prod
BOOT = bootflash:cat4500e-universal.SPA.03.08.01.E.152-4.E1.bin,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2101
Fabric State = ACTIVE
Control Plane State = STANDBY

Executing the command on VSS member switch role = VSS Standby, id = 2

show switch virtual link port-channel
Executing the command on VSS member switch role = VSS Active, id = 1

Group Port-channel Protocol Ports
------+-------------+-----------+-------------------
100 Po100(SU) - Te1/3/1(w) Te1/4/1(P)
101 Po101(SU) - Te2/3/1(w) Te2/4/1(P)

Executing the command on VSS member switch role = VSS Standby, id = 2

Group Port-channel Protocol Ports
------+-------------+-----------+-------------------
100 Po100(SU) - Te1/3/1(D) Te1/4/1(P)
101 Po101(SU) - Te2/4/1(P)

[balaji.bandi]

If they are split brain - you need to bring down one of them from network and correct the one having issue in right slot to fix.

If the customer not allowing do any changes, then his call how long he want to have downtime. (this is no one fault, this might have verified by the customer when the deployed time and signed off after testing)

Make sure take the configuration backup out of the box, and if they are dual homed all the connection on the downside then you can make this changes, make sure you remove all connection the one you bringing down, and make correction and bring VSL and DAD and  check is the VSS formed and before you connect other Data cables.

show below document to customer :

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-4500-series-switches/118597-config-vss-00.html

[adrian.ciubotariu.lacatusu]

right now, it's not in split brain. My perplexity is about the current state.  I only wonder if it is normal for the Port-channel to be in Waiting to be aggregated and that the cards are seen as Provision and not as ok with the show module.

[balaji.bandi]

No that is not ok state for me,

 I found myself two active / active chassis.

what do you mean by this ?

if all working you should see all slots in the SVL

#show switch virtual redundancy
#show switch virtual dual-active fast-hello
#show switch virtual link port-channel

connect console to provision slot and check what status is that, and reload the module and capture all the boot logs

[adrian.ciubotariu.lacatusu]

below is an overview:

In the network under consideration, we find ourselves with a Cisco 4500 device configured with two Supervisors per chassis. In detail, the Module 3 Chassis 1 Supervisor is connected to the Module 3 Chassis 2 Supervisor, and the Module 4 Supervisor is connected to its corresponding Module 4 Supervisor. Initially, since I did not have direct access to the 4500, I connected to the 6500 device, as indicated by the network diagram. I found that the spanning-tree protocol had put all VLANs propagated via port-channel 12, i.e., the one to the 4500, into a blocked state.

Without yet having console access to the 4500, I speculated that the problem might be related to the Virtual Switching Link (VSL), which could have caused an Active/Active state between the two 4500s. Therefore, I isolated chassis 1 to restore connectivity with chassis 2. During these operations, chassis 1 seemed to be stuck and would not even allow console access, leading us to decide to reboot it.

After some time, the second chassis also stopped working. The technicians on site reported that the Supervisor LEDs were red. We then reset the first chassis, isolated the second chassis, and then rebooted the second chassis. After these operations, the situation completely normalized, except for one specific link, which is in an Up/Down state waiting to be aggregated to the Port-Channel.

[balaji.bandi]

These all the summary is normal our triage the issue.

When you connected the device, have you collected to logs.

i more intrested to see the module showing provision (might have stuck in ROMMON?)

[Reza Sharifi]

Hi,

You have a reverse active supervisor in the chassis. In one chassis, sup in blade 3 is in provision mode, and in the other chassis, blade 4. So, I recommend using one sup in each chassis (blade 3) and building the VSS based on that. Quad sup VSS has never worked that well, as when one sup goes down, the chassis will reboot anyway.

HTH

[adrian.ciubotariu.lacatusu]

Hi,

I found this architecture , we are talking about an implementation of 10 years ago, which has never restarted. Unfortunately, the customer does not allow changes at this time. I only wonder if it is normal for the Port-channel to be in Waiting to be aggregated and that the cards are seen as Provision and not as ok with the show module.

[Leo Laohoo]

At the very least, please upgrade the firmware of the supervisor cards to something "recent", like 3.8.10 or 3.11.11.