Cannot get VLAN in spanning tree forwarding state

[rsjordan00]

We have a pair of 6500 switches, each having a trunk going to each access switch. We set the spanning tree priority on Core1 so it is the root bridge for all VLANs. We have two different types of access switches:

3550 setup as VTP client and ISL

2960 setup as VTP transparent and dot1q.

Pruning is disabled but we use "switchport trunk allowed vlan" to restrict which VLANs go through each trunk. When we need to permit a VLAN through a trunk, we simply run "switchport trunk allowed vlan add <VLANID>" on the access switch and both core switches. If it is a 2960 in VTP transparent mode, we must set the VLAN to active. Once this is done, a "show int trunk" will reflect the new VLAN in "Vlans in spanning tree forwarding state and not pruned" for Core1.

I recently went through this process to add VLAN 250 on a 3550 access switch, but the VLAN is not listed in STP forwarding state and not pruned. I tried removing the VLAN from the trunks and redoing it, but there is was no change. I tried adding VLAN 257, but the same behavior happened. I then tried trunking the same VLANs to a few other access switches. Three other 3550s experienced the same behavior, but I was able to trunk the VLAN to a few 2960 switches. At this point, I figured it might be related to some kind of limitation of VTP or the 3550 switches. I provisioned a new 3550 with the same IOS and settings (VTP client, ISL). To my surprise, all VLANs configured were in STP forwarding state and not pruned.

Running show spanning-tree on the core and access switch shows VLAN 250 as designated FWD. I confirmed we are not hitting the limits in "show spanning tree summary totals" on the Core or Access switches. I also confirmed we are not hitting the virtual port limit by running "show vlan virtual-port slot x."

Does anyone have any other ideas on what could be causing this issue? My next action might be to shut/no shut the uplink to Core1 from the access switch, but I'm not sure if that will fix it and even if it does, I have no clue what caused the issue.

[Rolf Fischer]

How do you add VLANs on the 3550 (vtp client)? Are the 6500 configurted as vtp servers and you just configure new VLANs here? If so: Do you see the same vtp revision number on your 3550?

[rsjordan00]

Yes, the Core1 (6500) is a VTP server. I did confirm both the core and access switch have the same revision number and they are within the max VLANs:

Core1#sh vtp status

VTP Version                     : 2

Configuration Revision          : 304

Maximum VLANs supported locally : 1005

Number of existing VLANs        : 292

VTP Operating Mode              : Server

VTP Domain Name                 : *****

VTP Pruning Mode                : Disabled

Access-D3#sh vtp status

VTP Version                     : running VTP1 (VTP2 capable)

Configuration Revision          : 304

Maximum VLANs supported locally : 1005

Number of existing VLANs        : 292

VTP Operating Mode              : Client

VTP Domain Name                 : *****

VTP Pruning Mode                : Disabled

[Rolf Fischer]

> Number of existing VLANs        : 292

What type of STP are you running?

There are limitations of PVSTP+ / Rapid PVSTP+ on 2k and 3k platforms: Max 128 per-VLAN STP instances!

http://www.cisco.com/en/US/docs/switches/lan/catalyst3550/software/release/12.2_44_se/configuration/guide/swstp.html#wp1150172

If your're not running MSTP, there's the serious risk of bridging-loops.

If, in contrast, your're running MSTP, maybe here's the problem.

- Instance/VLAN-Mapping?

- Revision?

HTH

Rolf

[graham smart]

- Has the L2 vlan been created? ( Silly question really.. But this is normally what stops it from showing in a sh span tr )

conf t

vlan xxx

name whatever

end

We we see the swicthes config?

( Config of ports with the vlan on and an output of a show vlan )

-Graham
Please note: My comments are simply suggestions. I cannot be held liable for any loss of data, life or marbles due to following my instructions.

Live chat software for websites. Increase sales.

[glen.grant]

  Sounds like vtp to  the 3550 is not working correctly.  Do the vtp revision numbers match between the 6500 and the 3550's ?  All vlans are created on the 6500 for vtp .  Is the trunk actually working correctly ?  If you hardcode the trunk "switchport mode trunk "  even if it says it's trunking which it will as long as you have a L1 connection it still may not be working correctly.  You can try using "switchport mode dynamic desirable" on each side of the 6500 to 3550 link and see what happens. Sorry didnt see Fischers post .  These are things to check , vtp revision number should match on each side otherwise trunking  and vtp is not working correctly.  Verify vtp domain names match exactly. Verify native vlans match on each side of the trunk.  Do we know there is an actual operational problem?  If you put a device into the new vlan on 3550 can it get to other subnets on the 6500 ?  Think Fischer hit it on the head if there is only 128 pvst sessions which means the 3550 will not allocate a spanning tree instance in that case for the vlan and thus the readings you are seeing .  If you are restricting the trunk on the 3550 side though it should take care of that issue.

Sample-- match on both sides

interf fast/01

switchport

switchport trunk encap ISL

switchport trunk allowed vlan x,x,x

switchport mode dynamic desirable

[rsjordan00]

To answer your questions:

1. Our core is running pvst, our access switches are running rapid-pvst. I'm not sure why it was done this way. I figured r-pvst is backwards compatible.

2. Yes, the VLAN is set to active on the access switch.

3. The trunk is working with the other VLANs. Just not the new VLANs I recently tried to add.

4. The VTP revision number matches on the access switch and core.

I'm really curious about the 128 pvst sessions. I thought that manual pruning via "switchport trunk allowed vlan" would help. So, if we have 292 active VLANs in our VTP domain, but only 12 VLANs are allowed to that trunk. Can someone point me to documentation on this 128 session limit and how to check our usage?

And thanks for all the responses.

[Rolf Fischer]

You're right, manual VLAN pruning on trunks (switchport trunk allowed vlan ...) also can limit the PVSTP-Instances.

If no (trunk)port ist active for a given VLAN, there is no need for a STP instance for that VLAN.

A handy command is "show spanning-tree summary [totals]"

[rsjordan00]

I opened a TAC case (624076911), and the engineer found the following bug:

CSCtf07138 Bug Details

VLANs remained pruned after VTP Pruning disabled

Symptom:After VTP Pruning is disabled or VTP is moved to transparent mode, VLANs remained pruned on trunks.

Conditions:Unknown

Workaround:Shut / no shut on the port will fix the issue

Originally, we did have pruning enabled but we disabled it several months ago since we are manually pruning. This particular VLAN was active in our VTP domain, but not on this access switch, so it would have originally been pruned. I did run "show int gi0/1 pruning" but it only says that VTP pruning is disabled. I'm going to shut/no shut the uplinks on the access switch within the next night or so.

[rsjordan00]

I forgot to update this post. This was related to bug CSCtf07138. I ran a shut/no shut on the uplink and it fixed the issue.

[Rolf Fischer]

Thanks for sharing this information!

[CHRISTOPHER KANE]

I wanted to share my thanks to the original poster. We ran into exactly this issue yesterday. We had the situation in an IDF/MDF. The IDF had just been upgraded and we noticed we had accidently left off Rapid PVST and forgot to set VTP mode to Transparent. So we made those changes only to the IDF switch, the MDF hadn't been upgraded/replaced yet. But it was the MDF switch that developed the problem seemingly introduced by this bug.

Our symptoms were that devices who already had IP addresses continued to work fine. But new devices (ex. phones and Access Points) could not get a DHCP address. We started running packet captures in multiple points in the network. We could see the device boot up and work through the normal DHCP steps but the DHCP server didn't seem to be responding.

We found that the MDF switch was not forwarding the DHCP server response down the trunk interfaces to the IDF switch. The MDF switch had erroneously decided to VTP prune the VLANs from the trunk ports. We issued the shut/no shut on the MDF switch as suggested in this post and immediately the MDF switch began to correctly forward the necessary VLANs down the trunks.

Here's what the trunk interfaces looked like when the problem existed:

Port        Vlans allowed and active in management domain

Gi1/0/2     1-3,10,16-23,25-26,30,32,40,56,58,71-77,99,102,104,110,118,120,124,128,141,146,150,152,156-158,160,170,180,190,200,202,204,214,221-222,250,301,303,314,356,520,526,672,720,920,925,995-996,998-999

Gi1/0/3     1-3,10,16-23,25-26,30,32,40,56,58,71-77,99,102,104,110,118,120,124,128,141,146,150,152,156-158,160,170,180,190,200,202,204,214,221-222,250,301,303,314,356,520,526,672,720,920,925,995-996,998-999

Port        Vlans in spanning tree forwarding state and not pruned

Gi1/0/2     1

Gi1/0/3     1,200,221

Here's what the trunk interfaces looked like after we fixed it with a simple shut/no shut of the trunk interfaces on the MDF switch:

Port        Vlans allowed and active in management domain

Gi1/0/2     1-3,10,16-23,25-26,30,32,40,56,58,71-77,99,102,104,110,118,120,124,128,141,146,150,152,156-158,160,170,180,190,200,202,204,214,221-222,250,301,303,314,356,520,526,672,720,920,925,995-996,998-999

Gi1/0/3     1-3,10,16-23,25-26,30,32,40,56,58,71-77,99,102,104,110,118,120,124,128,141,146,150,152,156-158,160,170,180,190,200,202,204,214,221-222,250,301,303,314,356,520,526,672,720,920,925,995-996,998-999

Port        Vlans in spanning tree forwarding state and not pruned

Gi1/0/2     1-3,10,16-19,21-23,25-26,30,32,40,56,58,71-77,102,104,110,118,124,128,141,146,150,152,156-158,160,170,180,190,202,204,214,222,250,301,303,314,356,520,526,672,920,925,995-996,998-999

Gi1/0/3     1-3,10,16-23,25-26,30,32,40,56,58,71-77,99,102,104,110,118,120,124,128,141,146,150,152,156-158,160,170,180,190,200,202,204,214,221-222,250,301,303,314,356,520,526,672,720,920,925,995-996,998-999

Again, thanks to the original poster for sharing this problem, the Bug ID and the fix.

Later,

-chris