Catalyst 2950 Behaving rather strangly...

[cknipe]

Hi,

I'm going to be honest from the strart, this is my first attempt at VLANs. I managed to get everything up and running by myself, but obviously I did something wrong...

Here's some of the issues that I am having.

- FastEthernet Ports will simply for no reason, shut down. fa0/3 and fa0/24 did this already. I tried manually enabling them again (no shutdown), as well as pulling out / replacing the cables. Nothing worked, power cycling the switch is the only way I know of (up to now), to get the ports back to a operational state.

- I'm getting ALLOT of CRC Errors (+- 30 per 5 minutes, on a IDLE port), virtually on ALL the ports. If it was one or two ports, I could believe that it was a faulty cable / NIC on the other end, but because this is happening on EVERY SINGLE FastEthernet Interface, again, I have to question this from the Switch's perspective, and not the LANs.

<snip>

wsmd-vlanmgr01.ournet.co.za#sh int fa0/3

FastEthernet0/3 is up, line protocol is up (connected)

Hardware is Fast Ethernet, address is 0012.7f7e.22c3 (bia 0012.7f7e.22c3)

Description: WSMD-CORE02 - International Access Router

MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,

reliability 254/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Carrier delay is 100 msec

Full-duplex, 100Mb/s, media type is 100BaseTX

input flow-control is unsupported output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input never, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:41:10

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 232000 bits/sec, 43 packets/sec

5 minute output rate 235000 bits/sec, 44 packets/sec

109128 packets input, 72987313 bytes, 0 no buffer

Received 6 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

521 input errors, 521 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

0 input packets with dribble condition detected

110715 packets output, 73418266 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

</snip>

Virtually, no load on the Interface - yet, quite a large number of CRC errors, considering the interface counters was reset a mere 41 minutes ago.

- I cannot enable (no shutdown) more than 1 VLAN at a time. Again, I'm no VLAN expert, so I don't know whether or not this is normal. The VLANs do work though. The VLAN database also does not show that the VLAN joined all the ports it is configured on. Yet, the VLAN is however ACTIVE on all the ports it has been configured on.

- From plugging in the cable, to being able to transmit data over the VLAN takes a VERY long time (more than 30 seconds). Again, is this normal????

- The ports are VERY sensitive. Once I pluged a cable in, simply touching the plug causes the plug to loose the connection and the Interface resets to down. It's rather irritating accidently moving (or touching basically) a UTP cable, and wham, there goes the port's connection. If the VLANs didn't take so long to reconfigure, then I might still let this pass, but at this stage, it's a huge problem.

If you can maybe shed some light for me, I'll appreciate it TREMENDIOUSLY. Also, allot of this may have to do with my configuration. Frankly, I have ALLOT of faith in Cisco, so I'm pretty sure these are configuration issues.... I've attached a dump of my configuration below....

--

Chris.

[Kevin Dorrell]

Chris,

I cannot comment on all of your config because there is a lot of stuff there I have never used before, so I am not qualified to comment. However, there were a couple of points I would make:

1. Why do you have the spanning-tree link-type shared? I looked at the doc for this command, and I couldn't work out what you are doing with it, particularly with links that are forced full duplex.

2. If you set the speed or duplex of a link, then the other end must be configured - it should not be left auto. That may be why you are racking up so many errors. I find the best way is to set everything auto unless I have a good reason to force it. If you do force it, force it on the connected equipment as well.

3. Try leaving out the aggressive UDLD for the moment, at least on one interface and see if that helps. UDLD should only be enabled if the equipment at the other end of the cable is also doing UDLD.

Yes, you expect 30 seconds delay from plugging is a cable to the port going into forwarding. You can bypass this 2 x 15 second delay on ports connected to hosts only by setting spanning-tree portfast on the interface. But don't do it on a link connected to another switch!

Hope this helps. Let us know how it goes.

Kevin Dorrell

Luxembourg

[sstudsdahl]

Chris,

Kevin makes some excellent recommendations. I would agree that the CRC errors are likely a speed/duplex mismatch between the switchport and the device connected to it. Here are some other things you might consider.

With the "spanning-tree link-type shared" configured on the full duplex interfaces you are not allowing that switchport to participate in the spanning-tree protocol when using RSPT+. This is the same thing that the "spanning-tree bpdufilter enable" command does.

With the "spanning-tree bpduguard enable" command, you are telling the switch to not send or receive BPDU's out the port. This is a good command, along with bpdugaurd, to use when you do not want a device attached to the switchport from participating in your spanning-tree.

The dot1x commands that you have configured, in particular the "dot1x port-control force-unauthorized" are likely the reason that your ports are shutting down. This command tells the switch to not allow communication on the switchport and to not repspond to EAPOL (802.1x Authentication) packets received from the device connected to the port. The 802.1x authentication would allow the port to allow traffic to pass once a successful authentication occurred. If you are not using 802.1x authentication into the network, I would leave these commands out.

As to the issue with only being able to have a single VLAN interface up at a time, this is a limitation of the 2950. This VLAN interface is used more for managing the switch that anything else. A switch that allows you to have more than one VLAN interface up at a time, like a Catalyst 3550, would actually route packets from one VLAN to the other. The 2950 doesn't support this. It is possible that there is an exception to this in a switch that would allow you to have more than one VLAN interface up at a time and not route between them, but I am not sure as to what it would be,

Another command that you might consider would be:

"errdisable recovery cause all"

This will tell the switch that anytime a switchport gets shutdown due to an error, to re-enable the port after a period of time which can be configure with the command "errdisable recovery interval " command.

A macro command for configuring a switchport for a connection to a PC/router (non-spanning-tree speaking device) would be to use the command "switchport host" under the interface configuration mode. This will place the port directly into a forwarding state by adding the command "spanning-tree portfast" as Kevin mentioned. It also places the port into "access mode" and disabled the FastEtherChannel group the port is assigned to be default.

Steve

[cknipe]

Hi Steve,

Thanks for your (and Kevin's) suggestions. I have tried them - some have worked, some have not. The good news, is that it seems the CRC errors are gone. You where right about the speed / duplex issue.

I have also taken your other recommendations to heart - unfortunately to no avail.

All my ports has PCs / routers connected to them. Ports 1 - 3 basically runs "VLAN Capable" devices, hence, I configured a trunk on them to allow those routers to access more than 1 VLAN. Port 7 is the only one that will have a switch connected to it (so I have not configured the spanning-tree portfast on this one interface).

Now, all my interfaces basically have the same configuration:

interface FastEthernet0/x

description ADSL Router (International Access)

switchport access vlan 600

switchport mode access

carrier-delay msec 100

delay 1

spanning-tree portfast

spanning-tree bpduguard enable

However (ESPECIALLY on FA0/5 & FA0/6), my problems are not resolved. sh int fa0/5 shows the interface up and running, ex:

FastEthernet0/5 is up, line protocol is up (connected)

Hardware is Fast Ethernet, address is 0012.7f7e.22c5 (bia 0012.7f7e.22c5)

Description: ADSL Router (National Access)

MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Carrier delay is 100 msec

Full-duplex, 100Mb/s, media type is 100BaseTX

input flow-control is unsupported output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:32, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

181 packets input, 16361 bytes, 0 no buffer

Received 8 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 8 multicast, 0 pause input

0 input packets with dribble condition detected

339 packets output, 28234 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

However, on the switch, the light remains amber and never goes over to green. I also cannot transmit / receive data over it.

FA0/6 is dead (even after a power cycle). This is a PERFECT example of what I was talking about earlier when the ports would just mysteriously shut down. The output for fa0/6 shows the following:

FastEthernet0/6 is down, line protocol is down (err-disabled)

Hardware is Fast Ethernet, address is 0012.7f7e.22c6 (bia 0012.7f7e.22c6)

<-- cut to reduce size -->

It's the err-disabled that is catching my eye, and hence, I have no idea what this is, or where it is coming from.

[Kevin Dorrell]

Chris,

Normally, I would expect to see the amber light on the port that is err-disabled. Are you sure that you are counting the ports correctly? On a 2950 switch, they are counted 1 to 12, as opposed to most routers, which start counting from 0.

Let me stick my neck out and say that I think that F0/5 is actually working correctly, and that the problem is on F0/6. Please could you post the full show int F0/6 and we can have a go at diagnosing it. Also, show int F0/6 switchport might show us something.

Meanwhile, I'll go back and look at your original F0/6 config to see if I notice anything strange. There is rather a lot in the config for that line; perhaps you could post the config for the interface as it is now, and talk us through your reasons for departing from the defaults in each line of the config.

When the port goes into errdisable, do you get any console messages?

Kevin Dorrell

Luxembourg

[cknipe]

Hi Kevin,

The interface stats:

wsmd-vlanmgr01.ournet.co.za#sh int fa0/6

FastEthernet0/6 is down, line protocol is down (err-disabled)

Hardware is Fast Ethernet, address is 0012.7f7e.22c6 (bia 0012.7f7e.22c6)

Description: ADSL Router (International Access)

MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Carrier delay is 100 msec

Auto-duplex, Auto-speed, media type is 100BaseTX

input flow-control is unsupported output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:23, output 00:00:23, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

1377 packets input, 308517 bytes, 0 no buffer

Received 56 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 56 multicast, 0 pause input

0 input packets with dribble condition detected

2401 packets output, 199394 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

wsmd-vlanmgr01.ournet.co.za#sh int fa0/6 sw

wsmd-vlanmgr01.ournet.co.za#sh int fa0/6 switchport

Name: Fa0/6

Switchport: Enabled

Administrative Mode: static access

Operational Mode: down

Administrative Trunking Encapsulation: dot1q

Negotiation of Trunking: Off

Access Mode VLAN: 600 (VLAN0600)

Trunking Native Mode VLAN: 1 (default)

Voice VLAN: none

Administrative private-vlan host-association: none

Administrative private-vlan mapping: none

Administrative private-vlan trunk native VLAN: none

Administrative private-vlan trunk encapsulation: dot1q

Administrative private-vlan trunk normal VLANs: none

Administrative private-vlan trunk private VLANs: none

Operational private-vlan: none

Trunking VLANs Enabled: ALL

Pruning VLANs Enabled: 2-1001

Capture Mode Disabled

Capture VLANs Allowed: ALL

Protected: false

Appliance trust: none

wsmd-vlanmgr01.ournet.co.za#

Config snipetts:

interface FastEthernet0/6

description ADSL Router (International Access)

switchport access vlan 600

switchport mode access

carrier-delay msec 100

delay 1

spanning-tree portfast

spanning-tree bpduguard enable

There is a small ADSL Router plugged into the port. Basically 0/5 and 0/6 are exactly the same - the only difference is that they are in different VLANs.

Because I am a little worried my session here will timeout, have a look at this so long, I will run back to the switch momentarily to make SURE which lights is on / off / amber.

Unfortunately, the switch is located quite far away, and I can't access the switch via console. Again, I know this is possible (I just dont know how!!) If you can perhaps just dump me the one or two lines to get debug messages on a vty line, I'll happily enable them.

[cknipe]

I had a look at the switch again. On the front of the switch, the ports are marked 1x through to 24x. Thus, 1x = fa 0/1 and 24x is fa 0/24, surely.

This is the part which baffles me COMPLETELY now. From my previous post 0/6 is err-disabled. Yet, x6 is GREEN!!!! x5 is OFF (Completely), yet, sh int fa0/5 shows the interface running. I can take a photo of it as well if it's really required, I'm 100% possitive about this.

wsmd-vlanmgr01.ournet.co.za#sh int fa0/5

FastEthernet0/5 is down, line protocol is down (err-disabled)

Hardware is Fast Ethernet, address is 0012.7f7e.22c5 (bia 0012.7f7e.22c5)

Description: ADSL Router (National Access)

MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Carrier delay is 100 msec

Auto-duplex, Auto-speed, media type is 100BaseTX

input flow-control is unsupported output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:18, output 00:00:19, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

735 packets input, 93720 bytes, 0 no buffer

Received 63 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 63 multicast, 0 pause input

0 input packets with dribble condition detected

1710 packets output, 134291 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

wsmd-vlanmgr01.ournet.co.za#sh int fa0/6

FastEthernet0/6 is up, line protocol is up (connected)

Hardware is Fast Ethernet, address is 0012.7f7e.22c6 (bia 0012.7f7e.22c6)

Description: ADSL Router (International Access)

MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Carrier delay is 100 msec

Full-duplex, 100Mb/s, media type is 100BaseTX

input flow-control is unsupported output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:51, output 00:00:01, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

1438 packets input, 315408 bytes, 0 no buffer

Received 62 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 62 multicast, 0 pause input

0 input packets with dribble condition detected

2566 packets output, 211102 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

[Kevin Dorrell]

What sort of cables are you using for these connections, and how long? Could we have a cabling issue here?

If I were you, I would set the delay and the carrier-delay back to the defaults, default delay, unless you have a specific reason for setting them. The defaults are usually pretty well thought out, and the switches usually work out-of-the-box for basic functions.

You can get your console messages on your vty session by typing term mon.

Kevin Dorrell

Luxembourg

[cknipe]

Ok. I've attached a new copy of the active running config (after I have made all the recommended changes). I've also attached a jpg which outlines the network diagram, which should give some more clarity on what we require (although, I have to admit we made some changes and it's not 100% accurate).

From a networking point of view, we basically decided to rather go with VLANs and in doing so, to bring down the number of NICs required in WSMD-CORE01, WSMD-CORE02, and GIE001.cust-gw.

What we thus have, is VLANs 100 (198.19.0.8/29), 400 (198.19.0.0/29) and 500 (PPPoE) on FastEthernet 0/1

VLANs 300 (192.168.1.0/24) should run between ports FastEthernet 0/2 and FastEthernet 0/7 (which will have a switch / hub connected to it).

VLANs 200 (198.19.0.32/27), 400 (198.19.0.0/29) and 500 (PPPoE) is running from FastEthernet 0/3.

0/4 is linked to a Wireless Access Point (198.19.0.10) which forms part of VLAN100.

0/5 and 0/6 is the PPPoE connected to two ADSL Routers (that is giving us the problem now).

0/7 is connected to a 10/100 switch, giving access to 192.168.1.0/24 and putting everything in VLAN300.

The rest of the ports are shutdown (not used), and ports 0/17 - 0/24 forms a "switch" on the 2950, putting all the ports inside VLAN200.

Where we are now, everything is working 100% with the exception of FE0/5 and FE0/6.

I'm also 100% POSSITIVE about this... 0/5 is running (according to IOS), yet, no light is on for the port. 0/6 is err-disabled, yet, the light on the switch is Green. I know this doesn't make sense, but this is how it is... I can take pictures of the switch to proove this if required.

[Kevin Dorrell]

This may sound silly, but I have seen hardware where the PCB is mounted so badly in the case that the LEDs are visible through the adjacent port window, especially when viewed obliquely. Are you sure that is not the case here? What about the first and last ports?

Normally, a yellow light means disabled, no light means enabled but not connected, and green light means OK.

Just another thought ... are you sure you are looking at the port status? The LEDs can mean different things according to the display mode. Press the "mode" button to change the display mode and to view different things. To view port status, the "stat" LED should be lit.

Kevin Dorrell

Luxembourg

[cknipe]

Ok....

I will make a small video clip power cycling the switch... Not sure where I can post it, but you will get it. I am 100% sure.. No, 500%.

Cabling problems.... Never had them before, as I did say originally, I have tried NUMERIOUS cables...

Turning on logging / warnings... We may be getting somewhere....

01:51:53: %PM-4-ERR_RECOVER: Attempting to recover from bpduguard err-disable state on Fa0/6

01:51:55: %SPANTREE-2-BLOCK_BPDUGUARD: Received BPDU on port FastEthernet0/5 with BPDU Guard enabled. Disabling port.

01:51:55: %PM-4-ERR_DISABLE: bpduguard error detected on Fa0/5, putting Fa0/5 in err-disable state

01:51:55: %LINK-3-UPDOWN: Interface FastEthernet0/6, changed state to up

01:51:55: %LINK-3-UPDOWN: Interface FastEthernet0/5, changed state to down

01:51:56: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/5, changed state to down

01:51:56: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/6, changed state to up

So yes, there is a configuration error on FA0/5 & 0/6. The fact that I am receiving a BPDU from a PLAIN & Simple ADSL router.... Well yeah, maybe thay are not so simple.... You'll know better what to recommend :)

[cknipe]

OK.

no spanning-tree bpduguard enable on fa0/5 and 0/6... Both ports are back online.... Thanks for all the help people... It would seem to me like I am back in business.

[Kevin Dorrell]

OK, that's pretty evident. Remove the bpduguard on F0/5 and F0/6, and the problem should go away. I presume the ADSL router has a built-in switch, and so does Spanning Tree. In that case it would be safest to remove the portfast on F0/5 and F0/6 as well.

If all this has been useful, please don't forget to checkmark the posting that resolved the problem, and to "Rate this post".

Kevin Dorrell

Luxembourg

[cknipe]

Hi Kevin,

I shall do that. Again you have been BRILLIANT to help me. And yes, you are correct.

This is a temporary solution though (We are testing / installing in a lab environment). On our production rollout, we will be utilising two different modems. I completely forgot it was only one modem currently - and thus yes, it caused the problem.

I'll keep portfast and bpduguard disabled for now, I presume I should be fairly OK to re-enable them once the second router has been installed.

Again, thank you tremendiously.