Switch port being reset for multiple times everyday

Guohua Zhang · ‎03-15-2012

Some our switch ports being reset for multiple times itself for no reason. It happened on 4948, 4506 and 6509 switches and hosts are all Dell PC with Windows 7. About 90% of similar PC are working fine without this type of problem.

Anybody knows possible reason? Thanks.

One log message on the switch. It happened in the middle night and without any interfere.

Mar 15 00:00:38 lhc3-4506-93.ncbi.nlm.nih.gov 175506: Mar 15 00:00:37.091 EDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet4/39, changed state to down

Mar 15 00:00:38 lhc3-4506-93.ncbi.nlm.nih.gov 175507: Mar 15 00:00:38.091 EDT: %LINK-3-UPDOWN: Interface GigabitEthernet4/39, changed state to down

Mar 15 00:00:42 lhc3-4506-93.ncbi.nlm.nih.gov 175511: Mar 15 00:00:41.139 EDT: %LINK-3-UPDOWN: Interface GigabitEthernet4/39, changed state to up

Mar 15 00:00:42 lhc3-4506-93.ncbi.nlm.nih.gov 175512: Mar 15 00:00:42.139 EDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet4/39

Kimberly Adams · ‎03-15-2012

Tony,

Are there any errors or anything on the switchport that you can see, like a show interface g4/39? How about the network card in the PC? Are the NIC drivers updated and stable? It the PC and the switchport having a hard time with Autonegotiation of speed and duplex?

This is some of the first things I would look at. Let us know.

Thanks,

Kimberly

Thanks and Cheers! Kimberly Please remember to rate helpful posts.

rsimoni · ‎03-15-2012

Hi Tony,

the fact that the 'issue' occurs only on ports connected to Dell PC seems to indicate that those PC's NICs get somehow disconnected... I would investigate with Dell, or looking on the Internet for Dell related issue with the particular NIC installed on those devices. It might be a driver issue, some kind of hibernation thing or who knows what.

Riccardo

Zaid Abuali · ‎03-15-2012

Hello Tony,

actually short link flap is caused by usual user or OS evens. For example, closing laptop lid usually putting it in standby or hibernate mode. Depending on the NIC driver the link may briefly flap and stays up while PC or laptop is in standby, hibernate or shut down mode.

otherwise if that is not the case then the problem might be in one of the following;-

* problem with the device connected to these interface (problem with the nic)

* problem with the cable in between.

* problem with the switch side.

can you add the following outputs it may help more:-

- show ver

- show log --> if there is more logs about this flapping than the one above

- show int GigabitEthernet4/39

Best Regards,

Zaid

Guohua Zhang · ‎03-15-2012

Thanks for all of your replies. We are currently inverstigating the NIC of those PC. The problem is we have not found any common point of those switches, switchports or hosts. BTW, there are about 90% of PCs work fine are also from Dell. Per your reques:

Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-IPBASEK9-M), Version 12.2(53)SG2, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Compiled Tue 16-Mar-10 03:16 by prod_rel_team

Image text-base: 0x10000000, data-base: 0x1259256C

ROM: 12.2(44r)SG3

Darkside Revision 4, Jawa Revision 20, Tatooine Revision 140, Forerunner Revision 1.78

lhc3-4506-93 uptime is 1 year, 3 weeks, 6 days, 21 hours, 16 minutes

System returned to ROM by power-on

System restarted at 13:11:17 EST Wed Feb 16 2011

System image file is "bootflash:cat4500e-ipbasek9-mz.122-53.SG2.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C4506 (MPC8548) processor (revision 6) with 524288K bytes of memory.

Processor board ID FOX103206SJ

MPC8548 CPU at 1.33GHz, Supervisor 6-E

Last reset from PowerUp

4 Virtual Ethernet interfaces

148 Gigabit Ethernet interfaces

2 Ten Gigabit Ethernet interfaces

511K bytes of non-volatile configuration memory.

Configuration register is 0x2102

lhc3-4506-93#sh int g4/39

GigabitEthernet4/39 is up, line protocol is up (connected)

Hardware is Gigabit Ethernet Port, address is 0018.ba2f.9c86 (bia 0018.ba2f.9c86)

Description: HOST:ncbipc0131 MAC:00256438aa6d. 2012-03-13 00:05:13

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000-TX

input flow-control is on, output flow-control is on

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:13, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 932000 bits/sec, 141 packets/sec

5 minute output rate 844000 bits/sec, 158 packets/sec

486937805 packets input, 239655849061 bytes, 0 no buffer

Received 12560193 broadcasts (2112835 multicasts)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 input packets with dribble condition detected

1948742136 packets output, 547387522735 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

Zaid Abuali · ‎03-15-2012

Hello Tony,

I can't see any problems here in the show interfaces, it seems that the interface is working fine, so you have to investigate with the PC's.

Best Regards,

Zaid

Leo Laohoo · ‎03-15-2012

Hmmmm ... 4900 switch.

Alright, let's try something out ... TDR!

1. Command: test cable tdr inter g4/39;

2. Wait for approximately between 5 to 7 seconds. Traditionally, 4500 and 6500 line cards will take 60 seconds for the test to complete.

3. Command: sh cable tdr int g4/39;

4. Please post your output.

I'm suspecting either the Pair A or Pair D is faulty.

Guohua Zhang · ‎03-16-2012

Here it is the test result:

lhc3-4506-93#sh cable-diagnostics tdr interface g4/39

Interface Speed Local pair Cable length Remote channel Status

Gi4/39 1Gbps 1-2 N/A Unknown Terminated

3-6 N/A Unknown Terminated

4-5 N/A Unknown Terminated

7-8 N/A Unknown Terminated

We did have problem with speed negotiate problem for some of PCs, (not exact consistance with these PC). Somehow we saw 10M with both switchport and PC with 1G interfaces (we can manully set static to 1000 on the switch). Could be related.

Kimberly Adams · ‎03-16-2012

Hey Tony,

When it comes to speed and duplex settings between two devices, if it is currently in autonegotiation and you are having issues, then by all means speed lock it. At this point it couldn't hurt. Recently I had an ASA that was speed locked but not liking it and I had to set it back to auto for the errors and issues to go away.

I have seen a lot of weird issues resolved with either setting a port and device to auto or speed locking them down. But do remember, if you speed lock the switchport for 1000 full then also make sure you do the same on the PC's NIC or you will have more issues. Both sides need to match for speed and duplex.

Thanks,

Kimberly

Thanks and Cheers! Kimberly Please remember to rate helpful posts.

Leo Laohoo · ‎03-16-2012

Hi Tony,

Thanks for the TDR output. (And this is why I hate doing TDR on a 4500 because the results are just "way out there!)

Do you have any 3560G or 3750G in your network? Can I ask if you try it there?

The result the 4900 is providing is not very conclusive. All 4 pairs are reporting as "Terminated". It's good but I'm not "sold" by it.