Troubleshooting bad communication with Cisco access switch

tanner.zaitt · ‎08-08-2018

Hi, I will show you my topology in picture.

The topology include:
3 Access switches
1 Distribution switch

2 Core switches
1 Router

The problem is with one access switch.
The problem is that I ping for example Access switch number 3 from Computer connected to Distribution switch with tooo much packets lost...

And I cant' understand why.
I can't determine what's happening..

Can you help me to troubleshoot the problem?

I can't connect to Access switch 3 with SSH with Putty from computer connected to Distribution switch because there i see to much packets lost between Distribution switch and Access Switch 3.
I can connect to Access switch 3 only with telnet session from Distribution switch.
I connect my computer to Distribution switch, I open my Putty application and I connect to Distribution switch with SSH.
Now from this SSH session on Distribution switch I open telnet to Access switch 3.

I can check two interfaces from two sides.
The side of Access switch 3 and the side of Distribution switch.

The side of Distribution switch:
I run this command "show interfaces status"
the result is:
Interface Description Status Vlan Duplex Speed Type
Gi0/17 Uplink to Access switch 3 connected trunk a-full 100 10/100/1000BaseTX

I run this command "show running-config interface g0/17 and the configuration of port is:
interface GigabitEthernet0/17
description Uplink to Access switch 3
switchport trunk allowed vlan 10,20,30
switchport mode trunk
speed 100

Now I run this command "show interfaces g0/17 and the result is:
Hardware is Gigabit Ethernet, address is 0024.50c9.0511 (bia 0024.50c9.0511)
Description: Uplink to Access switch 3
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 5/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:01, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 92637
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 2285000 bits/sec, 228 packets/sec
5 minute output rate 334000 bits/sec, 305 packets/sec
9866438115 packets input, 13725293535579 bytes, 0 no buffer
Received 450569547 broadcasts (450119136 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 450119136 multicast, 0 pause input
0 input packets with dribble condition detected
4944231272 packets output, 387638700354 bytes, 0 underruns
0 output errors, 0 collisions, 7 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

Now I run this command "show processes cpu history" and the results are:

11111111111111
8887777777777888887777755555666664444000000000088888777776
100
90
80
70
60
50
40
30
20
10 **********************************************************
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)

1111111111131111111111111111111111111111111111111111111111
4430214655884444304122466335433226344355177441033344777343
100
90
80
70
60
50
40 *
30 *
20 ***** ** * * ** ** ***
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%

4333245324562363236393626262335422533333236325512543324223522362235321
6487309356707474840694339508442108722472323256796015537298969227948837
100 *
90 *
80 *
70 * * *
60 * ** * * * * * * * * ** * * *
50 * * *** * * * * * * * * * ** * * * * *
40 * ** ** *** * *** * * * ** * * * ** **** * ** * **
30 **** ****************** ******** ******* ****** ***** * ************
20 **********************************************************************
10 ######################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%

The Distriution switch is
WS-C2960G-24TC-L

The ios of Distribution switch is

c2960-lanbasek9-mz.122-50.SE5.bin (IOS)

Now the side of Access switch 3:
The configuration of port g0/1 is:
interface GigabitEthernet0/1
switchport mode trunk
description Uplink to Distribution switch

The information of interface g0/1 is:
GigabitEthernet0/1 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0008.3064.5509 (bia 0008.3064.5509)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 13/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 196000 bits/sec, 333 packets/sec
5 minute output rate 5285000 bits/sec, 513 packets/sec
1629155 packets input, 124162866 bytes, 0 no buffer
Received 181988 broadcasts (45740 multicasts)
0 runts, 1 giants, 0 throttles
1 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 45740 multicast, 0 pause input
0 input packets with dribble condition detected
1572149 packets output, 1890853573 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

The cpu history of switch is:

11111
6666666666333336666644444444444444455555555554444444444444
100
90
80
70
60
50
40
30
20
10 ******************** **********
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)

11111 111111 111111111111111111 11 11111111 11111111111211
3112191221119110011110021000000811500110111801001111010300
100
90
80
70
60
50
40
30
20 *
10 ########*###*########*#####*#***#****#####*##*##########*#
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%

5
5
100
90
80
70
60 *
50 *
40 *
30 *
20 *
10 #
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%

Access switch 3 is WS-C2960-8TC-L

The ios here is

c2960-lanbasek9-mz.122-50.SE5.bin

I tried to shutdown all ports in Access switch 3 without port G0/1 the uplink port to Distribution switch but the situation is not changed.

Other switches like Access switch 1 and 2 are okay.

But this Access Switch 3....

I run this command "show log" and the result is:

*Mar 1 00:00:36.230: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
*Mar 1 00:00:37.379: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
*Mar 1 02:00:39: %SYS-6-CLOCKUPDATE: System clock has been updated from 00:00:39 UTC Mon Mar 1 1993 to 02:00:39 BG Mon Mar 1 1993, configured from console by console.
*Mar 1 02:00:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan121, changed state to down
*Mar 1 02:00:40: %SYS-5-CONFIG_I: Configured from memory by console
*Mar 1 02:00:40: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C2960 Software (C2960-LANBASEK9-M), Version 12.2(50)SE5, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Tue 28-Sep-10 13:44 by prod_rel_team
*Mar 1 02:00:40: %SSH-5-ENABLED: SSH 1.99 has been enabled
*Mar 1 02:00:40: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 192.168.2.38 Port 514 started - CLI initiated
*Mar 1 02:00:41: %LINK-5-CHANGED: Interface Vlan1, changed state to administratively down
*Mar 1 02:00:43: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
*Mar 1 02:00:43: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up
*Mar 1 02:00:43: %LINK-3-UPDOWN: Interface FastEthernet0/3, changed state to up
*Mar 1 02:00:43: %LINK-3-UPDOWN: Interface FastEthernet0/4, changed state to up
*Mar 1 02:00:43: %LINK-3-UPDOWN: Interface FastEthernet0/5, changed state to up
*Mar 1 02:00:43: %LINK-3-UPDOWN: Interface FastEthernet0/7, changed state to up
*Mar 1 02:00:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
*Mar 1 02:00:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
*Mar 1 02:00:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/3, changed state to up
*Mar 1 02:00:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/4, changed state to up
*Mar 1 02:00:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/5, changed state to up
*Mar 1 02:00:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/7, changed state to up
*Mar 1 02:00:48: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to up
*Mar 1 02:00:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan121, changed state to up
.Aug 8 16:05:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up
Aug 8 17:56:36: %SYS-5-CONFIG_I: Configured from console by Taner on vty0 (192.168.1.81)
Aug 8 17:57:12: %SYS-5-CONFIG_I: Configured from console by Taner on vty0 (192.168.1.81)
Aug 8 18:06:31: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
Aug 8 18:06:32: %LINK-5-CHANGED: Interface FastEthernet0/2, changed state to administratively down
Aug 8 18:06:32: %LINK-5-CHANGED: Interface FastEthernet0/3, changed state to administratively down
Aug 8 18:06:32: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
Aug 8 18:06:32: %LINK-5-CHANGED: Interface FastEthernet0/4, changed state to administratively down
Aug 8 18:06:33: %LINK-5-CHANGED: Interface FastEthernet0/5, changed state to administratively down
Aug 8 18:06:33: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
Aug 8 18:06:33: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/3, changed state to down
Aug 8 18:06:33: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/4, changed state to down
Aug 8 18:06:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/5, changed state to down
Aug 8 18:06:48: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
Aug 8 18:06:48: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up
Aug 8 18:06:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
Aug 8 18:06:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
Aug 8 18:06:51: %LINK-3-UPDOWN: Interface FastEthernet0/3, changed state to up
Aug 8 18:06:51: %LINK-3-UPDOWN: Interface FastEthernet0/4, changed state to up
Aug 8 18:06:52: %LINK-3-UPDOWN: Interface FastEthernet0/5, changed state to up
Aug 8 18:06:52: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/3, changed state to up
Aug 8 18:06:52: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/4, changed state to up
Aug 8 18:06:53: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/5, changed state to up
Aug 8 18:07:11: %LINK-5-CHANGED: Interface FastEthernet0/7, changed state to administratively down
Aug 8 18:07:12: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/7, changed state to down
Aug 8 18:07:37: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
Aug 8 18:07:37: %LINK-5-CHANGED: Interface FastEthernet0/2, changed state to administratively down
Aug 8 18:07:37: %LINK-5-CHANGED: Interface FastEthernet0/3, changed state to administratively down
Aug 8 18:07:38: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
Aug 8 18:07:38: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
Aug 8 18:07:38: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/3, changed state to down
Aug 8 18:07:38: %LINK-5-CHANGED: Interface FastEthernet0/4, changed state to administratively down
Aug 8 18:07:39: %LINK-5-CHANGED: Interface FastEthernet0/5, changed state to administratively down
Aug 8 18:07:39: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/4, changed state to down
Aug 8 18:07:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/5, changed state to down
Aug 8 18:08:31: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
Aug 8 18:08:31: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up
Aug 8 18:08:32: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
Aug 8 18:08:32: %LINK-3-UPDOWN: Interface FastEthernet0/3, changed state to up
Aug 8 18:08:32: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
Aug 8 18:08:32: %LINK-3-UPDOWN: Interface FastEthernet0/4, changed state to up
Aug 8 18:08:33: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/3, changed state to up
Aug 8 18:08:33: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/4, changed state to up
Aug 8 18:08:37: %LINK-3-UPDOWN: Interface FastEthernet0/5, changed state to up
Aug 8 18:08:38: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/5, changed state to up
Aug 8 18:08:49: %LINK-3-UPDOWN: Interface FastEthernet0/7, changed state to up
Aug 8 18:08:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/7, changed state to up
Aug 8 18:12:54: %SYS-5-CONFIG_I: Configured from console by Taner on vty0 (192.168.1.81)

I see here only my actions, nothing interesting.

The logs in the distribution switch are not interesting also.

I tried to reload Access switch 3 but doesn't help.

The communication with other Access switches is okay.

What can I check more?

I am sure here I don't have trouble with IP conflict,
I tried to shutdown interface from Distribution switch side to Access switch 3, and after this operation the ping stop and I know and I see there is no other device with the same IP address.

I think for loop, but I don's see the loop ....

I don't have ideas.

The ports from two sides looking fine..

I really can't understand what is the problem is here.

Leo Laohoo · ‎08-08-2018

12.2(50)SE train is not a stable version. Go with 12.2(55)SE.

tanner.zaitt · ‎08-09-2018

Hello we solve the problem.

Between Access switch 3 and Distribution switch the communication is not with copper network cable.

Distribution switch is in location A, Access switch 3 is in location B;
Between location A and B the communication is with fiber optic type multimode.

In location A is Media Converter A, in location B is Media Converter B.

We tried to connect two switches only with fiber optic with sfp modules and the problem is solved.

Sommething with media converters is wrong.