High latency issue in LAN - between two switches.

bimalnp4me · ‎05-06-2013

HI

PC -> core SW(6500) -> fiberswitch -> fiberSW - > core SW(4500) -> ------- ping time < 1ms

Reply from 10.10.3.7: bytes=32 time<1ms TTL=254

Reply from 10.10.3.7: bytes=32 time=1ms TTL=254

Reply from 10.10.3.7: bytes=32 time<1ms TTL=254

Ping statistics for 10.10.3.7:
Packets: Sent = 927, Received = 927, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 196ms, Average = 0ms

PC -> core SW(6500) -> fiberswitch -> fiberSW - > core SW(4500) -> Desk Switch (2960) ---- ping time is very inconsistent

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=5ms TTL=254

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=17ms TTL=254 -----------------------

Reply from 10.10.3.93: bytes=32 time=5ms TTL=254

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=5ms TTL=254

Reply from 10.10.3.93: bytes=32 time=72ms TTL=254 --------------------------

Reply from 10.10.3.93: bytes=32 time=6ms TTL=254

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=7ms TTL=254

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=11ms TTL=254 -----------------------

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=6ms TTL=254

Ping statistics for 10.10.3.93:
Packets: Sent = 2076, Received = 2062, Lost = 14 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 3ms, Maximum = 160ms, Average = 7ms

can enyone help me to troubleshoot this issue.. some times it even conectivity get lost..

Leo Laohoo · ‎05-06-2013

What troubleshooting have YOU done?

1. Did you check for interface line errors?

2. When does the high latency occurs?

Bilal Nawaz · ‎05-06-2013

Well we know that the problem is probably anwhere between here:

core SW(4500) -> Desk Switch (2960)

Since everything before that is less than or equal 1ms which is acceptable LAN performance right?

Can we do a 'show interface' on the 4500 and the 2960 for the interfaces which connect them together please?

Please rate useful posts and remember to mark any solved questions as answered. Thank you.

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

bimalnp4me · ‎05-06-2013

no line errors..

I'm new to this env.. issue was initialy there..

bimalnp4me · ‎05-06-2013

no line errors..

I'm new to this env.. issue was initialy there..

lauxtobias1 · ‎05-06-2013

Hi Bimal,

could you also provide the output of following commands:

show processes cpu history
show version

Maybe this is due to a IOS bug which is causing high CPU utilization.

s.kanth · ‎05-06-2013

Bimal,

1. Can you check duplix mismatch errors on both switches.

2. check recent logs for any errors.

Thanks

Sri

bimalnp4me · ‎05-06-2013

WareHouse_1#show processes memory

Processor Pool Total: 34988900 Used: 9477732 Free: 25511168

I/O Pool Total: 4186112 Used: 1646136 Free: 2539976

Driver te Pool Total: 1048576 Used: 40 Free: 1048536

WareHouse_1#show processes cpu

CPU utilization for five seconds: 6%/0%; one minute: 6%; five minutes: 6%

bimalnp4me · ‎05-06-2013

WareHouse_1#show ip interface brief

Interface IP-Address OK? Method Status Protocol

Vlan1 10.10.3.93 YES NVRAM up up

FastEthernet0/1 unassigned YES unset up up

FastEthernet0/2 unassigned YES unset down down

FastEthernet0/3 unassigned YES unset up up

FastEthernet0/4 unassigned YES unset up up

FastEthernet0/5 unassigned YES unset up up

FastEthernet0/6 unassigned YES unset up up

FastEthernet0/7 unassigned YES unset up up

FastEthernet0/8 unassigned YES unset up up

GigabitEthernet0/1 unassigned YES unset up up

WTM-WareHouse_1#show cdp neighbors

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge

S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone

Device ID Local Intrfce Holdtme Capability Platform Port ID

XWCORE2.X.com

Gig 0/1 174 S I WS-C4507R Gig 3/48

SEP0013C4C63D00 Fas 0/8 133 H P IP Phone Port 1

SEP0021A02D56AC Fas 0/1 132 H P IP Phone Port 1

WareHouse_1# show version

Cisco IOS Software, C2960 Software (C2960-LANBASEK9-M), Version 12.2(44)SE6, RELEASE SOFTWARE (fc1)

Compiled Mon 09-Mar-09 18:10 by gereddy

Image text-base: 0x00003000, data-base: 0x01100000

ROM: Bootstrap program is C2960 boot loader

BOOTLDR: C2960 Boot Loader (C2960-HBOOT-M) Version 12.2(35r)SE2, RELEASE SOFTWARE (fc1)

WareHouse_1 uptime is 17 weeks, 14 hours, 3 minutes

System returned to ROM by power-on

System image file is "flash:c2960-lanbasek9-mz.122-44.SE6/c2960-lanbasek9-mz.122-44.SE6.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C2960-8TC-L (PowerPC405) processor (revision C0) with 61440K/4088K bytes of memory.

Processor board ID XXXXXXXXXXXXX

Last reset from power-on

1 Virtual Ethernet interface

8 FastEthernet interfaces

1 Gigabit Ethernet interface

The password-recovery mechanism is enabled.

64K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address : XXXXXXXXXXXXXXXX

Motherboard assembly number : 73-10611-07

Power supply part number : 341-0208-01

Motherboard serial number : FOC140359C9

Power supply serial number : LIT13510LWX

Model revision number : C0

Motherboard revision number : D0

Model number : WS-C2960-8TC-L

System serial number : FOC1404V0WF

Top Assembly Part Number : 800-28132-02

Top Assembly Revision Number : A0

Version ID : V02

CLEI Code Number : COM8A00ARB

Hardware Board Revision Number : 0x01

Switch Ports Model SW Version SW Image

------ ----- ----- ---------- ----------

* 1 9 WS-C2960-8TC-L 12.2(44)SE6 C2960-LANBASEK9-M

Configuration register is 0xF

bimalnp4me · ‎05-06-2013

Reply from 10.10.3.93: bytes=32 time=129ms TTL=254

Reply from 10.10.3.93: bytes=32 time=28ms TTL=254

Reply from 10.10.3.93: bytes=32 time=26ms TTL=254

Reply from 10.10.3.93: bytes=32 time=164ms TTL=254

Reply from 10.10.3.93: bytes=32 time=129ms TTL=254

Reply from 10.10.3.93: bytes=32 time=106ms TTL=254

Reply from 10.10.3.93: bytes=32 time=110ms TTL=254

Reply from 10.10.3.93: bytes=32 time=141ms TTL=254

Reply from 10.10.3.93: bytes=32 time=97ms TTL=254

Request timed out.

Reply from 10.10.3.93: bytes=32 time=159ms TTL=254

Reply from 10.10.3.93: bytes=32 time=116ms TTL=254

Reply from 10.10.3.93: bytes=32 time=130ms TTL=254

Request timed out.

Reply from 10.10.3.93: bytes=32 time=8ms TTL=254

Reply from 10.10.3.93: bytes=32 time=4ms TTL=254

Reply from 10.10.3.93: bytes=32 time=45ms TTL=254

Reply from 10.10.3.93: bytes=32 time=140ms TTL=254

Reply from 10.10.3.93: bytes=32 time=81ms TTL=254

Reply from 10.10.3.93: bytes=32 time=105ms TTL=254

Reply from 10.10.3.93: bytes=32 time=182ms TTL=254

Bilal Nawaz · ‎05-06-2013

On the 2960 can you please do 'show interface Gig 0/1'

On the 4507R can you also show us 'show int Gig 3/48'

Thank you

Please rate useful posts and remember to mark any solved questions as answered. Thank you.

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

bimalnp4me · ‎05-06-2013

SWCORE2#show interfaces GigabitEthernet 3/48

GigabitEthernet3/48 is up, line protocol is up (connected)

Hardware is Gigabit Ethernet Port, address is 0013.7f64.fd9f (bia 0013.7f64.fd9f)

Description: ###WareHOUSE##

MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,

reliability 255/255, txload 35/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000-TX

input flow-control is off, output flow-control is off

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:09, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 17120466

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 51000 bits/sec, 26 packets/sec

5 minute output rate 14013000 bits/sec, 1913 packets/sec

603576762 packets input, 222995427904 bytes, 0 no buffer

Received 215844037 broadcasts (101082233 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 input packets with dribble condition detected

7958878068 packets output, 3814037747442 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

WareHouse_1#show interfaces Gig 0/1

GigabitEthernet0/1 is up, line protocol is up (connected)

Hardware is Gigabit Ethernet, address is ec44.76ff.db89 (bia ec44.76ff.db89)

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,

reliability 255/255, txload 1/255, rxload 20/255

Encapsulation ARPA, loopback not set

Keepalive not set

Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000BaseTX

input flow-control is off, output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 8026000 bits/sec, 1112 packets/sec

5 minute output rate 40000 bits/sec, 17 packets/sec

3447611911 packets input, 1613371505792 bytes, 0 no buffer

Received 931660812 broadcasts (462987703 multicasts)

0 runts, 0 giants, 0 throttles

1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 462987703 multicast, 0 pause input

0 input packets with dribble condition detected

153756051 packets output, 58314509602 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

Leo Laohoo · ‎05-06-2013

Are the ping times from the PC to the management IP Address of the the switches involved?

Can you post the "sh interface " and "sh controller e "?

bimalnp4me · ‎05-06-2013

WareHouse_1#show controllers ethernet-controller Gig 0/1

Transmit GigabitEthernet0/1 Receive

2481345501 Bytes 2925250836 Bytes

145930333 Unicast frames 2516123349 Unicast frames

6779361 Multicast frames 462998037 Multicast frames

1052141 Broadcast frames 468687974 Broadcast frames

0 Too old frames 2466301232 Unicast bytes

0 Deferred frames 1623208448 Multicast bytes

0 MTU exceeded frames 2381842690 Broadcast bytes

0 1 collision frames 0 Alignment errors

0 2 collision frames 1 FCS errors

0 3 collision frames 0 Oversize frames

0 4 collision frames 171465 Undersize frames

0 5 collision frames 0 Collision fragments

0 6 collision frames

0 7 collision frames 25375590 Minimum size frames

0 8 collision frames 1656591706 65 to 127 byte frames

0 9 collision frames 513507334 128 to 255 byte frames

0 10 collision frames 280950707 256 to 511 byte frames

0 11 collision frames 206106014 512 to 1023 byte frames

0 12 collision frames 189194709 1024 to 1518 byte frames

0 13 collision frames 0 Overrun frames

0 14 collision frames 0 Pause frames

0 15 collision frames

0 Excessive collisions 0 Symbol error frames

0 Late collisions 0 Invalid frames, too large

0 VLAN discard frames 576254766 Valid frames, too large

0 Excess defer frames 0 Invalid frames, too small

1059255 64 byte frames 0 Valid frames, too small

104685963 127 byte frames

6885273 255 byte frames 0 Too old frames

7803365 511 byte frames 0 Valid oversize frames

2278629 1023 byte frames 0 System FCS error frames

21695573 1518 byte frames 0 RxPortFifoFull drop frame

9353777 Too large frames

0 Good (1 coll) frames

0 Good (>1 coll) frames

Leo Laohoo · ‎05-06-2013

Undersized frames????? That's not good.

Can you elaborate what is connected to this port?