WS-C3560X-24 Packet drops and High latency issues

Anil M · ‎01-17-2024

Hello,

We have a Cisco L3 WS-C3560X-24 switch configured as a gateway for our internal network. For last one week, we are seeing high latenct issues to the internal hosts and public hosts. (When I checked the logs, there are continuous messages on flapping between 2 interfaces), I have verified for any network loops, there is nothing,

Disconnected all the switches, and connected only L3, Sophos, and a wifi access point(Cisco 2900 mobility express). still the latency issue persists it goes to <100ms from 3ms >, ping drops after 20-30 requests and comes back normal.

I am unable to figure out wether the switch should be replaced or configurations has to be changed.
I have attached the tech-support and interfaces output files as well. Please suggest what should be done to resolve the issues.

IOS version: c3560e-ipbasek9-mz.150-2.SE
- This L3 is acting as a core router, we have Sophos XGS4300 at the perimeter level.

The CPU usage is high, But there are no high-consuming processes, please refer below

L3switch12#show processes cpu sorted

CPU utilization for five seconds: 14%/2%; one minute: 14%; five minutes: 14%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

83 211568 41538 5093 2.08% 1.92% 1.83% 0 RedEarth Tx Mana

12 114563 198143 578 1.43% 1.01% 1.01% 0 ARP Input

171 116799 9518 12271 1.27% 0.71% 0.72% 0 HL3U bkgrd proce

53 6214 180 34522 0.63% 0.07% 0.02% 0 Per-minute Jobs

82 43759 56857 769 0.47% 0.38% 0.42% 0 RedEarth I2C dri

232 6885 50393 136 0.31% 0.13% 0.08% 0 Spanning Tree

139 796 125 6368 0.31% 0.06% 0.11% 1 SSH Process

89 1608 4228 380 0.15% 0.02% 0.00% 0 Adjust Regions

214 16544 41366 399 0.15% 0.20% 0.19% 0 IP Input

42 1493 14213 105 0.15% 0.02% 0.00% 0 Net Background

346 981 331026 2 0.15% 0.01% 0.00% 0 MMON MENG

180 17534 2138 8201 0.15% 0.15% 0.15% 0 HQM Stack Proces

4 184671 8196 22531 0.00% 2.03% 1.64% 0 Check heaps

11 0 1 0 0.00% 0.00% 0.00% 0 Image License br

10 24088 178 135325 0.00% 0.18% 0.17% 0 Licensing Auto U

13 156 11179 13 0.00% 0.00% 0.00% 0 ARP Background

14 0 1 0 0.00% 0.00% 0.00% 0 CEF MIB API

15 0 9 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT

19 0 2132 0 0.00% 0.00% 0.00% 0 IPC Event Notifi

16 0 1 0 0.00% 0.00% 0.00% 0 Policy Manager

21 0 179 0 0.00% 0.00% 0.00% 0 IPC Dynamic Cach

22 0 1 0 0.00% 0.00% 0.00% 0 IPC Session Serv

17 17 7 2428 0.00% 0.00% 0.00% 0 Entity MIB API

8 0 210 0 0.00% 0.00% 0.00% 0 WATCH_AFS

18 0 1 0 0.00% 0.00% 0.00% 0 IFS Agent Manage

20 16 10365 1 0.00% 0.00% 0.00% 0 IPC Mcast Pendin

27 95 2677 35 0.00% 0.00% 0.00% 0 IPC Seat Manager

28 0 611 0 0.00% 0.00% 0.00% 0 IPC Check Queue

6 0 1 0 0.00% 0.00% 0.00% 0 DiscardQ Backgro

24 0 10365 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim

23 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager

32 34 2132 15 0.00% 0.00% 0.00% 0 IPC Loadometer

33 33 4 8250 0.00% 0.00% 0.00% 0 PrstVbl

34 0 1 0 0.00% 0.00% 0.00% 0 Crash writer

35 8 1 8000 0.00% 0.00% 0.00% 0 Exception contro

36 17 10617 1 0.00% 0.00% 0.00% 0 GraphIt

9 0 1 0 0.00% 0.00% 0.00% 0 License Client N

38 0 2 0 0.00% 0.00% 0.00% 0 XML Proxy Client

39 0 1 0 0.00% 0.00% 0.00% 0 ARP Snoop

40 66 10594 6 0.00% 0.00% 0.00% 0 Dynamic ARP Insp

41 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd

7 0 2 0 0.00% 0.00% 0.00% 0 Timers

25 8 10365 0 0.00% 0.00% 0.00% 0 IPC Deferred Por

44 0 211 0 0.00% 0.00% 0.00% 0 Logger

26 0 1 0 0.00% 0.00% 0.00% 0 IPC Process leve

46 0 3 0 0.00% 0.00% 0.00% 0 CTS CORE

Appreciate the help. Thanks.

Leo Laohoo · ‎01-17-2024

@Anil M wrote:
I have some chatgpt suggestions based on the CPU history and show interfaces command outputs.

Anil M · ‎01-19-2024

can anyone help, please?

Joseph W. Doherty · ‎01-20-2024

Suggestion - in the future, ask for help and provide information, especially as requested, w/o including Chat/gpt analysis. My guess, most not much interest in debugging it.

BTW, I'm not slightly Chat/gpt, but similar situation if you posted Larry says "x" or I "know" the cause is "x". I.e. general such is just an impediment to analysis.

balaji.bandi · ‎01-19-2024

We have a Cisco L3 WS-C3560X-24 switch configured as a gateway for our internal network. For last one week, we are seeing high latenct issues to the internal hosts and public hosts.

1. what IOS code running?

2. post - show version and show run

3. Can you post example high latency output here ?

4. how is your ping responce from PC to switch gateway ?

5. how is your network diagram looks like ?

6. have you checked the interface connected to ISP side any packet drops ?

7. what kind of bandwidth you have, what % it was used.

8 you have attached the output information - you have not mentioned how these are connected in the network (post diagram)

example - we see output packet drops here on below - after reload they not there .

GigabitEthernet0/1

GigabitEthernet0/5

GigabitEthernet0/19

GigabitEthernet0/23

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Leo Laohoo · ‎01-20-2024

@Anil M wrote:
For last one week, we are seeing high latenct issues to the internal hosts and public hosts.

If ChatGPT can spat out useless drivel based on one sentence alone, then "good luck".