06-29-2024 07:43 AM
Hello
So I have DSL 40/5 Service and for the past 2 years I have had no issues. As of this past 3 weeks I have these insane lag spikes. It could stay normal for 30 minutes or 5 hours, then it happens. My 30ms goes to 2000-3000 ms. It either will not go back down, or will after a while. I have removed every device except the ISRC1111 and the PC plugged into it, same thing. Currently my DSL Modem is in Bridge mode and ISR is doing PPPoE.
To test it, I eliminated everything Cisco and simply used my DSL Modem in PPPoE and 2 days went by 0 issues and my continuous ping stayed 30-60 ms. With that said, when using my ISR C1111 as PPPoE and the lag starts, if I reboot the ISR it all goes back to normal...for 30 mins maybe or several hours.
I really do not know how to troubleshoot this, especially when all in same exact config has been fine. Also, even doing the traceroute and ping ON the ISR, same madness.
What all would be needed from me to help in helping me figure it out?
Solved! Go to Solution.
07-02-2024 08:14 AM
Alright, so to answer this, no, I have not installed anything at all like that on any host or any patch or software. I know it’s never the answer that makes sense, but truly I have not changed anything. Which is why the first week or so it as back and forth with my ISP thinking maybe they had dslam issues etc. I’ll disable what you both mentioned and we can go that route.
07-02-2024 09:35 AM
Ok, but don't overlook the IOS upgrade from 17.13.01a to 17.14.01a. When did that take place?
07-02-2024 09:54 AM
Truly could not tell specifics, but it was because of the 2000-3000 ms latency that I upgraded to begin with. I would never have even thought about upgraded had I not had an issue. Cause I know “newer” just means new bugs .
06-29-2024 07:00 PM
What firmware is the router on?
07-02-2024 11:12 AM
Darn. Still doing it.
07-04-2024 10:11 AM
Downgraded again, still Same issue. I mean at this stage it either has to be 1. Defective ISR. 2. Being attacked at random (like 20 times a day) or 3. something in my config.
I can not wrap my head around this and for the life of me I can not figure it out but can anyone refresh my mind or assist me in this;
I want to “unplug” everything out of my ISR and make GE 0/1/7 have a network of let’s say 192.168.20.0 and use the WAN IP (dialer 1?) so I can eliminate everything and see if just the ISR has issues. I’ve done NAT’s different ACL’s etc. I can’t believe it’s so simple but I can’t figure it out!, based on my running-config showed earlier
07-04-2024 10:33 AM
do
show cpu history
check if there is peak in history
MHM
07-04-2024 10:46 AM - edited 07-04-2024 10:49 AM
show processes cpu history
1 11111
100
90
80
70
60
50
40
30
20
10
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per second (last 60 seconds)
2
111 1 111 11 111111117 11 1111 2 1 111 113 1 111 11
100
90
80
70
60
50
40
30 *
20 *
10 #
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
2 47 2 7
734735222452
100
90
80 *
70 * *
60 * *
50 * *
40 ** *
30 * ** * *
20 * ** * *
10 * ** * **
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
07-04-2024 11:06 AM
That answer you
Cpu is up to 80% make latency' and I suggest before disable snmp did you do that ?
MHM
07-04-2024 11:08 AM
I did yes, I did it the moment you mentioned it.
07-04-2024 11:06 AM
This just happened like 10 minutes ago 1000ms lag for like 5 minutes.
111 11 1 211 111 111 1 111 11 111111117 11 1111 2 1
100
90
80
70
60
50
40
30 *
20 *
10 #
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
07-04-2024 11:13 AM
Do
Show ip traffic
Twice
See which count is increasing rapidly'
I think it ARP to make also sure check show arp see how many arp incomplete in table
MHM
07-04-2024 11:18 AM
Well currently system "running fine" but here are both.
show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 172.16.1.1 - 00ea.bded.c4f4 ARPA Vlan8
Internet 172.16.1.2 48 d0e0.42d9.6e48 ARPA Vlan8
Internet 192.168.10.1 - 00ea.bded.c4f4 ARPA Vlan10
Internet 192.168.10.179 27 5254.00f0.439e ARPA Vlan10
Internet 192.168.20.1 - 00ea.bded.c4f4 ARPA Vlan1
show ip traffic
IP statistics:
Rcvd: 21185157 total, 39955 local destination
0 format errors, 0 checksum errors, 1 bad hop count
0 unknown protocol, 0 not a gateway
0 security failures, 0 bad options, 0 with options
Opts: 0 end, 0 nop, 0 basic security, 0 loose source route
0 timestamp, 0 extended security, 0 record route
0 stream ID, 0 strict source route, 0 alert, 0 cipso, 0 ump
0 other, 0 ignored
Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble
95 fragmented, 190 fragments, 37 couldn't fragment
0 invalid hole
Bcast: 0 received, 0 sent
Mcast: 108 received, 0 sent
Sent: 21122 generated, 14693943 forwarded
Drop: 0 encapsulation failed, 0 unresolved, 1453 no adjacency
0 no route, 0 unicast RPF, 0 forced drop, 0 unsupported-addr
0 options denied, 0 source IP address zero
ICMP statistics:
Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 3 unreachable
29014 echo, 0 echo reply, 0 mask requests, 0 mask replies, 0 quench
0 parameter, 0 timestamp, 0 timestamp replies, 0 info request, 0 other
0 irdp solicitations, 86 irdp advertisements
0 time exceeded, 0 info replies
Sent: 0 redirects, 6437 unreachable, 0 echo, 29014 echo reply
0 mask requests, 0 mask replies, 0 quench, 0 timestamp, 0 timestamp repl ies
0 info reply, 1 time exceeded, 0 parameter problem
0 irdp solicitations, 0 irdp advertisements
UDP statistics:
Rcvd: 21831 total, 0 checksum errors, 21831 no port 0 finput
Sent: 0 total, 0 forwarded broadcasts
TCP statistics:
Rcvd: 18032 total, 1 checksum errors, 3642 no port
Sent: 14718 total
PIMv2 statistics: Sent/Received
Total: 0/0, 0 checksum errors, 0 format errors
Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, Hellos: 0/0
Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0
Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
Queue drops: 0
State-Refresh: 0/0
IGMP statistics: Sent/Received
Total: 0/0, Format errors: 0/0, Checksum errors: 0/0
Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0
DVMRP: 0/0, PIM: 0/0
Queue drops: 0
Report VRF mismatch drops: 0
EIGRP-IPv4 statistics:
Rcvd: 0 total
Sent: 0 total
NHRP statistics:
Sent: Total 0
0 Resolution Request 0 Resolution Reply 0 Registration Request
0 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
Rcvd: Total 0
0 Resolution Request 0 Resolution Reply 0 Registration Request
0 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
OSPF statistics:
Last clearing of OSPF traffic counters never
Rcvd: 0 total, 0 checksum errors
0 hello, 0 database desc, 0 link state req
0 link state updates, 0 link state acks
Sent: 0 total
0 hello, 0 database desc, 0 link state req
0 link state updates, 0 link state acks
BGP statistics:
Rcvd: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh, 0 unrecognized
Sent: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh
ARP statistics:
Rcvd: 153 requests, 5 replies, 0 reverse, 0 other
Sent: 6 requests, 171 replies (0 proxy), 0 reverse
Drop due to input queue full: 0
HoM#show ip traffic
IP statistics:
Rcvd: 21197205 total, 39985 local destination
0 format errors, 0 checksum errors, 1 bad hop count
0 unknown protocol, 0 not a gateway
0 security failures, 0 bad options, 0 with options
Opts: 0 end, 0 nop, 0 basic security, 0 loose source route
0 timestamp, 0 extended security, 0 record route
0 stream ID, 0 strict source route, 0 alert, 0 cipso, 0 ump
0 other, 0 ignored
Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble
95 fragmented, 190 fragments, 37 couldn't fragment
0 invalid hole
Bcast: 0 received, 0 sent
Mcast: 108 received, 0 sent
Sent: 21154 generated, 14699671 forwarded
Drop: 0 encapsulation failed, 0 unresolved, 1453 no adjacency
0 no route, 0 unicast RPF, 0 forced drop, 0 unsupported-addr
0 options denied, 0 source IP address zero
ICMP statistics:
Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 3 unreachable
29014 echo, 0 echo reply, 0 mask requests, 0 mask replies, 0 quench
0 parameter, 0 timestamp, 0 timestamp replies, 0 info request, 0 other
0 irdp solicitations, 86 irdp advertisements
0 time exceeded, 0 info replies
Sent: 0 redirects, 6437 unreachable, 0 echo, 29014 echo reply
0 mask requests, 0 mask replies, 0 quench, 0 timestamp, 0 timestamp replies
0 info reply, 1 time exceeded, 0 parameter problem
0 irdp solicitations, 0 irdp advertisements
UDP statistics:
Rcvd: 21831 total, 0 checksum errors, 21831 no port 0 finput
Sent: 0 total, 0 forwarded broadcasts
TCP statistics:
Rcvd: 18062 total, 1 checksum errors, 3642 no port
Sent: 14750 total
PIMv2 statistics: Sent/Received
Total: 0/0, 0 checksum errors, 0 format errors
Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, Hellos: 0/0
Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0
Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
Queue drops: 0
State-Refresh: 0/0
IGMP statistics: Sent/Received
Total: 0/0, Format errors: 0/0, Checksum errors: 0/0
Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0
DVMRP: 0/0, PIM: 0/0
Queue drops: 0
Report VRF mismatch drops: 0
EIGRP-IPv4 statistics:
Rcvd: 0 total
Sent: 0 total
NHRP statistics:
Sent: Total 0
0 Resolution Request 0 Resolution Reply 0 Registration Request
0 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
Rcvd: Total 0
0 Resolution Request 0 Resolution Reply 0 Registration Request
0 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
OSPF statistics:
Last clearing of OSPF traffic counters never
Rcvd: 0 total, 0 checksum errors
0 hello, 0 database desc, 0 link state req
0 link state updates, 0 link state acks
Sent: 0 total
0 hello, 0 database desc, 0 link state req
0 link state updates, 0 link state acks
BGP statistics:
Rcvd: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh, 0 unrecognized
Sent: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh
ARP statistics:
Rcvd: 153 requests, 5 replies, 0 reverse, 0 other
Sent: 6 requests, 171 replies (0 proxy), 0 reverse
Drop due to input queue full: 0
07-04-2024 11:27 AM
ARP is OK
there is ICMP count and UDP conut
for UDP count
check
show ip nat statistics <<- share it here
for ICMP there is high number of ICMP toward router ? are you run any IP SLA or EEM ?
MHM
07-04-2024 11:29 AM
In truth I do not know what IP SLA or EEM is, so I would have to say no. Only "server" type things I am running are several email servers.
Total active translations: 124 (9 static, 115 dynamic; 124 extended)
Outside interfaces:
Dialer1, Virtual-Access2
Inside interfaces:
Vlan1, Vlan8, Vlan10
Hits: 25561608 Misses: 37922
Expired translations: 37807
Dynamic mappings:
-- Inside Source
[Id: 1] access-list 1 interface Dialer1 refcount 0
[Id: 2] access-list 4 pool 179 refcount 2
pool 179: id 3, netmask 255.255.255.252
start 207.108.121.179 end 207.108.121.179
type generic, total addresses 1, allocated 1 (100%), misses 0
[Id: 3] access-list 5 pool 178 refcount 0
pool 178: id 2, netmask 255.255.255.252
start 207.108.121.178 end 207.108.121.178
type generic, total addresses 1, allocated 0 (0%), misses 0
[Id: 4] access-list 6 pool 182 refcount 102
pool 182: id 4, netmask 255.255.255.252
start 207.108.121.182 end 207.108.121.182
type generic, total addresses 1, allocated 1 (100%), misses 0
[Id: 5] access-list 7 pool 177 refcount 2
pool 177: id 1, netmask 255.255.255.252
start 207.108.121.177 end 207.108.121.177
type generic, total addresses 1, allocated 1 (100%), misses 0
[Id: 6] access-list 8 pool 181 refcount 4
pool 181: id 5, netmask 255.255.255.252
start 207.108.121.181 end 207.108.121.181
type generic, total addresses 1, allocated 1 (100%), misses 0
[Id: 7] access-list 9 pool 180 refcount 0
pool 180: id 6, netmask 255.255.255.252
start 207.108.121.180 end 207.108.121.180
type generic, total addresses 1, allocated 0 (0%), misses 0
[Id: 8] access-list DMZ-TO-OUTSIDE pool DMZ_OUT refcount 0
pool DMZ_OUT: id 7, netmask 255.255.255.252
start 207.108.121.179 end 207.108.121.179
type generic, total addresses 1, allocated 0 (0%), misses 0
nat-limit statistics:
max entry: max allowed 200000, used 115, missed 0
In-to-out drops: 2058 Out-to-in drops: 1046
Pool stats drop: 0 Mapping stats drop: 0
Port block alloc fail: 0
IP alias add fail: 0
Limit entry add fail: 0
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide