Solved: Re: Help 2811 router doesn't have enough power to handle Interne - Page 2

tranminhc · ‎05-22-2013

Dear forum,

I has just setup a 2811 router to do only Internet NAT feature, but it seem to be exhausted. Few days ago the it's CPU always above 50%, and this morning it's completely 99 100%. I feel upset when console to it, so lag.

Here is some command output:

RTR-TED-0002#

RTR-TED-0002 04:08:30 AM Thursday May 23 2013 UTC

11 11

999999999999999999999009999990099999999999999999999999999999

100 ************************************************************

90 ************************************************************

80 ************************************************************

70 ************************************************************

60 ************************************************************

50 ************************************************************

40 ************************************************************

30 ************************************************************

20 ************************************************************

10 ************************************************************

0....5....1....1....2....2....3....3....4....4....5....5....6

0 5 0 5 0 5 0 5 0 5 0

CPU% per second (last 60 seconds)

11111

000009999999998888788787866766676654476666766644656766777768

000006169977413199970721046186575588703263055648085430031064

100 #####* *##**

90 #####**######* ** *

80 #####**######*#*##***** * * *

70 #####*#######*#########*# ******** * * **** ** *****#

60 #####*####################*##**###* ******#** ***##*######

50 ###################################**######### *############

40 ############################################################

30 ############################################################

20 ############################################################

10 ############################################################

0....5....1....1....2....2....3....3....4....4....5....5....6

0 5 0 5 0 5 0 5 0 5 0

CPU% per minute (last 60 minutes)

* = maximum CPU% # = average CPU%

11 1

86993379999344900699999098845989575344677999999 3

4997354899934690019999809757892280429196872999939

100 ** **** *** ******* * * ****

90 ** **** *** ********* * * ******

80 * ** **** *** ********* *** ********

70 **** ***** *** ********* #** * *****##**

60 **#* *#*** ****######*** *#**** *****##**

50 #*#* *#*** *****#######****#***** * ****####*

40 ###* **##*# ******#########**#*#*** **#***####* *

30 ###***###*#***##*##########*##*##*****##*###### *

20 ####**#####**##############*##*##*****######### *

10 ############################################### *

0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..

0 5 0 5 0 5 0 5 0 5 0 5 0

CPU% per hour (last 72 hours)

* = maximum CPU% # = average CPU%

RTR-TED-0002#show int fa0/1

FastEthernet0/1 is up, line protocol is up

Hardware is MV96340 Ethernet, address is 0015.fa2f.d539 (bia 0015.fa2f.d539)

Internet address is 10.124.1.5/16

MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,

reliability 255/255, txload 57/255, rxload 4/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 17:40:35

Input queue: 0/75/195/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 1650000 bits/sec, 1658 packets/sec

5 minute output rate 22564000 bits/sec, 2205 packets/sec

48946016 packets input, 1173184643 bytes

Received 208721 broadcasts, 0 runts, 0 giants, 20 throttles

44779 input errors, 0 CRC, 0 frame, 0 overrun, 44779 ignored

0 watchdog

0 input packets with dribble condition detected

72907133 packets output, 3637260615 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 unknown protocol drops

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

RTR-TED-0002#show int fa0/0

FastEthernet0/0 is up, line protocol is up

Hardware is MV96340 Ethernet, address is 0015.fa2f.d538 (bia 0015.fa2f.d538)

MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,

reliability 255/255, txload 4/255, rxload 58/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 1d22h, output 00:00:00, output hang never

Last clearing of "show interface" counters 17:40:20

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 22753000 bits/sec, 2218 packets/sec

5 minute output rate 1700000 bits/sec, 1647 packets/sec

73338917 packets input, 4276920895 bytes

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

122824 input errors, 0 CRC, 0 frame, 0 overrun, 122824 ignored

0 watchdog

0 input packets with dribble condition detected

48756504 packets output, 1340293905 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 unknown protocol drops

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

RTR-TED-0002#show int dialer 1

Dialer1 is up, line protocol is up (spoofing)

Hardware is Unknown

Internet address is 113.162.120.22/32

MTU 1500 bytes, BW 100000 Kbit/sec, DLY 20000 usec,

reliability 255/255, txload 6/255, rxload 144/255

Encapsulation PPP, loopback not set

Keepalive set (10 sec)

DTR is pulsed for 1 seconds on reset

Interface is bound to Vi2

Last input never, output never, output hang never

Last clearing of "show interface" counters 2d01h

Input queue: 76/75/71454/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: weighted fair

Output queue: 0/1000/64/0 (size/max total/threshold/drops)

Conversations 0/0/256 (active/max active/max total)

Reserved Conversations 0/0 (allocated/max allocated)

Available Bandwidth 75000 kilobits/sec

5 minute input rate 56848000 bits/sec, 5388 packets/sec

5 minute output rate 2363000 bits/sec, 3583 packets/sec

242770462 packets input, 491800724 bytes

160293318 packets output, 1190208362 bytes

Bound to:

Virtual-Access2 is up, line protocol is up

Hardware is Virtual Access interface

MTU 1500 bytes, BW 100000 Kbit/sec, DLY 20000 usec,

reliability 255/255, txload 6/255, rxload 145/255

Encapsulation PPP, LCP Open

Listen: CDPCP

Open: IPCP

PPPoE vaccess, cloned from Dialer1

Vaccess status 0x44, loopback not set

Keepalive set (10 sec)

DTR is pulsed for 5 seconds on reset

Interface is bound to Di1 (Encapsulation PPP)

Last input 00:00:00, output never, output hang never

Last clearing of "show interface" counters 1d23h

Input queue: 0/75/120991/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 56880000 bits/sec, 5391 packets/sec

5 minute output rate 2425000 bits/sec, 3723 packets/sec

242677192 packets input, 345776708 bytes, 7687 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

160316349 packets output, 1192328063 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 unknown protocol drops

0 output buffer failures, 0 output buffers swapped out

0 carrier transitions

RTR-TED-0002#show ip traffic

IP statistics:

Rcvd: 4273236 total, 2463884 local destination

0 format errors, 0 checksum errors, 63 bad hop count

2 unknown protocol, 0 not a gateway

0 security failures, 0 bad options, 0 with options

Opts: 0 end, 0 nop, 0 basic security, 0 loose source route

0 timestamp, 0 extended security, 0 record route

0 stream ID, 0 strict source route, 0 alert, 0 cipso, 0 ump

0 other

Frags: 7 reassembled, 0 timeouts, 0 couldn't reassemble

35186 fragmented, 70375 fragments, 21334 couldn't fragment

Bcast: 143 received, 0 sent

Mcast: 0 received, 0 sent

Sent: 998921 generated, 419708376 forwarded

Drop: 961 encapsulation failed, 0 unresolved, 0 no adjacency

1155 no route, 0 unicast RPF, 0 forced drop

0 options denied

Drop: 0 packets with source IP address zero

Drop: 0 packets with internal loop back IP address

0 physical broadcast

ICMP statistics:

Rcvd: 87 format errors, 1 checksum errors, 0 redirects, 51566 unreachable

522 echo, 0 echo reply, 0 mask requests, 0 mask replies, 6 quench

0 parameter, 0 timestamp, 0 timestamp replies, 0 info request, 0 other

0 irdp solicitations, 0 irdp advertisements

364 time exceeded, 0 info replies

Sent: 245 redirects, 271507 unreachable, 0 echo, 522 echo reply

0 mask requests, 0 mask replies, 0 quench, 0 timestamp, 0 timestamp replies

0 info reply, 63 time exceeded, 0 parameter problem

0 irdp solicitations, 0 irdp advertisements

TCP statistics:

Rcvd: 777897 total, 73 checksum errors, 31327 no port

Sent: 726588 total

UDP statistics:

Rcvd: 1633366 total, 13 checksum errors, 1633353 no port

Sent: 0 total, 0 forwarded broadcasts

BGP statistics:

Rcvd: 0 total, 0 opens, 0 notifications, 0 updates

0 keepalives, 0 route-refresh, 0 unrecognized

Sent: 0 total, 0 opens, 0 notifications, 0 updates

0 keepalives, 0 route-refresh

IP-EIGRP statistics:

Rcvd: 0 total

Sent: 0 total

PIMv2 statistics: Sent/Received

Total: 0/0, 0 checksum errors, 0 format errors

Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, Hellos: 0/0

Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0

Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0

Queue drops: 0

State-Refresh: 0/0

IGMP statistics: Sent/Received

Total: 0/0, Format errors: 0/0, Checksum errors: 0/0

Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0

DVMRP: 0/0, PIM: 0/0

Queue drops: 0

OSPF statistics:

Rcvd: 0 total, 0 checksum errors

0 hello, 0 database desc, 0 link state req

0 link state updates, 0 link state acks

Sent: 0 total

0 hello, 0 database desc, 0 link state req

0 link state updates, 0 link state acks

ARP statistics:

Rcvd: 701648 requests, 2 replies, 0 reverse, 0 other

Sent: 745584 requests, 62 replies (0 proxy), 0 reverse

Drop due to input queue full: 0

RTR-TED-0002#show ip nat statistics

Total active translations: 0 (0 static, 0 dynamic; 0 extended)

Peak translations: 97586, occurred 01:28:59 ago

Outside interfaces:

Dialer1

Inside interfaces:

FastEthernet0/1

Hits: 421422215 Misses: 0

CEF Translated packets: 418688607, CEF Punted packets: 1882968

Expired translations: 7728354

Dynamic mappings:

-- Inside Source

[Id: 1] access-list 1 interface Dialer1 refcount 4294967259

Appl doors: 0

Normal doors: 0

Queued Packets: 0

*May 21 06:32:12.858: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:32:46.030: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:33:16.854: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:33:48.906: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:34:20.334: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:37:05.106: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:57:52.338: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 21 06:58:56.450: %IP_VFR-4-FRAG_TABLE_OVERFLOW: FastEthernet0/1: the fragment table has reached its maximum threshold 16

*May 22 05:22:21.497: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Dialer1: the fragment table has reached its maximum threshold 16

*May 22 05:22:55.809: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Dialer1: the fragment table has reached its maximum threshold 16

*May 22 05:30:06.637: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Dialer1: the fragment table has reached its maximum threshold 16

*May 22 08:35:17.740: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Dialer1: the fragment table has reached its maximum threshold 16

I highlighted some info which I thinks they are clue for my problem. It seem that there are too much traffic download from Internet, and the router cannot handle.

The logging was for 2 days ago, today it has no logs buffered. Three days ago I has upgrade to OS c2800nm-advipservicesk9-mz.124-22.T5 to improve %CPU but it does not work.

Feel free to not consider the fragmented counter. Because the counter indicate the time I customized the MTU and MSS. Now the MTU is default without configuring and the MSS is 1436 for interface Dialer 1.

This company has about over 600 users. I would like to know can Router 281 handle the NAT job with such number of users? If not, which router model is good to work in this case.

Any response would be appreciated.

Thanks

antonio.guirado · ‎05-23-2013

Hello,

I agree Paolo to update IOS but I remember you that 97000 sessions for 600 users are 160 sessions per user. A browser can open 4 TCP connections simultaneously. I have bad experiences with 80K sessions in a NPE-G1 c7200 router. Please, after upgrade if the problem still exists consider tune nat timers. A higher number only can be explained by a TCP NAT timer of 24 hours. If it is not enough study you internal network because your users can be very aggressive generating TCP connections. As last alternative, consider change the PPPoE connection to your checkpoint.

Regards.

tranminhc · ‎05-23-2013

I decided to choose IOS c2800nm-advipservicesk9-mz.124-25g.bin. I will consider tune the timer after a test time.

I will replug the router into network at tomorrow lunch time, then report the result one day after that.

Anyway, thanks you to all quick response.

paolo bevilacqua · ‎05-24-2013

That is not a good choice at all. Use latest IOS.

tranminhc · ‎06-16-2013

Sorry, because of not updating the case for long time.

There were some other problems between me and the customer then they delayed the process of this case.

Today I start it again. I will use the OS: c2800nm-advsecurityk9-mz.151-4.M6.bin

Thanks

Help 2811 router doesn't have enough power to handle Internet NAT???