Re: High CPU Interrupt Rate on 7206vxr - Page 2

Nidal Safadi · ‎01-18-2014

Hello Everybody;

I had a very stange problem we suffer from serveral months, searching all internet sites disccusions with no luck to get a solution, here are the details:

I'm a service provider which provide ADSL service, we have 5 LNSs "BRAS" to terminat PPP users via LAC of Upper service provider.

Each LNS type is Cisco 7206vxr-NPE-G2 ( c7200p-adventerprisek9-mz.151-4.M1.bin ) handle 1800 PPP sessions and also each LNS connected to 13 LAC via vpdn over l2tp tunnels, and each LNS handle (400 - 500 Mbps) of traffic which make total of 2.5 Gbps to Internet Router.

LNS configuration not include any additional configuration else it runs OSPF to collect connected routes and send it to Internet Router.

The problem is we suffer from High cpu utlization of Interrupts. Each LNS have CPU rate of : 90/70.

Please could any one give me any advice to solve this or it's a normal thing?

Note: ip cef enabled

i can provide you any show results

Here some of show results:

LNS#show cef drop

% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'

IPv4 CEF Drop Statistics

Slot Encap_fail Unresolved Unsupported No_route No_adj ChkSum_Err

RP 0 0 2562691697 90 0 12160

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show cef not-cef-switched

% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'

IPv4 CEF Packets passed on to next switching layer

Slot No_adj No_encap Unsupp'ted Redirect Receive Options Access Frag

RP 0 0 2562929756 0 68566096 0 126945 0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show interfaces switching

GigabitEthernet0/1 " Interface face upper service provider "

Throttle count 0

Drops RP 176 SP 0

SPD Flushes Fast 471823 SSE 0

SPD Aggress Fast 0

SPD Priority Inputs 1924470 Drops 0

Protocol IP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 718302296 4126662098 1112298943 1451433415

Cache misses 0 - - -

Fast 3305818606 1149552345 661100592 1370347133

Auton/SSE 0 0 0 0

Protocol DEC MOP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 0 0 1738 133826

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Protocol ARP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 4780 286800 1526 91560

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Protocol CDP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 17399 8247126 19362 7163940

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Protocol Other

Switching path Pkts In Chars In Pkts Out Chars Out

Process 0 0 104308 6258480

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

GigabitEthernet0/2 " Inerface connected to Internet Router "

Throttle count 0

Drops RP 959104 SP 0

SPD Flushes Fast 15007215 SSE 0

SPD Aggress Fast 0

SPD Priority Inputs 1364797 Drops 0

Protocol IP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 999401649 1612699701 712502786 2118640235

Cache misses 0 - - -

Fast 1083143063 4057627019 3036792652 3002157150

Auton/SSE 0 0 0 0

Protocol DEC MOP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 0 0 1738 133826

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Protocol ARP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 75 4500 1254 75240

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Protocol CDP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 19355 8245225 19354 7160980

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Protocol Other

Switching path Pkts In Chars In Pkts Out Chars Out

Process 0 0 104314 6258840

Cache misses 0 - - -

Fast 0 0 0 0

Auton/SSE 0 0 0 0

Some of Virtual-Access Users result:

Virtual-Access10

Throttle count 0

Drops RP 0 SP 0

SPD Flushes Fast 0 SSE 0

SPD Aggress Fast 0

SPD Priority Inputs 25451 Drops 0

Protocol IP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 385798 16849653 457847 635790618

Cache misses 0 - - -

Fast 681084 63626366 26634169 1192416118

Auton/SSE 0 0 0 0

Protocol Other

Switching path Pkts In Chars In Pkts Out Chars Out

Process 3 96 86610 1386434

Cache misses 0 - - -

Fast 19576850 2329991433 0 0

Auton/SSE 0 0 0 0

Virtual-Access12

Throttle count 0

Drops RP 0 SP 0

SPD Flushes Fast 0 SSE 0

SPD Aggress Fast 0

SPD Priority Inputs 8543 Drops 0

Protocol IP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 15848 8018299 21948 2215706

Cache misses 0 - - -

Fast 310092 46529741 22224211 1403568072

Auton/SSE 0 0 0 0

Protocol Other

Switching path Pkts In Chars In Pkts Out Chars Out

Process 4 128 85123 1362688

Cache misses 0 - - -

Fast 17113093 2030440478 0 0

Auton/SSE 0 0 0 0

Virtual-Access19

Throttle count 0

Drops RP 0 SP 0

SPD Flushes Fast 0 SSE 0

SPD Aggress Fast 0

SPD Priority Inputs 8123 Drops 0

Protocol IP

Switching path Pkts In Chars In Pkts Out Chars Out

Process 1637093 139266284 2423448 3162258987

Cache misses 0 - - -

Fast 0 0 28885932 2671057232

Auton/SSE 0 0 0 0

Protocol Other

Switching path Pkts In Chars In Pkts Out Chars Out

Process 0 0 85303 1365572

Cache misses 0 - - -

Fast 20771820 2246068842 0 0

Auton/SSE 0 0 0 0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show ip cef switching statistics

Path Reason Drop Punt Punt2Host

RP LES Packet destined for us 0 35893991 0

RP LES TTL expired 0 0 170

RP LES Fragmentation failed, DF 389 0 0

RP LES Features 9615849 0 0

RP LES Unclassified reason 9 997284686 0

RP LES Neighbor resolution req 0 1 0

RP LES Fragmentation no pak 0 0 3441

RP LES Total 9616247 1033178678 3611

RP PAS No route 90 0 1

RP PAS Packet destined for us 2919 32696467 216

RP PAS Incomplete adjacency 21772 0 0

RP PAS Bad checksum 12161 0 0

RP PAS TTL expired 0 0 11118798

RP PAS Bad IP packet length 37 0 0

RP PAS Features 964811261 557882479 131388

RP PAS Unclassified reason 294249 997284686 0

RP PAS Neighbor resolution req 1 0 0

RP PAS Total 965142490 1587863632 11250403

All Total 974758737 2621042310 11254014

2 seconds after another show command

LNS#show ip cef switching statistics

Path Reason Drop Punt Punt2Host

RP LES Packet destined for us 0 35894734 0

RP LES TTL expired 0 0 170

RP LES Fragmentation failed, DF 389 0 0

RP LES Features 9615935 0 0

RP LES Unclassified reason 9 997291274 0

RP LES Neighbor resolution req 0 1 0

RP LES Fragmentation no pak 0 0 3442

RP LES Total 9616333 1033186009 3612

RP PAS No route 90 0 1

RP PAS Packet destined for us 2919 32696508 216

RP PAS Incomplete adjacency 21772 0 0

RP PAS Bad checksum 12161 0 0

RP PAS TTL expired 0 0 11118890

RP PAS Bad IP packet length 37 0 0

RP PAS Features 964819941 557889345 131389

RP PAS Unclassified reason 294250 997291274 0

RP PAS Neighbor resolution req 1 0 0

RP PAS Total 965151171 1587877127 11250496

All Total 974767504 2621063136 11254108

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show ip cef switching statistics feature

IPv4 CEF input features:

Path Feature Drop Consume Punt Punt2Host Gave route

RP LES uRPF 17 0 0 0 0

RP LES CAR 74919 0 0 0 0

RP PAS Virtual Fragment 54857 6440840 0 4470 0

RP PAS Access List 117919 0 0 126642 0

RP PAS uRPF 247018 0 0 0 0

RP PAS CAR 11870139 0 0 0 0

Total 12364869 6440840 0 131112 0

IPv4 CEF output features:

Path Feature Drop Consume Punt Punt2Host New i/f

RP LES CAR 9541552 0 0 0 0

RP PAS Access List 415655 0 0 303 0

RP PAS TCP Adjust MSS 0 0 557939001 0 0

RP PAS CAR 952181909 0 0 0 0

Total 962139116 0 557939001 303 0

IPv4 CEF post-encap features:

Path Feature Drop Consume Punt Punt2Host New i/f

Total 0 0 0 0 0

IPv4 CEF for us features:

Path Feature Drop Consume Punt Punt2Host New i/f

Total 0 0 0 0 0

IPv4 CEF punt features:

Path Feature Drop Consume Punt Punt2Host New i/f

Total 0 0 0 0 0

IPv4 CEF local features:

Path Feature Drop Consume Punt Punt2Host Gave route

Total 0 0 0 0 0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show ip traffic

IP statistics:

Rcvd: 1731030398 total, 29323905 local destination

7 format errors, 230 checksum errors, 9755726 bad hop count

0 unknown protocol, 0 not a gateway

0 security failures, 0 bad options, 9238659 with options

Opts: 0 end, 0 nop, 0 basic security, 0 loose source route

0 timestamp, 0 extended security, 0 record route

0 stream ID, 0 strict source route, 9238659 alert, 0 cipso, 0 ump

0 other

Frags: 9717546 reassembled, 6040 timeouts, 0 couldn't reassemble

443666552 fragmented, 887330011 fragments, 778 couldn't fragment

Bcast: 90024 received, 0 sent

Mcast: 1362936 received, 685736 sent

Sent: 154123399 generated, 1231263328 forwarded

Drop: 913 encapsulation failed, 0 unresolved, 0 no adjacency

0 no route, 250302 unicast RPF, 0 forced drop

0 options denied

Drop: 0 packets with source IP address zero

Drop: 0 packets with internal loop back IP address

0 physical broadcast

Reinj: 0 in input feature path, 0 in output feature path

ICMP statistics:

Rcvd: 0 format errors, 147 checksum errors, 0 redirects, 22 unreachable

104596 echo, 2 echo reply, 0 mask requests, 0 mask replies, 0 quench

0 parameter, 0 timestamp, 0 timestamp replies, 0 info request, 0 other

0 irdp solicitations, 0 irdp advertisements

10662 time exceeded, 0 info replies

Sent: 0 redirects, 132461 unreachable, 0 echo, 104596 echo reply

0 mask requests, 0 mask replies, 0 quench, 0 timestamp, 0 timestamp replies

0 info reply, 188205 time exceeded, 0 parameter problem

0 irdp solicitations, 0 irdp advertisements

TCP statistics:

Rcvd: 25409 total, 3 checksum errors, 59 no port

Sent: 24035 total

BGP statistics:

Rcvd: 0 total, 0 opens, 0 notifications, 0 updates

0 keepalives, 0 route-refresh, 0 unrecognized

Sent: 0 total, 0 opens, 0 notifications, 0 updates

0 keepalives, 0 route-refresh

EIGRP-IPv4 statistics:

Rcvd: 0 total

Sent: 0 total

PIMv2 statistics: Sent/Received

Total: 0/0, 0 checksum errors, 0 format errors

Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, Hellos: 0/0

Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0

Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0

Queue drops: 0

State-Refresh: 0/0

IGMP statistics: Sent/Received

Total: 0/0, Format errors: 0/0, Checksum errors: 0/0

Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0

DVMRP: 0/0, PIM: 0/0

Queue drops: 0

UDP statistics:

Rcvd: 27819278 total, 0 checksum errors, 83282 no port

Sent: 152988495 total, 0 forwarded broadcasts

OSPF statistics:

Last clearing of OSPF traffic counters never

Rcvd: 1363785 total, 0 checksum errors

110059 hello, 172 database desc, 1 link state req

1113254 link state updates, 140294 link state acks

Sent: 685982 total

110015 hello, 173 database desc, 52 link state req

285043 link state updates, 290701 link state acks

ARP statistics:

Rcvd: 30911 requests, 365 replies, 0 reverse, 0 other

Sent: 370 requests, 34367 replies (0 proxy), 0 reverse

Drop due to input queue full: 0

443666552 fragmented, 887330011 fragments and reassembled Increases Hevaily in IP Statistics

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

interface Virtual-Template1

ip unnumbered GigabitEthernet0/1

ip access-group denysomething out

no ip redirects

no ip unreachables

no ip proxy-arp

ip mtu 1492

ip virtual-reassembly in

ip verify unicast reverse-path

ip tcp adjust-mss 1436

no logging event link-status

peer default ip address pool ADSL-POOL

no ipv6 redirects

ppp authentication pap vpdn

ppp authorization vpdn

ppp accounting vpdn

ppp ipcp dns xx.xx.xx.xx xx.xx.xx.xx

ppp ipcp wins reject

end

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

Thanks !! waiting for your experience to solve with me this problem.

Regards

Dr.X · ‎01-24-2014

Plz post relative commands relate to routing on each of lns and 2nd router
The static routes & ospf
& sh ip protocols

Sent from Cisco Technical Support iPhone App

Dr.X · ‎01-26-2014

any news ?

plz post

sh run | i ip route

at the edgeR router

regards