cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5588
Views
10
Helpful
6
Replies
Highlighted

IP CEF - ARP cache issue. Help!!

We have a Cisco 881 router, which is crashing.  We have seen that the ARP cache fills up so much it causes things to crash, our phones go down..  We dont know why this however IP CEF seems to be doing it, when we disable it goes away however disabling IP CEF causes our L2TP tunnel to become inoperable also.  So why does IP CEF cause thousands of AR entries and how can we limit that!?  Below is the error, sample of the ARP cache and our config.  You will notice we also have a /31 given to us on WAN interface, this was given to us by our service provider.  Any help gratefully appreciated, this is really strange I cant find other examples on internet.

The error:

Nov  1 04:21:57.474: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x81F083F4, alignment 16

Pool: Processor  Free: 55176  Cause: Not enough free memory

Alternate Pool: I/O  Free: 2352  Cause: Not enough free memory

-Process= "IP ARP Adjacency", ipl= 4, pid= 93,  -Traceback= 0x81ECDDBCz 0x81EEFEACz 0x81F083F8z 0x80EF09C0z 0x80EF2A98z 0x80EF30A8z 0x80EEBAD0z 0x80EB20E0z 0x80E77ABCz 0x80EB5D58z 0x80E6D590z 0x80E6D728z 0x80EFA8DCz 0x80EFC4A8z 0x80EFC7FCz 0x80F4B6F4z

Nov  1 04:21:57.474: %COMMON_FIB-3-NOMEM: Memory allocation failure for validating prefix in IPv4 CEF [0x80E5C570] (fatal) (1 subsequent failure).

Nov  1 04:21:57.474: %COMMON_FIB-4-DISABLING: IPv4 CEF is being disabled due to a fatal error.

----

ARP Cache grows and grows! (this is only a small part of it)


Internet  92.244.102.35           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  93.106.58.124           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  93.152.2.184            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  93.186.25.33            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  94.30.30.45             0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  94.65.135.105           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  94.229.169.157          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.26.58.189            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.71.47.220            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.145.233.189          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.157.171.219          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.158.166.29           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.158.240.98           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.175.133.195          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.36.236           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.36.251           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.83.81            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.112.74           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.112.89           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.115.217          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.120.2            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.120.8            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.177.124.190          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  95.182.12.183           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  96.47.67.105            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  96.55.244.191           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  98.190.6.100            0   0007.7dd9.f802  ARPA   FastEthernet4

Protocol  Address          Age (min)  Hardware Addr   Type   Interface

Internet  98.234.32.235           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  98.252.226.26           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  99.38.10.203            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  99.109.224.219          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  107.20.249.243          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  108.20.186.227          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  108.86.142.112          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.91.172.101          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.124.49.200          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.16.245          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.16.252          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.18.221          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.19.17           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.19.45           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.19.201          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.19.211          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.204.85.47           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  109.205.186.36          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  110.77.234.108          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  110.168.77.87           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  110.247.149.121         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  112.205.49.228          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  112.210.175.127         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  113.193.168.17          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  114.42.221.29           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  115.128.5.228           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  115.184.122.151         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  115.232.185.22          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  117.203.146.240         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  117.254.242.187         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  120.60.38.252           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  121.45.140.38           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  121.131.138.101         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  122.102.190.84          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  122.108.13.150          0   0007.7dd9.f802  ARPA   FastEthernet4

Protocol  Address          Age (min)  Hardware Addr   Type   Interface

Internet  122.172.28.101          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  122.175.137.98          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  122.176.254.29          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  122.177.29.83           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  123.21.184.150          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  123.238.66.114          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  124.43.23.194           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  124.82.35.40            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  124.168.77.84           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  124.208.84.162          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  124.244.155.215         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  126.107.20.125          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  128.211.201.213         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  129.6.15.28             0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  129.242.185.231         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  130.37.34.52            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  130.83.203.72           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  130.159.196.118         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  141.2.212.16            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  142.68.86.23            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  149.3.130.112           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  149.13.32.15            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  151.25.14.212           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  151.29.245.230          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  169.254.105.207         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  174.7.191.135           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  174.36.215.20           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  174.37.211.68           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  174.101.224.116         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  177.16.236.144          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.16.128.74           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.35.158.192          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.36.99.58            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.74.253.234          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.125.39.47           0   0007.7dd9.f802  ARPA   FastEthernet4

Protocol  Address          Age (min)  Hardware Addr   Type   Interface

Internet  178.125.101.227         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.165.47.48           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.176.19.60           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.187.145.12          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.239.102.36          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  178.239.102.37          -   0007.7d99.043c  ARPA   FastEthernet4

Internet  178.252.106.192         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  182.93.214.99           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  183.107.161.196         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  183.182.90.82           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  183.185.112.81          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  184.17.83.33            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  186.205.70.102          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.2.249.152           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.25.156.11           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.27.215.132          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.52.196.71           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.54.38.56            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.65.176.153          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.66.72.107           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.187.35.39           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  188.249.145.161         0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  189.81.57.140           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  190.191.98.97           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  190.245.158.24          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  192.168.0.1             0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  192.168.10.5            -   0007.7d99.0438  ARPA   Vlan10

Internet  192.168.100.10          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  192.168.237.1           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  192.168.254.1           -   0007.7d99.0438  ARPA   Vlan2

Internet  192.168.254.2          28   64d9.893b.61c1  ARPA   Vlan2

Internet  192.168.254.3          28   d0c2.829e.af41  ARPA   Vlan2

Protocol  Address          Age (min)  Hardware Addr   Type   Interface

Internet  192.168.254.111         0   0016.76df.8e42  ARPA   Vlan2

Internet  192.168.254.240         0   0015.17c0.b3b4  ARPA   Vlan2

Internet  192.175.48.1            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  194.165.188.82          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  195.74.128.6            0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  195.74.130.12           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  196.41.255.44           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.7.51.72             0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.216.144          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.216.146          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.216.147          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.216.148          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.216.149          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.217.144          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.217.146          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.217.149          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.217.172          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  199.47.217.173          0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  201.1.196.233           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  202.83.41.183           0   0007.7dd9.f802  ARPA   FastEthernet4

Internet  204.9.163.158           0   0007.7dd9.f802  ARPA   FastEthernet4

----

Our config

#show run

Building configuration...

Current configuration : 6521 bytes

!

! Last configuration change at 08:51:40 GMT Tue Nov 1 2011 by admin

!

version 15.0

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

no service password-encryption

!

!

boot-start-marker

boot-end-marker

!

logging buffered 51200 warnings

!

no aaa new-model

!

monitor session 1 source interface Fa3

monitor session 1 destination interface Fa1

memory-size iomem 10

clock timezone GMT 0

clock summer-time BST date Mar 27 2011 1:00 Oct 30 2011 2:00

!

ip source-route

!

!

ip dhcp excluded-address 192.168.254.1 192.168.254.99

!

!

ip cef

no ip domain lookup

ip domain name yourdomain.com

ip name-server 8.8.8.8

no ipv6 cef

l2tp-class l2tpclass1

!

!

!

vpdn enable

!

license udi pid CISCO881-K9 sn FCZ152692N2

!

!

vtp mode client

!

!

pseudowire-class communicator

encapsulation l2tpv2

protocol l2tpv2 l2tpclass1

ip local interface FastEthernet4

!

!

interface FastEthernet0

switchport access vlan 4

!

interface FastEthernet1

description Phones

switchport access vlan 3

!

interface FastEthernet2

switchport access vlan 4

!

interface FastEthernet3

description Trunk to switch

switchport mode trunk

!

interface FastEthernet4

description WAN

ip address x.x.x.x 255.255.255.254

ip nat outside

ip virtual-reassembly

duplex auto

speed auto

no cdp enable

!

interface Virtual-PPP1

ip unnumbered Vlan3

ppp chap hostname x.x.x.x

ppp chap password 0 xxxxxxx

ppp direction callout

no cdp enable

pseudowire x.x.x.x 10 pw-class x

!

interface Vlan1

description $ETH-SW-LAUNCH$$INTF-INFO-HWIC 4ESW$

no ip address

ip tcp adjust-mss 1452

shutdown

!

interface Vlan2

ip address 192.168.254.1 255.255.255.0

ip nat inside

ip virtual-reassembly

!

interface Vlan3

ip address 10.60.97.129 255.255.255.192

ip helper-address 10.50.32.32

ip tcp adjust-mss 1452

!

interface Vlan4

ip address x.x.x.x 255.255.255.248

!

!

ip forward-protocol nd

no ip http server

ip http access-class 23

ip http authentication local

no ip http secure-server

ip http timeout-policy idle 60 life 86400 requests 10000

!

ip dns server

ip nat inside source list 15 interface FastEthernet4 overload

ip route 0.0.0.0 0.0.0.0 FastEthernet4

ip route 10.0.0.0 255.0.0.0 Virtual-PPP1

ip route x.x.x.x 255.255.255.248 Vlan4

!

logging 192.168.254.111

access-list 15 permit any

access-list 23 permit x.x.x.x

Many thanks!

Nicholas

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
VIP Expert

Disclaimer

The     Author of this posting offers the information contained within this     posting without consideration and with the reader's understanding  that    there's no implied or expressed suitability or fitness for any   purpose.   Information provided is for informational purposes only and   should not   be construed as rendering professional advice of any kind.   Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In     no event shall Author be liable for any damages whatsoever   (including,   without limitation, damages for loss of use, data or   profit) arising  out  of the use or inability to use the posting's   information even if  Author  has been advised of the possibility of  such  damage.

Posting

ip route 0.0.0.0 0.0.0.0 FastEthernet4

change to:

ip route 0.0.0.0 0.0.0.0 x.x.x.x !next hop IP

View solution in original post

6 REPLIES 6
Highlighted
VIP Expert

Disclaimer

The     Author of this posting offers the information contained within this     posting without consideration and with the reader's understanding  that    there's no implied or expressed suitability or fitness for any   purpose.   Information provided is for informational purposes only and   should not   be construed as rendering professional advice of any kind.   Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In     no event shall Author be liable for any damages whatsoever   (including,   without limitation, damages for loss of use, data or   profit) arising  out  of the use or inability to use the posting's   information even if  Author  has been advised of the possibility of  such  damage.

Posting

ip route 0.0.0.0 0.0.0.0 FastEthernet4

change to:

ip route 0.0.0.0 0.0.0.0 x.x.x.x !next hop IP

View solution in original post

Highlighted

couldnt see the wood for the trees!

many thanks

Nicholas

Highlighted

Hello,

If I have 2 web servers in the network with NAT.

Who is in my case next hop ip ? The ip from WAN (GigabitEthernet0/1) ?

----------------

ip dhcp pool pool1

import all

network 192.168.1.0 255.255.255.0

dns-server x.x.x.1 x.x.x.2

default-router 192.168.1.1

!

interface GigabitEthernet0/0

description $ETH-LAN$$FW_INSIDE$

ip address 192.168.1.1 255.255.255.0

no ip redirects

no ip unreachables

no ip proxy-arp

ip nat inside

ip virtual-reassembly in

zone-member security in-zone

duplex auto

speed auto

no cdp enable

no mop enabled

!

interface GigabitEthernet0/1

description $ETH-WAN$$FW_OUTSIDE$

ip address 88.x.x.157 255.255.255.128

  no ip redirects

no ip unreachables

no ip proxy-arp

ip verify unicast reverse-path

ip nat outside

ip ips sdm_ips_rule in

ip virtual-reassembly in

zone-member security out-zone

duplex auto

speed auto

no cdp enable

no mop enabled

!

ip forward-protocol nd

!

ip http server

ip http access-class 2

ip http authentication local

ip http secure-server

!

ip nat inside source list 1 interface GigabitEthernet0/1 overload

ip nat inside source static 192.168.1.88 34.x.x.178

ip nat inside source static 192.168.1.89 34.x.x.179

ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/1

-----------------

How can I do this ?

Highlighted

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

It would be the IP address of the device on the "otherside" of G0/1.  Sh ARP or a traceroute should show it.

Highlighted

Hello,

Thank''s for your answer

So GigabitEthernet0/1 is the Wan and has an ip address 88.x.x.157 255.255.255.128 directly connected to internet provider.

So my commads should be :

no ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/1

ip route 0.0.0.0 0.0.0.0 88.x.x.157

Will work this with nat ?


Highlighted

Disclaimer

The   Author of this posting offers the information contained within this   posting without consideration and with the reader's understanding that   there's no implied or expressed suitability or fitness for any purpose.   Information provided is for informational purposes only and should not   be construed as rendering professional advice of any kind. Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In   no event shall Author be liable for any damages whatsoever (including,   without limitation, damages for loss of use, data or profit) arising  out  of the use or inability to use the posting's information even if  Author  has been advised of the possibility of such damage.

Posting

Yes to the first and believe also yes to the second.

Content for Community-Ad