- Cisco Community
- Technology and Support
- Networking
- Routing
- Memory fragmentation
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Memory fragmentation
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2010 06:24 AM - edited 03-04-2019 10:31 AM
Hi All,
I am getting the below error logs frequently on my one of the 1841 router.
*Nov 19 14:07:10.716: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:07:40.713: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:08:10.735: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:08:40.732: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:09:10.750: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:09:40.756: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:09:43.931: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: admin] [Source: 192.168.100.49] [localport: 22] at 16:09:43 utc Fri Nov 19 2010
*Nov 19 14:10:10.757: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:10:40.755: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
*Nov 19 14:11:10.752: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34 0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
Please find the below are the output for further investigation:
#sh proc mem sorted allocated
Processor Pool Total: 190999824 Used: 29224932 Free: 161774892
I/O Pool Total: 29360128 Used: 29306688 Free: 53440
PID TTY Allocated Freed Holding Getbufs Retbufs Process
137 0 2859094816 2858457440 132920 0 0 Crypto IKMP
185 0 273269292 345209228 12972 0 0 SNMP ENGINE
183 0 214553896 214541688 23956 0 0 IP SNMP
55 0 212616600 11734776 57884 4069173346 6358576 IP Input
1 0 196088524 199566416 616196 0 0 Chunk Manager
5 0 137055296 314045036 29859228 180904525 4241561880 Pool Manager
184 0 71938712 0 13016 0 0 PDU DISPATCHER
0 0 55626520 58264372 187324 360576 72720 *Dead*
0 0 38547796 10079028 23698708 0 0 *Init*
172 0 29748140 3050132 129964 22147524 0 Per-minute Jobs
169 0 28756268 28837024 9468 0 0 crypto sw pk pro
193 0 5725424 5630808 100664 0 0 IP-EIGRP: PDM
167 0 4169760 4167420 15312 0 0 Syslog
94 0 2612888 14590056 14500 0 0 TCP Protocols
33 0 1793756 1315644 9972 397460 0 Per-Second Jobs
39 0 656064 1272 636792 0 0 USB Startup
177 0 652412 822852 8064 0 0 NHRP
138 0 587768 548348 220636 0 0 IPSEC key engine
17 0 256876 0 266848 113400 0 EEM ED Syslog
PID TTY Allocated Freed Holding Getbufs Retbufs Process
134 0 236956 120368 76004 0 0 Crypto ACL
175 194 163440 124472 51528 0 0 SSH Process
190 0 149156 140224 73108 0 0 RADIUS
96 0 136580 420 37028 0 0 DHCPD Receive
127 0 127040 428 135584 0 0 Crypto WUI
79 0 113468 0 73328 0 0 CEF process
10 0 97160 1093848 10024 18720 0 ARP Input
51 0 95332 8928 6972 0 0 AAA Server
176 0 94664 0 11444 0 0 SSH Event handle
76 0 66144 0 76116 0 0 IP RIB Update
29 0 58352 296 23128 0 0 Net Background
150 0 28188 277032 6972 0 0 AAA SEND STOP EV
141 0 24912 0 37884 0 0 Crypto Delete Ma
128 0 24196 24196 6972 0 0 Crypto Support
26 0 12584 7124 18252 0 0 Crypto SS Proces
0 0 12052 2341456 12052 0 0 *Sched*
160 0 11056 1960 16068 0 0 EEM Server
166 0 9956 856 19072 0 0 EEM Policy Direc
164 0 9436 1784 14468 0 0 VLAN Manager
59 0 5392 196 6972 2268 0 MOP Protocols
194 0 4108 0 7024 0 0 IP-EIGRP: HELLO
4 0 3348 4148 10124 0 0 Check heaps
PID TTY Allocated Freed Holding Getbufs Retbufs Process
3 0 2480 18184 12988 0 0 Tunnel Security
48 0 2444 196 9220 0 0 DTP Protocol
123 0 2400 1432 6972 0 0 LOCAL AAA
16 0 1592 0 8564 0 0 Entity MIB API
131 0 1372 596 13748 0 0 Crypto CA
117 0 1136 196 7912 0 0 URL filter proc
186 0 1072 196 13904 0 0 IP SNMPV6
124 0 956 196 7732 0 0 TPLUS
102 0 940 0 10912 0 0 HTTP CORE
54 0 788 196 7564 0 0 CDP Protocol
30 0 584 312 13244 0 0 IDB Work
81 0 492 156 7464 0 0 L2MM
191 0 488 196 7316 0 0 NTP
60 0 392 392 12972 0 0 PPP Hooks
118 0 392 392 6972 0 0 Crypto HW Proc
115 0 368 196 7144 0 0 SDEE Management
129 0 284 0 25256 0 0 RPMS_PROC_MAIN
49 0 196 196 6972 0 0 dot1x
47 0 196 196 6972 0 0 PI MATM Aging Pr
57 0 196 196 6972 0 0 TurboACL
58 0 196 196 6972 0 0 TurboACL chunk
32 0 196 2548 6972 0 0 TTY Background
PID TTY Allocated Freed Holding Getbufs Retbufs Process
46 0 196 196 6972 0 0 Call Management
42 0 196 196 6972 0 0 SM Monitor
38 0 196 196 3972 0 0 Eswilp Storm Con
50 0 196 196 6972 0 0 AAA Dictionary R
37 0 196 196 6972 0 0 ESWPPM
122 0 196 196 6972 0 0 LINE AAA
178 0 196 196 6972 0 0 IP Flow Backgrou
11 0 196 196 6972 0 0 ATM Idle Timer
171 0 196 196 6972 0 0 Compute load avg
69 0 196 196 12972 0 0 L2TP mgmt daemon
12 0 196 196 6972 0 0 AAA high-capacit
71 0 196 196 9972 0 0 EAPoUDP Process
72 0 196 196 9972 0 0 IP Host Track Pr
165 0 196 0 7168 0 0 DHCPD Database
74 0 196 196 12972 0 0 KRB5 AAA
15 0 196 196 6972 0 0 DDR Timers
31 0 196 196 12972 0 0 Logger
77 0 196 196 12972 0 0 PPP IP Route
78 0 196 196 12972 0 0 PPP IPCP
162 0 196 196 6972 0 0 RMON Deferred Se
161 0 196 196 6972 0 0 RMON Recycle Pro
19 0 196 196 6972 0 0 Serial Backgroun
PID TTY Allocated Freed Holding Getbufs Retbufs Process
21 0 196 196 6972 0 0 SMART
22 0 196 196 6972 0 0 GraphIt
2 0 196 196 3972 0 0 Load Meter
85 0 196 196 6972 0 0 RLM groups Proce
23 0 196 196 12972 0 0 Dialer event
87 0 196 196 6972 0 0 SCTP Main Proces
6 0 196 196 6972 0 0 Timers
89 0 196 196 6972 0 0 RUDPV1 Main Proc
125 0 196 196 6972 0 0 VSP_MGR
179 0 196 196 6972 0 0 IP NAT Ager
8 0 196 196 6972 0 0 Environmental mo
174 0 196 0 25168 0 0 tHUB
119 0 196 196 6972 0 0 ENABLE AAA
108 0 196 196 6972 0 0 RBSCP Background
25 0 196 196 12972 0 0 XML Proxy Client
97 0 196 196 6972 0 0 Dialer Forwarder
107 0 196 196 6972 0 0 PPP SSS
99 0 196 196 9972 0 0 Adj Manager
100 0 196 196 12972 0 0 ATM OAM Input
101 0 196 196 12972 0 0 ATM OAM TIMER
106 0 196 196 6972 0 0 PPP Bind
105 0 196 196 12972 0 0 X.25 Background
PID TTY Allocated Freed Holding Getbufs Retbufs Process
173 0 164 0 7136 0 0 CEF Scanner
149 0 164 0 7136 0 0 DATA Collector
148 0 164 0 7136 0 0 DATA Transfer Pr
75 0 76 0 10048 0 0 IP Background
98 0 0 1652176 6972 0 0 IP Cache Ager
62 0 0 0 12972 0 0 SSS Manager
103 0 0 0 6972 0 0 RARP Input
111 0 0 0 6972 0 0 DHCPD Timer
109 0 0 0 12972 0 0 L2F management d
104 0 0 0 6972 0 0 PAD InCall
114 0 0 0 12972 0 0 IPS Timer
110 0 0 0 6972 0 0 Inspect Timer
112 0 0 0 6972 0 0 Authentication P
56 0 0 0 6972 0 0 ICMP event handl
34 0 0 0 6972 0 0 AggMgr Process
113 0 0 0 6972 0 0 Auth-proxy AAA B
28 0 0 0 6972 0 0 Critical Bkgnd
93 0 0 46056 12972 0 0 TCP Timer
61 0 0 0 6972 0 0 CRYPTO IKMP IPC
91 0 0 0 6972 0 0 bsm_xmt_proc
52 0 0 0 6972 0 0 AAA ACCT Proc
90 0 0 0 6972 0 0 bsm_timers
PID TTY Allocated Freed Holding Getbufs Retbufs Process
116 0 0 0 6972 0 0 IPv6 Inspect Tim
27 0 0 0 6972 0 0 Inode Table Dest
86 0 0 0 6972 0 0 SNMP Timers
84 0 0 0 6972 0 0 IP Traceroute
130 0 0 0 6972 0 0 crypto engine pr
45 0 0 0 6972 0 0 Platform Compact
132 0 0 0 8972 0 0 Crypto PKI-CRL
133 0 0 0 8972 0 0 Crypto SSL
92 0 0 0 9972 0 0 CES Client SVC R
135 0 0 0 6972 0 0 CRYPTO QoS proce
136 0 0 0 6972 0 0 Crypto INT
82 0 0 0 6972 0 0 MRD
20 0 0 0 6972 0 0 RO Notify Timers
126 0 0 0 12972 0 0 encrypt proc
140 0 0 0 6972 0 0 Crypto PAS Proc
63 0 0 0 12972 0 0 SSS Test Client
142 0 0 0 8972 0 0 Key Proc
143 0 0 0 9972 0 0 Crypto Device Up
144 0 0 0 6972 0 0 Crypto Hardware
145 0 0 0 6972 0 0 Multi-ISA Event
146 0 0 0 6972 0 0 Multi-ISA Cleanu
147 0 0 0 6972 0 0 PM Callback
PID TTY Allocated Freed Holding Getbufs Retbufs Process
88 0 0 0 6972 0 0 IUA Main Process
83 0 0 0 6972 0 0 IGMPSN
18 0 0 0 6972 0 0 HC Counter Timer
151 0 0 0 9972 0 0 EEM ED CLI
152 0 0 0 9972 0 0 EEM ED Counter
153 0 0 0 9972 0 0 EEM ED Interface
154 0 0 0 9972 0 0 EEM ED IOSWD
155 0 0 0 9972 0 0 EEM ED Memory-th
156 0 0 0 9972 0 0 EEM ED None
157 0 0 0 9972 0 0 EM ED OIR
158 0 0 0 9972 0 0 EEM ED SNMP
159 0 0 0 9972 0 0 EEM ED Timer
44 0 0 0 3972 0 0 IGMP Snooping Re
80 0 0 0 6972 0 0 Socket Timers
64 0 0 0 6972 0 0 SSS Feature Mana
163 0 0 0 6972 0 0 Syslog Traps
139 0 0 0 6972 0 0 IPSEC manual key
73 0 0 0 6972 0 0 IPv6 RIB Redistr
14 0 0 0 12972 0 0 Policy Manager
13 0 0 0 6972 0 0 AAA_SERVER_DEADT
168 0 0 0 6972 0 0 VPDN Scal
70 0 0 0 6972 0 0 X.25 Encaps Mana
PID TTY Allocated Freed Holding Getbufs Retbufs Process
170 0 0 0 6972 0 0 Net Input
68 0 0 0 12972 0 0 L2X SSS manager
67 0 0 0 13032 0 0 L2X Socket proce
120 0 0 0 6972 0 0 EM Background Pr
121 0 0 0 6972 0 0 Key chain liveke
9 0 0 0 24972 0 0 Crash writer
66 0 0 0 12972 0 0 VPDN call manage
7 0 0 0 12972 0 0 OIR Handler
65 0 0 0 6972 0 0 SSS Feature Time
53 0 0 0 6972 0 0 ACCT Periodic Pr
180 0 0 0 7028 0 0 IP NAT WLAN
181 0 0 0 6972 0 0 IP VFR proc
24 0 0 0 6972 0 0 SERIAL A'detect
43 0 0 0 3972 0 0 IGMP Snooping Pr
41 0 0 0 3972 0 0 dev_device_remov
40 0 0 0 3972 0 0 dev_device_inser
187 0 0 0 13028 0 0 SNMP ConfCopyPro
188 0 0 0 12972 0 0 SNMP Traps
189 0 0 0 6972 0 0 NAT MIB Helper
35 0 0 0 6972 0 0 Token Daemon
36 0 0 0 6972 0 0 Transport Port A
192 0 0 0 18100 0 0 IP-EIGRP Router
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 0 0 396944 0 0 *MallocLite*
95 0 0 0 24972 0 0 COPS
58520772 Total
#sh proc mem sort getbufs
Processor Pool Total: 190999824 Used: 29224932 Free: 161774892
I/O Pool Total: 29360128 Used: 29306688 Free: 53440
PID TTY Allocated Freed Holding Getbufs Retbufs Process
55 0 212616600 11734776 57884 4069173346 6358576 IP Input
5 0 139553696 316543436 29859228 180904525 4241561880 Pool Manager
172 0 29751740 3053732 129964 22147524 0 Per-minute Jobs
33 0 1793756 1315644 9972 397460 0 Per-Second Jobs
0 0 55626520 58264372 187324 360576 72720 *Dead*
17 0 256876 0 266848 113400 0 EEM ED Syslog
10 0 97160 1093848 10024 18720 0 ARP Input
59 0 5392 196 6972 2268 0 MOP Protocols
3 0 2480 18184 12988 0 0 Tunnel Security
2 0 196 196 3972 0 0 Load Meter
1 0 196088524 199566416 616196 0 0 Chunk Manager
8 0 196 196 6972 0 0 Environmental mo
0 0 0 0 396944 0 0 *MallocLite*
0 0 12052 2341456 12052 0 0 *Sched*
4 0 3348 4148 10124 0 0 Check heaps
6 0 196 196 6972 0 0 Timers
13 0 0 0 6972 0 0 AAA_SERVER_DEADT
14 0 0 0 12972 0 0 Policy Manager
0 0 38547796 10079028 23698708 0 0 *Init*
PID TTY Allocated Freed Holding Getbufs Retbufs Process
16 0 1592 0 8564 0 0 Entity MIB API
7 0 0 0 12972 0 0 OIR Handler
9 0 0 0 24972 0 0 Crash writer
19 0 196 196 6972 0 0 Serial Backgroun
20 0 0 0 6972 0 0 RO Notify Timers
11 0 196 196 6972 0 0 ATM Idle Timer
22 0 196 196 6972 0 0 GraphIt
23 0 196 196 12972 0 0 Dialer event
24 0 0 0 6972 0 0 SERIAL A'detect
25 0 196 196 12972 0 0 XML Proxy Client
12 0 196 196 6972 0 0 AAA high-capacit
27 0 0 0 6972 0 0 Inode Table Dest
15 0 196 196 6972 0 0 DDR Timers
29 0 58352 296 23128 0 0 Net Background
30 0 584 312 13244 0 0 IDB Work
31 0 196 196 12972 0 0 Logger
32 0 196 2548 6972 0 0 TTY Background
18 0 0 0 6972 0 0 HC Counter Timer
34 0 0 0 6972 0 0 AggMgr Process
35 0 0 0 6972 0 0 Token Daemon
36 0 0 0 6972 0 0 Transport Port A
37 0 196 196 6972 0 0 ESWPPM
PID TTY Allocated Freed Holding Getbufs Retbufs Process
38 0 196 196 3972 0 0 Eswilp Storm Con
39 0 656064 1272 636792 0 0 USB Startup
21 0 196 196 6972 0 0 SMART
41 0 0 0 3972 0 0 dev_device_remov
42 0 196 196 6972 0 0 SM Monitor
43 0 0 0 3972 0 0 IGMP Snooping Pr
44 0 0 0 3972 0 0 IGMP Snooping Re
45 0 0 0 6972 0 0 Platform Compact
26 0 12584 7124 18252 0 0 Crypto SS Proces
47 0 196 196 6972 0 0 PI MATM Aging Pr
48 0 2444 196 9220 0 0 DTP Protocol
49 0 196 196 6972 0 0 dot1x
50 0 196 196 6972 0 0 AAA Dictionary R
51 0 95332 8928 6972 0 0 AAA Server
52 0 0 0 6972 0 0 AAA ACCT Proc
53 0 0 0 6972 0 0 ACCT Periodic Pr
54 0 788 196 7564 0 0 CDP Protocol
28 0 0 0 6972 0 0 Critical Bkgnd
56 0 0 0 6972 0 0 ICMP event handl
57 0 196 196 6972 0 0 TurboACL
58 0 196 196 6972 0 0 TurboACL chunk
40 0 0 0 3972 0 0 dev_device_inser
PID TTY Allocated Freed Holding Getbufs Retbufs Process
60 0 392 392 12972 0 0 PPP Hooks
61 0 0 0 6972 0 0 CRYPTO IKMP IPC
62 0 0 0 12972 0 0 SSS Manager
63 0 0 0 12972 0 0 SSS Test Client
64 0 0 0 6972 0 0 SSS Feature Mana
65 0 0 0 6972 0 0 SSS Feature Time
66 0 0 0 12972 0 0 VPDN call manage
67 0 0 0 13032 0 0 L2X Socket proce
68 0 0 0 12972 0 0 L2X SSS manager
69 0 196 196 12972 0 0 L2TP mgmt daemon
70 0 0 0 6972 0 0 X.25 Encaps Mana
71 0 196 196 9972 0 0 EAPoUDP Process
72 0 196 196 9972 0 0 IP Host Track Pr
73 0 0 0 6972 0 0 IPv6 RIB Redistr
74 0 196 196 12972 0 0 KRB5 AAA
75 0 76 0 10048 0 0 IP Background
76 0 66144 0 76116 0 0 IP RIB Update
77 0 196 196 12972 0 0 PPP IP Route
78 0 196 196 12972 0 0 PPP IPCP
79 0 113468 0 73328 0 0 CEF process
80 0 0 0 6972 0 0 Socket Timers
81 0 492 156 7464 0 0 L2MM
PID TTY Allocated Freed Holding Getbufs Retbufs Process
82 0 0 0 6972 0 0 MRD
83 0 0 0 6972 0 0 IGMPSN
46 0 196 196 6972 0 0 Call Management
85 0 196 196 6972 0 0 RLM groups Proce
86 0 0 0 6972 0 0 SNMP Timers
87 0 196 196 6972 0 0 SCTP Main Proces
88 0 0 0 6972 0 0 IUA Main Process
89 0 196 196 6972 0 0 RUDPV1 Main Proc
90 0 0 0 6972 0 0 bsm_timers
91 0 0 0 6972 0 0 bsm_xmt_proc
92 0 0 0 9972 0 0 CES Client SVC R
93 0 0 46056 12972 0 0 TCP Timer
94 0 2612888 14590056 14500 0 0 TCP Protocols
84 0 0 0 6972 0 0 IP Traceroute
96 0 136580 420 37028 0 0 DHCPD Receive
97 0 196 196 6972 0 0 Dialer Forwarder
98 0 0 1652176 6972 0 0 IP Cache Ager
99 0 196 196 9972 0 0 Adj Manager
100 0 196 196 12972 0 0 ATM OAM Input
101 0 196 196 12972 0 0 ATM OAM TIMER
102 0 940 0 10912 0 0 HTTP CORE
103 0 0 0 6972 0 0 RARP Input
PID TTY Allocated Freed Holding Getbufs Retbufs Process
104 0 0 0 6972 0 0 PAD InCall
105 0 196 196 12972 0 0 X.25 Background
106 0 196 196 6972 0 0 PPP Bind
107 0 196 196 6972 0 0 PPP SSS
108 0 196 196 6972 0 0 RBSCP Background
109 0 0 0 12972 0 0 L2F management d
110 0 0 0 6972 0 0 Inspect Timer
111 0 0 0 6972 0 0 DHCPD Timer
112 0 0 0 6972 0 0 Authentication P
113 0 0 0 6972 0 0 Auth-proxy AAA B
114 0 0 0 12972 0 0 IPS Timer
115 0 368 196 7144 0 0 SDEE Management
116 0 0 0 6972 0 0 IPv6 Inspect Tim
117 0 1136 196 7912 0 0 URL filter proc
118 0 392 392 6972 0 0 Crypto HW Proc
119 0 196 196 6972 0 0 ENABLE AAA
120 0 0 0 6972 0 0 EM Background Pr
121 0 0 0 6972 0 0 Key chain liveke
122 0 196 196 6972 0 0 LINE AAA
123 0 2400 1432 6972 0 0 LOCAL AAA
124 0 956 196 7732 0 0 TPLUS
125 0 196 196 6972 0 0 VSP_MGR
PID TTY Allocated Freed Holding Getbufs Retbufs Process
126 0 0 0 12972 0 0 encrypt proc
127 0 127040 428 135584 0 0 Crypto WUI
128 0 24196 24196 6972 0 0 Crypto Support
129 0 284 0 25256 0 0 RPMS_PROC_MAIN
130 0 0 0 6972 0 0 crypto engine pr
131 0 1372 596 13748 0 0 Crypto CA
132 0 0 0 8972 0 0 Crypto PKI-CRL
133 0 0 0 8972 0 0 Crypto SSL
134 0 236956 120368 76004 0 0 Crypto ACL
135 0 0 0 6972 0 0 CRYPTO QoS proce
136 0 0 0 6972 0 0 Crypto INT
137 0 2859170160 2858532784 132920 0 0 Crypto IKMP
138 0 587768 548348 220636 0 0 IPSEC key engine
139 0 0 0 6972 0 0 IPSEC manual key
140 0 0 0 6972 0 0 Crypto PAS Proc
141 0 24912 0 37884 0 0 Crypto Delete Ma
142 0 0 0 8972 0 0 Key Proc
143 0 0 0 9972 0 0 Crypto Device Up
144 0 0 0 6972 0 0 Crypto Hardware
145 0 0 0 6972 0 0 Multi-ISA Event
146 0 0 0 6972 0 0 Multi-ISA Cleanu
147 0 0 0 6972 0 0 PM Callback
PID TTY Allocated Freed Holding Getbufs Retbufs Process
148 0 164 0 7136 0 0 DATA Transfer Pr
149 0 164 0 7136 0 0 DATA Collector
150 0 28188 277032 6972 0 0 AAA SEND STOP EV
151 0 0 0 9972 0 0 EEM ED CLI
152 0 0 0 9972 0 0 EEM ED Counter
153 0 0 0 9972 0 0 EEM ED Interface
154 0 0 0 9972 0 0 EEM ED IOSWD
155 0 0 0 9972 0 0 EEM ED Memory-th
156 0 0 0 9972 0 0 EEM ED None
157 0 0 0 9972 0 0 EM ED OIR
158 0 0 0 9972 0 0 EEM ED SNMP
159 0 0 0 9972 0 0 EEM ED Timer
160 0 11056 1960 16068 0 0 EEM Server
161 0 196 196 6972 0 0 RMON Recycle Pro
162 0 196 196 6972 0 0 RMON Deferred Se
163 0 0 0 6972 0 0 Syslog Traps
164 0 9436 1784 14468 0 0 VLAN Manager
165 0 196 0 7168 0 0 DHCPD Database
166 0 9956 856 19072 0 0 EEM Policy Direc
167 0 4172352 4170012 15312 0 0 Syslog
168 0 0 0 6972 0 0 VPDN Scal
169 0 28756268 28837024 9468 0 0 crypto sw pk pro
PID TTY Allocated Freed Holding Getbufs Retbufs Process
170 0 0 0 6972 0 0 Net Input
171 0 196 196 6972 0 0 Compute load avg
95 0 0 0 24972 0 0 COPS
173 0 164 0 7136 0 0 CEF Scanner
174 0 196 0 25168 0 0 tHUB
175 194 199444 160476 51528 0 0 SSH Process
176 0 94664 0 11444 0 0 SSH Event handle
177 0 652412 822852 8064 0 0 NHRP
178 0 196 196 6972 0 0 IP Flow Backgrou
179 0 196 196 6972 0 0 IP NAT Ager
180 0 0 0 7028 0 0 IP NAT WLAN
181 0 0 0 6972 0 0 IP VFR proc
183 0 214563320 214551112 23956 0 0 IP SNMP
184 0 71938712 0 13016 0 0 PDU DISPATCHER
185 0 273269292 345209228 12972 0 0 SNMP ENGINE
186 0 1072 196 13904 0 0 IP SNMPV6
187 0 0 0 13028 0 0 SNMP ConfCopyPro
188 0 0 0 12972 0 0 SNMP Traps
189 0 0 0 6972 0 0 NAT MIB Helper
190 0 149156 140224 73108 0 0 RADIUS
191 0 488 196 7316 0 0 NTP
192 0 0 0 18100 0 0 IP-EIGRP Router
PID TTY Allocated Freed Holding Getbufs Retbufs Process
193 0 5725424 5630808 100664 0 0 IP-EIGRP: PDM
194 0 4108 0 7024 0 0 IP-EIGRP: HELLO
58520772 Total
#sh proc mem sort holdin
Processor Pool Total: 190999824 Used: 29222932 Free: 161776892
I/O Pool Total: 29360128 Used: 29306688 Free: 53440
PID TTY Allocated Freed Holding Getbufs Retbufs Process
5 0 140606696 317596436 29859228 180904525 4241561880 Pool Manager
0 0 38547796 10079028 23698708 0 0 *Init*
39 0 656064 1272 636792 0 0 USB Startup
1 0 196088524 199568464 614148 0 0 Chunk Manager
0 0 0 0 396944 0 0 *MallocLite*
17 0 256876 0 266848 113400 0 EEM ED Syslog
138 0 587768 548348 220636 0 0 IPSEC key engine
0 0 55626520 58264372 187324 360576 72720 *Dead*
127 0 127040 428 135584 0 0 Crypto WUI
137 0 2859207840 2858570464 132920 0 0 Crypto IKMP
172 0 29751740 3053732 129964 22147524 0 Per-minute Jobs
193 0 5725424 5630808 100664 0 0 IP-EIGRP: PDM
76 0 66144 0 76116 0 0 IP RIB Update
134 0 236956 120368 76004 0 0 Crypto ACL
79 0 113468 0 73328 0 0 CEF process
190 0 149156 140224 73108 0 0 RADIUS
55 0 212616600 11734776 57884 4069173346 6358576 IP Input
175 194 222124 183156 51528 0 0 SSH Process
141 0 24912 0 37884 0 0 Crypto Delete Ma
PID TTY Allocated Freed Holding Getbufs Retbufs Process
96 0 136580 420 37028 0 0 DHCPD Receive
129 0 284 0 25256 0 0 RPMS_PROC_MAIN
174 0 196 0 25168 0 0 tHUB
9 0 0 0 24972 0 0 Crash writer
95 0 0 0 24972 0 0 COPS
183 0 214564832 214552624 23956 0 0 IP SNMP
29 0 58352 296 23128 0 0 Net Background
166 0 9956 856 19072 0 0 EEM Policy Direc
26 0 12584 7124 18252 0 0 Crypto SS Proces
192 0 0 0 18100 0 0 IP-EIGRP Router
160 0 11056 1960 16068 0 0 EEM Server
167 0 4173216 4170876 15312 0 0 Syslog
94 0 2612888 14590056 14500 0 0 TCP Protocols
164 0 9436 1784 14468 0 0 VLAN Manager
186 0 1072 196 13904 0 0 IP SNMPV6
131 0 1372 596 13748 0 0 Crypto CA
30 0 584 312 13244 0 0 IDB Work
67 0 0 0 13032 0 0 L2X Socket proce
187 0 0 0 13028 0 0 SNMP ConfCopyPro
184 0 71938712 0 13016 0 0 PDU DISPATCHER
3 0 2480 18184 12988 0 0 Tunnel Security
23 0 196 196 12972 0 0 Dialer event
PID TTY Allocated Freed Holding Getbufs Retbufs Process
25 0 196 196 12972 0 0 XML Proxy Client
31 0 196 196 12972 0 0 Logger
185 0 273270472 345210408 12972 0 0 SNMP ENGINE
109 0 0 0 12972 0 0 L2F management d
7 0 0 0 12972 0 0 OIR Handler
126 0 0 0 12972 0 0 encrypt proc
188 0 0 0 12972 0 0 SNMP Traps
114 0 0 0 12972 0 0 IPS Timer
105 0 196 196 12972 0 0 X.25 Background
101 0 196 196 12972 0 0 ATM OAM TIMER
100 0 196 196 12972 0 0 ATM OAM Input
14 0 0 0 12972 0 0 Policy Manager
93 0 0 46056 12972 0 0 TCP Timer
78 0 196 196 12972 0 0 PPP IPCP
77 0 196 196 12972 0 0 PPP IP Route
74 0 196 196 12972 0 0 KRB5 AAA
69 0 196 196 12972 0 0 L2TP mgmt daemon
68 0 0 0 12972 0 0 L2X SSS manager
66 0 0 0 12972 0 0 VPDN call manage
63 0 0 0 12972 0 0 SSS Test Client
62 0 0 0 12972 0 0 SSS Manager
60 0 392 392 12972 0 0 PPP Hooks
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 12052 2341456 12052 0 0 *Sched*
176 0 94664 0 11444 0 0 SSH Event handle
102 0 940 0 10912 0 0 HTTP CORE
4 0 3348 4148 10124 0 0 Check heaps
75 0 76 0 10048 0 0 IP Background
10 0 97160 1093848 10024 18720 0 ARP Input
158 0 0 0 9972 0 0 EEM ED SNMP
157 0 0 0 9972 0 0 EM ED OIR
159 0 0 0 9972 0 0 EEM ED Timer
33 0 1793756 1315644 9972 397460 0 Per-Second Jobs
154 0 0 0 9972 0 0 EEM ED IOSWD
71 0 196 196 9972 0 0 EAPoUDP Process
72 0 196 196 9972 0 0 IP Host Track Pr
153 0 0 0 9972 0 0 EEM ED Interface
156 0 0 0 9972 0 0 EEM ED None
155 0 0 0 9972 0 0 EEM ED Memory-th
152 0 0 0 9972 0 0 EEM ED Counter
151 0 0 0 9972 0 0 EEM ED CLI
143 0 0 0 9972 0 0 Crypto Device Up
99 0 196 196 9972 0 0 Adj Manager
92 0 0 0 9972 0 0 CES Client SVC R
169 0 28756268 28837024 9468 0 0 crypto sw pk pro
PID TTY Allocated Freed Holding Getbufs Retbufs Process
48 0 2444 196 9220 0 0 DTP Protocol
142 0 0 0 8972 0 0 Key Proc
132 0 0 0 8972 0 0 Crypto PKI-CRL
133 0 0 0 8972 0 0 Crypto SSL
16 0 1592 0 8564 0 0 Entity MIB API
177 0 652412 822852 8064 0 0 NHRP
117 0 1136 196 7912 0 0 URL filter proc
124 0 956 196 7732 0 0 TPLUS
54 0 788 196 7564 0 0 CDP Protocol
81 0 492 156 7464 0 0 L2MM
191 0 488 196 7316 0 0 NTP
165 0 196 0 7168 0 0 DHCPD Database
115 0 368 196 7144 0 0 SDEE Management
173 0 164 0 7136 0 0 CEF Scanner
148 0 164 0 7136 0 0 DATA Transfer Pr
149 0 164 0 7136 0 0 DATA Collector
180 0 0 0 7028 0 0 IP NAT WLAN
194 0 4108 0 7024 0 0 IP-EIGRP: HELLO
49 0 196 196 6972 0 0 dot1x
15 0 196 196 6972 0 0 DDR Timers
98 0 0 1652176 6972 0 0 IP Cache Ager
37 0 196 196 6972 0 0 ESWPPM
PID TTY Allocated Freed Holding Getbufs Retbufs Process
97 0 196 196 6972 0 0 Dialer Forwarder
51 0 95332 8928 6972 0 0 AAA Server
106 0 196 196 6972 0 0 PPP Bind
36 0 0 0 6972 0 0 Transport Port A
108 0 196 196 6972 0 0 RBSCP Background
104 0 0 0 6972 0 0 PAD InCall
91 0 0 0 6972 0 0 bsm_xmt_proc
111 0 0 0 6972 0 0 DHCPD Timer
112 0 0 0 6972 0 0 Authentication P
47 0 196 196 6972 0 0 PI MATM Aging Pr
45 0 0 0 6972 0 0 Platform Compact
46 0 196 196 6972 0 0 Call Management
116 0 0 0 6972 0 0 IPv6 Inspect Tim
89 0 196 196 6972 0 0 RUDPV1 Main Proc
88 0 0 0 6972 0 0 IUA Main Process
87 0 196 196 6972 0 0 SCTP Main Proces
120 0 0 0 6972 0 0 EM Background Pr
121 0 0 0 6972 0 0 Key chain liveke
90 0 0 0 6972 0 0 bsm_timers
123 0 2400 1432 6972 0 0 LOCAL AAA
86 0 0 0 6972 0 0 SNMP Timers
125 0 196 196 6972 0 0 VSP_MGR
PID TTY Allocated Freed Holding Getbufs Retbufs Process
85 0 196 196 6972 0 0 RLM groups Proce
13 0 0 0 6972 0 0 AAA_SERVER_DEADT
128 0 24196 24196 6972 0 0 Crypto Support
12 0 196 196 6972 0 0 AAA high-capacit
130 0 0 0 6972 0 0 crypto engine pr
35 0 0 0 6972 0 0 Token Daemon
84 0 0 0 6972 0 0 IP Traceroute
83 0 0 0 6972 0 0 IGMPSN
11 0 196 196 6972 0 0 ATM Idle Timer
50 0 196 196 6972 0 0 AAA Dictionary R
136 0 0 0 6972 0 0 Crypto INT
82 0 0 0 6972 0 0 MRD
8 0 196 196 6972 0 0 Environmental mo
139 0 0 0 6972 0 0 IPSEC manual key
140 0 0 0 6972 0 0 Crypto PAS Proc
42 0 196 196 6972 0 0 SM Monitor
80 0 0 0 6972 0 0 Socket Timers
52 0 0 0 6972 0 0 AAA ACCT Proc
144 0 0 0 6972 0 0 Crypto Hardware
145 0 0 0 6972 0 0 Multi-ISA Event
146 0 0 0 6972 0 0 Multi-ISA Cleanu
147 0 0 0 6972 0 0 PM Callback
PID TTY Allocated Freed Holding Getbufs Retbufs Process
122 0 196 196 6972 0 0 LINE AAA
135 0 0 0 6972 0 0 CRYPTO QoS proce
150 0 28188 277032 6972 0 0 AAA SEND STOP EV
53 0 0 0 6972 0 0 ACCT Periodic Pr
18 0 0 0 6972 0 0 HC Counter Timer
73 0 0 0 6972 0 0 IPv6 RIB Redistr
70 0 0 0 6972 0 0 X.25 Encaps Mana
19 0 196 196 6972 0 0 Serial Backgroun
56 0 0 0 6972 0 0 ICMP event handl
20 0 0 0 6972 0 0 RO Notify Timers
57 0 196 196 6972 0 0 TurboACL
65 0 0 0 6972 0 0 SSS Feature Time
34 0 0 0 6972 0 0 AggMgr Process
161 0 196 196 6972 0 0 RMON Recycle Pro
162 0 196 196 6972 0 0 RMON Deferred Se
163 0 0 0 6972 0 0 Syslog Traps
64 0 0 0 6972 0 0 SSS Feature Mana
58 0 196 196 6972 0 0 TurboACL chunk
32 0 196 2548 6972 0 0 TTY Background
28 0 0 0 6972 0 0 Critical Bkgnd
168 0 0 0 6972 0 0 VPDN Scal
59 0 5392 196 6972 2268 0 MOP Protocols
PID TTY Allocated Freed Holding Getbufs Retbufs Process
170 0 0 0 6972 0 0 Net Input
171 0 196 196 6972 0 0 Compute load avg
6 0 196 196 6972 0 0 Timers
61 0 0 0 6972 0 0 CRYPTO IKMP IPC
118 0 392 392 6972 0 0 Crypto HW Proc
27 0 0 0 6972 0 0 Inode Table Dest
119 0 196 196 6972 0 0 ENABLE AAA
110 0 0 0 6972 0 0 Inspect Timer
178 0 196 196 6972 0 0 IP Flow Backgrou
179 0 196 196 6972 0 0 IP NAT Ager
113 0 0 0 6972 0 0 Auth-proxy AAA B
181 0 0 0 6972 0 0 IP VFR proc
24 0 0 0 6972 0 0 SERIAL A'detect
22 0 196 196 6972 0 0 GraphIt
21 0 196 196 6972 0 0 SMART
103 0 0 0 6972 0 0 RARP Input
107 0 196 196 6972 0 0 PPP SSS
189 0 0 0 6972 0 0 NAT MIB Helper
44 0 0 0 3972 0 0 IGMP Snooping Re
2 0 196 196 3972 0 0 Load Meter
43 0 0 0 3972 0 0 IGMP Snooping Pr
41 0 0 0 3972 0 0 dev_device_remov
PID TTY Allocated Freed Holding Getbufs Retbufs Process
40 0 0 0 3972 0 0 dev_device_inser
38 0 196 196 3972 0 0 Eswilp Storm Con
58518724 Total
Can any one who experienced this issue, please do investigate on this and help me what needs to be done.
Thanks in advance :-)
Regards,
Naidu.
- Labels:
-
Other Routing
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-20-2010 05:00 PM
Hi ,
-Memory fragmentation occurs when a process allocates itself a bunch of small blocks of
memory. Then later frees those blocks of memory leaving holes in memory.
-It is called fragmentation because the free portions of memory appear to be made up for a
bunch of small "fragments" separated by a memory that is still allocated.
-The first signs of a memory fragmentation issue is usually a memory allocation error.
*Nov 19 14:10:10.757: %SYS-2-MALLOCFAIL: Memory allocation of 5196 bytes failed from 0x602A8BE0, alignment 32
Pool: I/O Free: 53440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 5 -Traceback= 0x60943E34
0x601D9734 0x601DE4B8 0x601DF91C 0x602A8BE8 0x601ED130 0x601ED2C0 0x6152FF00 0x6152FEE4
Regards
syed.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-21-2010 11:40 PM
Hi Syed,
Thanks for your response.
Is that situation will cause anything to production?
I thought to reboot in off working hours, is that good sense? If not what should do to free up the memory.
Regards,
Naidu.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-23-2010 01:09 AM
Your router is facing an IO memory leak. Don't focus on the fragmentation, it's only a side effect of the leak. The main issue is that you're running out of IO memory and that's why you see MALLOCFAIL in Pool:I/O.
To troubleshoot this, collect 'show buffers'. Look at which pool is heavily used (high 'total' number), then dump the packets in that pool with 'show buffer pool XXXX dump'. We'll see what kind of packets are stuck in the buffers.
If you have a recent IOS, you can also use 'show memory debug leak' (will trigger a high CPU for a minute or so).
If you reload, memory will be cleared but we'll lose the information to troubleshoot the problem.
Only reload if you need to restore service quickly. It won't fix the problem definitely. The problem will come back later when the memory will be depleted again.
Xavier
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-13-2010 06:49 AM
Hi Xavier,
Sorry for the delay...
The issue got fixed by reload at that time but it repeated since today morning.
Please find the below required info for troubleshoot and I am running following version in my router "flash:c1841-advipservicesk9-mz.124-19.bin".
#sh buffers
Buffer elements:
1117 in free list (1119 max allowed)
6600382 hits, 0 misses, 619 created
Public buffer pools:
Small buffers, 104 bytes (total 50, permanent 50):
48 in free list (20 min, 150 max allowed)
197060170 hits, 82363 misses, 0 trims, 0 created
11333 failures (0 no memory)
Middle buffers, 600 bytes (total 131, permanent 25, peak 139 @ 2w0d):
129 in free list (10 min, 150 max allowed)
370711936 hits, 26699 misses, 1966 trims, 2072 created
12323 failures (0 no memory)
Big buffers, 1536 bytes (total 50, permanent 50, peak 56 @ 2w6d):
49 in free list (5 min, 150 max allowed)
58344057 hits, 93533 misses, 32 trims, 32 created
9128 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 48, permanent 15, peak 126 @ 1w6d):
47 in free list (46 min, 16 max allowed)
21535891 hits, 12226015 misses, 9894200 trims, 9894233 created
71736 failures (17117 no memory)
Large buffers, 5024 bytes (total 4426, permanent 3, peak 4426 @ 02:10:05):
4426 in free list (4562 min, 1 max allowed)
68026 hits, 61684 misses, 20146258 trims, 20150681 created
3710 failures (175327 no memory)
Huge buffers, 55535 bytes (total 30, permanent 30, need 1):
30 in free list (1 min, 1 max allowed)
3493 hits, 217 misses, 8 trims, 8 created
217 failures (1 no memory)
Interface buffer pools:
Syslog ED Pool buffers, 600 bytes (total 150, permanent 150):
118 in free list (150 min, 150 max allowed)
481 hits, 0 misses
Header pools:
Header buffers, 0 bytes (total 768, permanent 768):
256 in free list (128 min, 1024 max allowed)
512 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
512 max cache size, 512 in cache
423 hits in cache, 0 misses in cache
Particle Clones:
1024 clones, 0 hits, 0 misses
Public particle pools:
F/S buffers, 256 bytes (total 768, permanent 768):
256 in free list (128 min, 1024 max allowed)
512 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
512 max cache size, 512 in cache
0 hits in cache, 0 misses in cache
Normal buffers, 1548 bytes (total 768, permanent 768):
768 in free list (128 min, 1024 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Private particle pools:
FastEthernet0/0 buffers, 1552 bytes (total 512, permanent 512):
0 in free list (0 min, 512 max allowed)
512 hits, 0 fallbacks
512 max cache size, 256 in cache
174966831 hits in cache, 0 misses in cache
FastEthernet0/1 buffers, 1552 bytes (total 512, permanent 512):
0 in free list (0 min, 512 max allowed)
512 hits, 0 fallbacks
512 max cache size, 256 in cache
243947361 hits in cache, 0 misses in cache
#sh ver
Cisco IOS Software, 1841 Software (C1841-ADVIPSERVICESK9-M), Version 12.4(19), RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Fri 29-Feb-08 18:45 by prod_rel_team
ROM: System Bootstrap, Version 12.3(8r)T9, RELEASE SOFTWARE (fc1)
BremenRTR uptime is 3 weeks, 1 day, 12 hours, 37 minutes
System returned to ROM by reload at 04:06:40 utc Sun Nov 21 2010
System restarted at 03:14:25 utc Sun Nov 21 2010
System image file is "flash:c1841-advipservicesk9-mz.124-19.bin"
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html you require further assistance please contact us by sending email to
export@cisco.com. 1841 (revision 6.0) with 233472K/28672K bytes of memory.
Processor board ID FCZ1037221F
2 FastEthernet interfaces
1 Virtual Private Network (VPN) Module
DRAM configuration is 64 bits wide with parity disabled.
191K bytes of NVRAM.
62720K bytes of ATA CompactFlash (Read/Write)
If
Cisco
Configuration register is 0x2102
#sh proc mem
Processor Pool Total: 191063264 Used: 28469700 Free: 162593564
I/O Pool Total: 29360128 Used: 29301968 Free: 58160
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 38549724 10068032 23700464 0 0 *Init*
0 0 12052 1275580 12052 0 0 *Sched*
0 0 30328608 33046472 176212 290268 290268 *Dead*
0 0 0 0 396868 0 0 *MallocLite*
1 0 36813708 36314760 505920 0 0 Chunk Manager
2 0 196 196 3972 0 0 Load Meter
3 0 1592 9132 12972 0 0 Tunnel Security
4 0 3296 15068 10072 0 0 Check heaps
5 0 3774760072 3802212796 27569480 339930064 362709604 Pool Manager
6 0 196 196 6972 0 0 Timers
7 0 0 0 12972 0 0 OIR Handler
8 0 196 196 6972 0 0 Environmental mo
9 0 0 0 24972 0 0 Crash writer
10 0 3052 578040 10024 0 0 ARP Input
11 0 196 196 6972 0 0 ATM Idle Timer
12 0 196 196 6972 0 0 AAA high-capacit
13 0 0 0 6972 0 0 AAA_SERVER_DEADT
14 0 0 0 12972 0 0 Policy Manager
15 0 196 196 6972 0 0 DDR Timers
PID TTY Allocated Freed Holding Getbufs Retbufs Process
16 0 1620 0 8592 0 0 Entity MIB API
17 0 256876 0 266848 113400 0 EEM ED Syslog
18 0 0 0 6972 0 0 HC Counter Timer
19 0 196 196 6972 0 0 Serial Backgroun
20 0 0 0 6972 0 0 RO Notify Timers
21 0 196 196 6972 0 0 SMART
22 0 196 196 6972 0 0 GraphIt
23 0 196 196 12972 0 0 Dialer event
24 0 0 0 6972 0 0 SERIAL A'detect
25 0 196 196 12972 0 0 XML Proxy Client
26 0 7912 3632 18228 0 0 Crypto SS Proces
27 0 0 0 6972 0 0 Inode Table Dest
28 0 0 0 6972 0 0 Critical Bkgnd
29 0 58352 312 23188 0 0 Net Background
30 0 584 312 13244 0 0 IDB Work
31 0 196 196 12972 0 0 Logger
32 0 196 472 6972 0 0 TTY Background
33 0 4446296 3967076 9972 397460 0 Per-Second Jobs
34 0 0 0 6972 0 0 AggMgr Process
35 0 0 0 6972 0 0 Token Daemon
36 0 0 0 6972 0 0 Transport Port A
37 0 196 196 6972 0 0 ESWPPM
PID TTY Allocated Freed Holding Getbufs Retbufs Process
38 0 196 196 3972 0 0 Eswilp Storm Con
39 0 656064 1272 636792 0 0 USB Startup
40 0 0 0 3972 0 0 dev_device_inser
41 0 0 0 3972 0 0 dev_device_remov
42 0 196 196 6972 0 0 SM Monitor
43 0 0 0 3972 0 0 IGMP Snooping Pr
44 0 0 0 3972 0 0 IGMP Snooping Re
45 0 0 0 6972 0 0 Platform Compact
46 0 196 196 6972 0 0 Call Management
47 0 196 196 6972 0 0 PI MATM Aging Pr
48 0 2444 196 9220 0 0 DTP Protocol
49 0 196 196 6972 0 0 dot1x
50 0 196 196 6972 0 0 AAA Dictionary R
51 0 53168 5060 6972 0 0 AAA Server
52 0 0 0 6972 0 0 AAA ACCT Proc
53 0 0 0 6972 0 0 ACCT Periodic Pr
54 0 732 196 7508 0 0 CDP Protocol
55 0 35043692 751588 57968 20653892 476952 IP Input
56 0 0 0 6972 0 0 ICMP event handl
57 0 196 196 6972 0 0 TurboACL
58 0 196 196 6972 0 0 TurboACL chunk
59 0 196 196 6972 0 0 MOP Protocols
PID TTY Allocated Freed Holding Getbufs Retbufs Process
60 0 392 392 12972 0 0 PPP Hooks
61 0 0 0 6972 0 0 CRYPTO IKMP IPC
62 0 0 0 12972 0 0 SSS Manager
63 0 0 0 12972 0 0 SSS Test Client
64 0 0 0 6972 0 0 SSS Feature Mana
65 0 0 0 6972 0 0 SSS Feature Time
66 0 0 0 12972 0 0 VPDN call manage
67 0 0 0 13032 0 0 L2X Socket proce
68 0 0 0 12972 0 0 L2X SSS manager
69 0 196 196 12972 0 0 L2TP mgmt daemon
70 0 0 0 6972 0 0 X.25 Encaps Mana
71 0 196 196 9972 0 0 EAPoUDP Process
72 0 196 196 9972 0 0 IP Host Track Pr
73 0 0 0 6972 0 0 IPv6 RIB Redistr
74 0 196 196 12972 0 0 KRB5 AAA
75 0 76 0 10048 0 0 IP Background
76 0 528004 461860 76116 0 0 IP RIB Update
77 0 196 196 12972 0 0 PPP IP Route
78 0 196 196 12972 0 0 PPP IPCP
79 0 103000 0 73328 0 0 CEF process
80 0 0 0 6972 0 0 Socket Timers
81 0 492 156 7464 0 0 L2MM
PID TTY Allocated Freed Holding Getbufs Retbufs Process
82 0 0 0 6972 0 0 MRD
83 0 0 0 6972 0 0 IGMPSN
84 0 0 0 6972 0 0 IP Traceroute
85 0 196 196 6972 0 0 RLM groups Proce
86 0 0 0 6972 0 0 SNMP Timers
87 0 196 196 6972 0 0 SCTP Main Proces
88 0 0 0 6972 0 0 IUA Main Process
89 0 196 196 6972 0 0 RUDPV1 Main Proc
90 0 0 0 6972 0 0 bsm_timers
91 0 0 0 6972 0 0 bsm_xmt_proc
92 0 0 0 9972 0 0 CES Client SVC R
93 0 0 4848 12972 0 0 TCP Timer
94 0 1420808 8873052 14504 0 0 TCP Protocols
95 0 0 0 24972 0 0 COPS
96 0 96120 196 37024 0 0 DHCPD Receive
97 0 196 196 6972 0 0 Dialer Forwarder
98 0 0 469104 6972 0 0 IP Cache Ager
99 0 196 196 9972 0 0 Adj Manager
100 0 196 196 12972 0 0 ATM OAM Input
101 0 196 196 12972 0 0 ATM OAM TIMER
102 0 940 0 10912 0 0 HTTP CORE
103 0 0 0 6972 0 0 RARP Input
PID TTY Allocated Freed Holding Getbufs Retbufs Process
104 0 0 0 6972 0 0 PAD InCall
105 0 196 196 12972 0 0 X.25 Background
106 0 196 196 6972 0 0 PPP Bind
107 0 196 196 6972 0 0 PPP SSS
108 0 196 196 6972 0 0 RBSCP Background
109 0 0 0 12972 0 0 L2F management d
110 0 0 0 6972 0 0 Inspect Timer
111 0 0 0 6972 0 0 DHCPD Timer
112 0 0 0 6972 0 0 Authentication P
113 0 0 0 6972 0 0 Auth-proxy AAA B
114 0 0 0 12972 0 0 IPS Timer
115 0 368 196 7144 0 0 SDEE Management
116 0 0 0 6972 0 0 IPv6 Inspect Tim
117 0 1136 196 7912 0 0 URL filter proc
118 0 392 392 6972 0 0 Crypto HW Proc
119 0 196 196 6972 0 0 ENABLE AAA
120 0 0 0 6972 0 0 EM Background Pr
121 0 0 0 6972 0 0 Key chain liveke
122 0 196 196 6972 0 0 LINE AAA
123 0 992 1316 6972 0 0 LOCAL AAA
124 0 956 196 7732 0 0 TPLUS
125 0 196 196 6972 0 0 VSP_MGR
PID TTY Allocated Freed Holding Getbufs Retbufs Process
126 0 0 0 12972 0 0 encrypt proc
127 0 126868 440 135400 0 0 Crypto WUI
128 0 11004 11004 6972 0 0 Crypto Support
129 0 284 0 25256 0 0 RPMS_PROC_MAIN
130 0 0 0 6972 0 0 crypto engine pr
131 0 1372 596 13748 0 0 Crypto CA
132 0 0 0 8972 0 0 Crypto PKI-CRL
133 0 0 0 8972 0 0 Crypto SSL
134 0 111448 61696 76176 0 0 Crypto ACL
135 0 0 0 6972 0 0 CRYPTO QoS proce
136 0 0 0 6972 0 0 Crypto INT
137 0 1889136204 1888661820 161136 0 0 Crypto IKMP
138 0 651920 350996 226020 0 0 IPSEC key engine
139 0 0 0 6972 0 0 IPSEC manual key
140 0 0 0 6972 0 0 Crypto PAS Proc
141 0 24912 0 37884 0 0 Crypto Delete Ma
142 0 0 0 8972 0 0 Key Proc
143 0 0 0 9972 0 0 Crypto Device Up
144 0 0 0 6972 0 0 Crypto Hardware
145 0 0 0 6972 0 0 Multi-ISA Event
146 0 0 0 6972 0 0 Multi-ISA Cleanu
147 0 0 0 6972 0 0 PM Callback
PID TTY Allocated Freed Holding Getbufs Retbufs Process
148 0 164 0 7136 0 0 DATA Transfer Pr
149 0 164 0 7136 0 0 DATA Collector
150 0 14536 141064 6972 0 0 AAA SEND STOP EV
151 0 0 0 9972 0 0 EEM ED CLI
152 0 0 0 9972 0 0 EEM ED Counter
153 0 0 0 9972 0 0 EEM ED Interface
154 0 0 0 9972 0 0 EEM ED IOSWD
155 0 0 0 9972 0 0 EEM ED Memory-th
156 0 0 0 9972 0 0 EEM ED None
157 0 0 0 9972 0 0 EM ED OIR
158 0 0 0 9972 0 0 EEM ED SNMP
159 0 0 0 9972 0 0 EEM ED Timer
160 0 11056 1960 16068 0 0 EEM Server
161 0 196 196 6972 0 0 RMON Recycle Pro
162 0 196 196 6972 0 0 RMON Deferred Se
163 0 0 0 6972 0 0 Syslog Traps
164 0 9436 1784 14468 0 0 VLAN Manager
165 0 196 0 7168 0 0 DHCPD Database
166 0 9956 856 19072 0 0 EEM Policy Direc
167 0 336640 334280 15332 0 0 Syslog
168 0 0 0 6972 0 0 VPDN Scal
169 0 18881344 18939524 9468 0 0 crypto sw pk pro
PID TTY Allocated Freed Holding Getbufs Retbufs Process
170 0 0 0 6972 0 0 Net Input
171 0 196 196 6972 0 0 Compute load avg
172 0 31875952 172800 1752916 26933366 0 Per-minute Jobs
173 0 164 0 7136 0 0 CEF Scanner
174 0 196 0 25168 0 0 tHUB
175 194 114944 81944 45320 0 0 SSH Process
176 0 72052 0 11408 0 0 SSH Event handle
177 0 417892 519468 7996 0 0 NHRP
178 0 196 196 6972 0 0 IP Flow Backgrou
179 0 196 196 6972 0 0 IP NAT Ager
180 0 0 0 6972 0 0 IP NAT WLAN
181 0 0 0 6972 0 0 IP VFR proc
183 0 140652952 140641084 24048 0 0 IP SNMP
184 0 47632284 0 12972 0 0 PDU DISPATCHER
185 0 175828208 223461284 12972 0 0 SNMP ENGINE
186 0 1072 196 13848 0 0 IP SNMPV6
187 0 0 0 12972 0 0 SNMP ConfCopyPro
188 0 0 0 12972 0 0 SNMP Traps
189 0 0 0 6972 0 0 NAT MIB Helper
190 0 112124 77400 73184 0 0 RADIUS
191 0 552 196 7328 0 0 NTP
192 0 0 0 18080 0 0 IP-EIGRP Router
PID TTY Allocated Freed Holding Getbufs Retbufs Process
193 0 3410496 3315556 101380 0 0 IP-EIGRP: PDM
194 0 0 0 7032 0 0 IP-EIGRP: HELLO
57762260 Total
Please give me the correct resolution to fix this issue permanently...
Regards,
Naidu.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-15-2010 05:01 AM
Hi All,
Can someone give me any suggestions on this....
Thanks in advance.
Regards,
Naidu.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-15-2010 05:19 AM
Hi,
The leak is apparently in the large pool, please provide:
- show buffer pool Large dump
- show mem debug leak
thanks,
Xavier
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-16-2010 12:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-01-2011 01:33 PM
This does not look like a buffer leak:
Large buffers, 5024 bytes (total 4426, permanent 3, peak 4426 @ 02:10:05):
4426 in free list (4562 min, 1 max allowed)
68026 hits, 61684 misses, 20146258 trims, 20150681 created
3710 failures (175327 no memory)
This looks like a misconfiguration.
Do you have lines in your configuraiton like:
buffer large permanent 3
buffer large min-free 4562
buffer large max-free 1
This would cause the symptoms above and would be a misconfiguration
resulting in IO memory issues.
Please capture the 'show run | i buffer'
........Mike
As always mark your question answered when it is.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:
