Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Cisco Community
- Technology and Support
- Networking
- Routing
- Re: Cisco 2921 Router interfaces suddenly go down, fixes after restart
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
1572
Views
0
Helpful
6
Replies
Cisco 2921 Router interfaces suddenly go down, fixes after restart
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2018 03:49 AM - edited 03-05-2019 11:01 AM
Hi,
We are using C2921-SECk9 router with 1GB RAM, its two or three interfaces go down every one or two months. The interfaces start working again after a reboot. What could be the cause?
System Bootstrap, Version 15.0 (1r) M15, RELEASE SOFTWARE (fc1)
Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.6 (3) M, RELEASE SOFTWARE (fc1)
Last month interface Gi0/0 & 0/2 went down, this time interfaces Gi0/0 & 0/1 went down.
Labels:
- Labels:
-
vEdge Routers
6 Replies 6
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2018 04:19 AM
Hi Jameel,
Can you please share show interface logs prior to reboot and was other ports working at point of issue ?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2018 08:04 AM
#show ip int b
Interface IP-Address OK? Method Status Protocol
Embedded-Service-Engine0/0 unassigned YES NVRAM administratively down down
GigabitEthernet0/0 xx.xx.xx.18 YES NVRAM down down
GigabitEthernet0/1 xx.xx.xx9 YES manual down down
GigabitEthernet0/1.1 unassigned YES unset down down
GigabitEthernet0/2 xx.xx.xx.2 YES NVRAM up up
FastEthernet0/1/0 unassigned YES unset up up
FastEthernet0/1/1 unassigned YES unset up up
FastEthernet0/1/2 unassigned YES unset up up
FastEthernet0/1/3 unassigned YES unset up up
NVI0 unassigned YES unset up up
Tunnel1 xx.xx.xx.9 YES NVRAM up down
Tunnel2 xx.xx.xx.9 YES NVRAM up down
Tunnel3 172.16.2.2 YES NVRAM up down
Vlan1 unassigned YES unset down down
Vlan2 xx.xx.xx.114 YES NVRAM up up
Vlan5 xx.xx.xx.142 YES NVRAM up up
Vlan10 xx.xx.xx.50 YES NVRAM up up
Vlan75 xx.xx.xx.122 YES NVRAM up up
#conf t
Enter configuration commands, one per line. End with CNTL/Z.
(config)#int gi0/0
(config-if)#shut
(config-if)#no shut
(config-if)#^Z
#show ip int b
Interface IP-Address OK? Method Status Protocol
Embedded-Service-Engine0/0 unassigned YES NVRAM administratively down down
GigabitEthernet0/0 xx.xx.xx.18 YES NVRAM down down
GigabitEthernet0/1 xx.xx.xx9 YES manual down down
GigabitEthernet0/1.1 unassigned YES unset down down
GigabitEthernet0/2 xx.xx.xx.2 YES NVRAM up up
FastEthernet0/1/0 unassigned YES unset up up
FastEthernet0/1/1 unassigned YES unset up up
FastEthernet0/1/2 unassigned YES unset up up
FastEthernet0/1/3 unassigned YES unset up up
NVI0 unassigned YES unset up up
Tunnel1 xx.xx.xx.9 YES NVRAM up down
Tunnel2 xx.xx.xx.9 YES NVRAM up down
Tunnel3 172.16.2.2 YES NVRAM up down
Vlan1 unassigned YES unset down down
Vlan2 xx.xx.xx.114 YES NVRAM up up
Vlan5 xx.xx.xx.142 YES NVRAM up up
Vlan10 xx.xx.xx.50 YES NVRAM up up
Vlan75 xx.xx.xx.122 YES NVRAM up up
#sh int gi0/0 <-- THIS INTERFACE & GI0/1 WENT DOWN
GigabitEthernet0/0 is down, line protocol is down
Hardware is CN Gigabit Ethernet, address is 84b8.02f2.e390 (bia 84b8.02f2.e390
)
Description: 42MBPS link
Internet address is xx.xx.xx.XXX/30
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Auto Duplex, Auto Speed, media type is RJ45
output flow-control is unsupported, input flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 01:53:44, output 01:53:40, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/353 (size/max/drops/flushes); Total output drops: 135917
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
1000028587 packets input, 3309242993 bytes, 135339 no buffer
Received 149347 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
3539296076 packets output, 2116284506 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
1 lost carrier, 0 no carrier, 24849 pause output
0 output buffer failures, 0 output buffers swapped out
XXXXXXX#sh int gi0/2 <-- THIS INTERFACE WAS UP AND WORKING
GigabitEthernet0/2 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is 84b8.02f2.e392 (bia 84b8.02f2.e392
)
Description: ASA OUTSIDE
Internet address is xx.xx.xx.XXX/24
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Auto Duplex, Auto Speed, media type is RJ45
output flow-control is unsupported, input flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 01:59:04, output 01:41:46, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/1806 (size/max/drops/flushes); Total output drops: 9
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
1387194064 packets input, 2665815417 bytes, 0 no buffer
Received 368 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
150855 input errors, 0 CRC, 1 frame, 150853 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
3747736113 packets output, 2139195830 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
1 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
sh buffer
Buffer elements:
1254 in free list
415111098 hits, 0 misses, 1242 created
Public buffer pools:
Small buffers, 104 bytes (total 60, permanent 50, peak 114 @ 2w3d):
31 in free list (20 min, 150 max allowed)
149871475 hits, 8751 misses, 10047 trims, 10057 created
369 failures (0 no memory)
Middle buffers, 600 bytes (total 25, permanent 25, peak 74 @ 1w3d):
24 in free list (10 min, 150 max allowed)
41227352 hits, 1688 misses, 1496 trims, 1496 created
318 failures (0 no memory)
Big buffers, 1536 bytes (total 48, permanent 50, peak 158 @ 7w0d):
48 in free list (5 min, 150 max allowed)
148839972 hits, 474 misses, 346 trims, 344 created
230 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10, peak 11 @ 7w0d):
10 in free list (0 min, 100 max allowed)
847 hits, 133 misses, 4 trims, 4 created
133 failures (0 no memory)
Large buffers, 5024 bytes (total 1, permanent 0, peak 3 @ 3w3d):
1 in free list (0 min, 10 max allowed)
8 hits, 131 misses, 710 trims, 711 created
130 failures (0 no memory)
Huge buffers, 18024 bytes (total 5, permanent 0, peak 10 @ 7w0d):
5 in free list (4 min, 10 max allowed)
2821 hits, 126 misses, 1428 trims, 1433 created
115 failures (0 no memory)
Interface buffer pools:
CF Small buffers, 104 bytes (total 101, permanent 100, peak 101 @ 7w0d):
101 in free list (100 min, 200 max allowed)
0 hits, 0 misses, 706 trims, 707 created
0 failures (0 no memory)
Generic ED Pool buffers, 512 bytes (total 101, permanent 100, peak 101 @ 7w0d):
101 in free list (100 min, 100 max allowed)
0 hits, 0 misses
CF Middle buffers, 600 bytes (total 101, permanent 100, peak 101 @ 7w0d):
101 in free list (100 min, 200 max allowed)
0 hits, 0 misses, 706 trims, 707 created
0 failures (0 no memory)
Syslog ED Pool buffers, 600 bytes (total 133, permanent 132, peak 133 @ 7w0d):
101 in free list (132 min, 132 max allowed)
1509 hits, 0 misses
CF Big buffers, 1536 bytes (total 26, permanent 25, peak 26 @ 7w0d):
26 in free list (25 min, 50 max allowed)
0 hits, 0 misses, 706 trims, 707 created
0 failures (0 no memory)
IPC buffers, 4096 bytes (total 2, permanent 2):
1 in free list (1 min, 8 max allowed)
1 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
CF VeryBig buffers, 4520 bytes (total 3, permanent 2, peak 3 @ 7w0d):
3 in free list (2 min, 4 max allowed)
0 hits, 0 misses, 706 trims, 707 created
0 failures (0 no memory)
CF Large buffers, 5024 bytes (total 2, permanent 1, peak 2 @ 7w0d):
2 in free list (1 min, 2 max allowed)
0 hits, 0 misses, 706 trims, 707 created
0 failures (0 no memory)
IPC Medium buffers, 16384 bytes (total 2, permanent 2):
2 in free list (1 min, 8 max allowed)
0 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
IPC Large buffers, 65535 bytes (total 17, permanent 16, peak 17 @ 7w0d):
17 in free list (16 min, 16 max allowed)
0 hits, 0 misses, 114235 trims, 114236 created
0 failures (0 no memory)
Header pools:
Header buffers, 0 bytes (total 768, permanent 768):
256 in free list (128 min, 1024 max allowed)
512 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
512 max cache size, 512 in cache
12171333 hits in cache, 0 misses in cache
Particle Clones:
1024 clones, 0 hits, 0 misses
Public particle pools:
F/S buffers, 1664 bytes (total 768, permanent 768):
256 in free list (128 min, 1024 max allowed)
512 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
512 max cache size, 512 in cache
0 hits in cache, 0 misses in cache
Normal buffers, 1676 bytes (total 3840, permanent 3840):
3840 in free list (128 min, 4096 max allowed)
279 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Private particle pools:
HQF Particle buffers, 0 bytes (total 2000, permanent 2000):
2000 in free list (500 min, 2000 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
IDS SM buffers, 240 bytes (total 128, permanent 128):
0 in free list (0 min, 128 max allowed)
128 hits, 0 fallbacks
128 max cache size, 128 in cache
0 hits in cache, 0 misses in cache
FastEthernet0/1/0 buffers, 1548 bytes (total 128, permanent 128):
0 in free list (0 min, 128 max allowed)
128 hits, 279 fallbacks
128 max cache size, 64 in cache
2789312635 hits in cache, 215 misses in cache
GigabitEthernet0/0 buffers, 1664 bytes (total 1024, permanent 1024):
0 in free list (0 min, 1024 max allowed)
1024 hits, 0 fallbacks
1024 max cache size, 768 in cache
1001136832 hits in cache, 0 misses in cache
GigabitEthernet0/1 buffers, 1664 bytes (total 1024, permanent 1024):
0 in free list (0 min, 1024 max allowed)
1024 hits, 0 fallbacks
1024 max cache size, 768 in cache
7359 hits in cache, 0 misses in cache
GigabitEthernet0/2 buffers, 1664 bytes (total 1024, permanent 1024):
0 in free list (0 min, 1024 max allowed)
1024 hits, 0 fallbacks
1024 max cache size, 768 in cache
1387196170 hits in cache, 0 misses in cache
#show logg
Oct 29 19:53:58.556 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.122, prot=50, spi=0x47455420(1195725856), srcaddr=45.79.106.170, input interface=Vlan10
Oct 29 20:50:56.178 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.32, prot=50, spi=0x47455420(1195725856), srcaddr=45.79.106.170, input interface=GigabitEthernet0/0
Oct 30 02:13:22.840 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.18, prot=50, spi=0x9146DA5F(2437339743), srcaddr=58.27.215.165, input interface=GigabitEthernet0/0
Oct 30 05:28:32.533 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.144, prot=50, spi=0x30303030(808464432), srcaddr=125.212.217.214, input interface=Vlan5
Oct 30 10:25:54.431 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.145, prot=50, spi=0x47455420(1195725856), srcaddr=45.79.106.170, input interface=Vlan5
Oct 30 13:42:03.742 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.145, prot=50, spi=0x30303030(808464432), srcaddr=198.20.99.130, input interface=Vlan5
Oct 30 15:32:01.167 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.18, prot=50, spi=0x86F47665(2264168037), srcaddr=203.128.252.110, input interface=GigabitEthernet0/0
Oct 30 15:56:48.057 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.18, prot=50, spi=0x7B224248(2065842760), srcaddr=182.180.166.56, input interface=GigabitEthernet0/0
Oct 30 18:04:10.283 PST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=xx.xx.xx.18, prot=50, spi=0x55B1163F(1437668927), srcaddr=58.27.215.165, input interface=GigabitEthernet0/0
Oct 30 20:57:33.088 PST: %LINK-3-UPDOWN: Interface GigabitEthernet0/0, changed state to down
Oct 30 20:58:21.108 PST: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2018 05:47 AM
Hello,
sounds like some sort of memory leak...
Try and upgrade to 15.7.3M3...
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2018 08:08 AM
Ok, I will try upgrading the IOS. But for clarification, I wanted to ask if the memory leaks reflect in show process memory command? I checked memory right after the issue and a lot of memory was free. Here is the output.
#show process mem
Processor Pool Total: 1352702428 Used: 141130184 Free: 1211572244
I/O Pool Total: 41943040 Used: 18626992 Free: 23316048
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 179834676 92309232 82031900 662 16601618 *Init*
0 0 12788 6363868 12788 41 41 *Sched*
0 0 130400404 125831484 1354648 243 241 *Dead*
0 0 0 0 591704 0 0 *MallocLite*
1 0 174919828 124940728 49986364 0 0 Chunk Manager
2 0 240 240 4264 0 0 Load Meter
3 0 29480 808 54608 0 0 LICENSE AGENT
4 0 65588 0 90852 0 0 EDDRI_MAIN
5 0 0 0 7264 0 0 RO Notify Timers
6 0 3376 240 10400 0 0 Check heaps
7 0 3417623384 3420550608 256536 49 0 Pool Manager
8 0 0 0 7264 0 0 DiscardQ Backgro
9 0 240 240 7264 0 0 Timers
10 0 0 0 4264 0 0 WATCH_AFS
11 0 0 0 7264 0 0 License Client N
12 0 0 0 7264 0 0 Image License br
13 0 27915784 27916516 7264 0 0 Licensing Auto U
14 0 183456 17316 181372 0 0 RF Slave Main Th
15 0 0 0 4264 0 0 RMI RM Notify Wa
16 0 240 240 7264 0 0 Environmental mo
17 0 0 0 7264 0 0 IPC Event Notifi
18 0 0 0 7264 0 0 IPC Dynamic Cach
19 0 0 0 7264 0 0 IPC Session Serv
20 0 0 0 7264 0 0 IPC Zone Manager
21 0 0 0 7264 0 0 IPC Periodic Tim
22 0 0 0 7264 0 0 IPC Deferred Por
23 0 0 0 7264 0 0 IPC Process leve
24 0 2104 0 9368 0 0 IPC Seat Manager
25 0 0 0 7264 0 0 IPC Check Queue
26 0 0 0 7264 0 0 IPC Seat RX Cont
27 0 0 0 7264 0 0 IPC Seat TX Cont
28 0 0 0 7264 0 0 IPC Keep Alive M
29 0 0 0 7264 0 0 IPC Loadometer
30 0 0 0 7264 0 0 IPC Session Deta
31 0 0 0 25264 0 0 Crash writer
32 0 380 0 7644 0 0 Exception contro
33 0 0 0 7128 0 0 CRYPTO IKMP IPC
34 0 0 0 7264 0 0 IFS Agent Manage
35 0 29360 12600 24024 13343527 13343527 ARP Input
36 0 177766548 177766548 7264 9333 9333 ARP Background
37 0 240 240 7264 0 0 ATM Idle Timer
38 0 0 0 7264 0 0 ATM ASYNC PROC
39 0 0 0 7264 0 0 CEF MIB API
40 0 0 0 7264 0 0 AAA_SERVER_DEADT
41 0 0 0 13264 0 0 Policy Manager
42 0 240 240 7264 0 0 DDR Timers
43 0 29024 340 36208 43 43 Entity MIB API
44 0 240 240 7264 0 0 PrstVbl
45 0 240 240 7264 0 0 Serial Backgroun
46 0 0 0 13264 0 0 License IPC stat
47 0 0 0 13264 0 0 License IPC serv
PID TTY Allocated Freed Holding Getbufs Retbufs Process
48 0 240 240 7264 0 0 SMART
49 0 1688 0 8952 0 0 RF SCTPthread
50 0 1688 0 8952 0 0 CHKPT RG SCTPthr
51 0 0 0 7264 13180459 13180459 ARP Snoop
52 0 240 240 7264 0 0 GraphIt
53 0 240 240 13264 0 0 Dialer event
54 0 0 0 7264 0 0 SERIAL A'detect
55 0 240 240 13264 0 0 XML Proxy Client
56 0 0 6232 7264 0 0 crypto sw pk pro
57 0 240 240 7264 0 0 ISM-VPN main
58 0 0 0 7264 0 0 ISM-VPN Cleanup
59 0 0 0 7264 0 0 Critical Bkgnd
60 0 228776 6712 19792 2051211 2051211 Net Background
61 0 2280 2280 13264 0 0 IDB Work
62 0 240 240 13264 2939 2939 Logger
63 0 77160 10388 7264 0 0 TTY Background
65 0 1044 0 14308 0 0 CTS CORE
66 0 1044 20320 14068 0 0 SXP CORE
67 0 67132 0 74396 0 0 IF-MGR control p
68 0 27468 0 17316 36 36 IF-MGR event pro
69 0 0 0 4264 0 0 Inode Table Dest
70 0 0 0 7264 0 0 cpf_msg_holdq_pr
71 0 0 0 7264 0 0 cpf_msg_rcvq_pro
72 0 0 0 7264 0 0 cpf_process_tpQ
73 0 0 0 7264 0 0 CHKPT Test clien
74 0 0 0 7264 0 0 CHKPT Test clien
75 0 0 0 7264 0 0 CHKPT Test clien
76 0 0 0 7264 0 0 CHKPT Test clien
77 0 0 0 7264 0 0 CHKPT Test clien
78 0 0 0 7264 0 0 CHKPT Test clien
79 0 0 0 7264 0 0 CHKPT Test clien
80 0 0 0 7264 0 0 CHKPT Test clien
81 0 0 0 7264 0 0 CHKPT Test clien
82 0 0 0 7264 0 0 CHKPT Test clien
83 0 0 0 7264 0 0 CHKPT Test clien
84 0 0 0 13264 0 0 Crypto PKI-HA
85 0 0 0 7264 0 0 IKE HA Mgr
86 0 0 0 7264 0 0 IPSEC HA Mgr
87 0 240 240 7264 0 0 TDM Management
88 0 18420 10588 11084 0 0 RF Master Main T
89 0 1688 0 8952 0 0 RF Master Status
90 0 0 0 7264 8589 8589 Net Input
91 0 240 240 7264 0 0 Compute load avg
92 0 0 3692 7264 0 0 Per-minute Jobs
93 0 147728820 147600472 138612 11197661 11197661 Per-Second Jobs
94 0 0 0 7264 0 0 AggMgr Process
95 0 0 0 7264 0 0 Token Daemon
96 0 0 1838896 7264 0 0 Transport Port A
97 0 0 0 7264 0 0 HC Counter Timer
98 0 240 240 7264 0 0 SM Monitor
99 0 240 240 7264 0 0 Bryce I2C CMD Qu
100 0 240 240 13264 0 0 SM Installer
101 0 0 0 4264 0 0 dev_device_inser
102 0 0 0 4264 0 0 dev_device_remov
103 0 0 0 13264 0 0 sal_dpc_process
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-01-2018 05:43 AM
Ok. I will upgrade and get back.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2018 03:24 PM
@Jameel Ahmed wrote:
Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.6(3)M, RELEASE SOFTWARE (fc1)
I think you need to upgrade the IOS.
Getting Started
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:
Customers Also Viewed These Support Documents
