ASR1002-X
Cisco IOS XE Software, Version 03.13.05.S - Extended Support Release
The problem I am seeing is that anytime I run a "show ip ospf [anything]" command, it takes an extremely long time to load each entry. For example, when I run "show ip ospf neighbor", it takes ~51 seconds to list my three ospf neighbors (~17 seconds to list each neighbor). Any header information for the command displays instantly. This goes for any related commands. However, I don't see this with any other show command. Currently, there are roughly 300 prefixes in the OSPF database, all in area 0 (which we are migrating away from as quickly as possible). I verified the size of the database is not the cause of the problem when I set OSPF to "passive-interface default" and reduced the size of the database to 9 prefixes (redistributed routes). With no OSPF neighbors, "show ip ospf neighbor" was instant, as expected, but "show ip ospf database" still took ~17 seconds to display each line. Additionally, when the terminal window fills up and is waiting for input (space or return) to display more, if I break the command to return to the prompt, it takes just as long to display the prompt again as it does to display each line of the show command.
CPU and memory utilization don't seem to have any issues. The displayed output was gathered while commands were running (displayed are the top 15):
ROUTER#show proc cpu sorted
Load for five secs: 0%/0%; one minute: 1%; five minutes: 1%
Time source is NTP, 10:58:36.639 PST Sat Dec 10 2016
CPU utilization for five seconds: 0%/0%; one minute: 1%; five minutes: 1%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
96 1194959 123183987 9 0.07% 0.00% 0.00% 0 100ms check
132 1140824 12320813 92 0.07% 0.01% 0.00% 0 Per-Second Jobs
179 2474073 192239861 12 0.07% 0.02% 0.00% 0 VRRS Main thread
194 118 1016 116 0.07% 0.00% 0.00% 2 SSH Process
208 18503069 1521573849 12 0.07% 0.12% 0.13% 0 Ethernet Msec Ti
241 3228536 382579698 8 0.07% 0.02% 0.00% 0 IPAM Manager
286 1184057 31097558 38 0.07% 0.01% 0.00% 0 TCP Timer
1 0 76 0 0.00% 0.00% 0.00% 0 Chunk Manager
2 520742 2464137 211 0.00% 0.00% 0.00% 0 Load Meter
3 78087 2663276 29 0.00% 0.00% 0.00% 0 LACP Protocol
4 0 16 0 0.00% 0.00% 0.00% 0 Retransmission o
5 0 5 0 0.00% 0.00% 0.00% 0 IPC ISSU Dispatc
6 3 12 250 0.00% 0.00% 0.00% 0 RF Slave Main Th
7 0 1 0 0.00% 0.00% 0.00% 0 EDDRI_MAIN
8 0 1 0 0.00% 0.00% 0.00% 0 RO Notify Timers
ROUTER#show proc memory sorted
Load for five secs: 0%/0%; one minute: 1%; five minutes: 1%
Time source is NTP, 10:53:30.263 PST Sat Dec 10 2016
Processor Pool Total: 4839512432 Used: 727294376 Free: 4112218056
lsmpi_io Pool Total: 6295128 Used: 6294296 Free: 832
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 850624440 511028880 314723752 0 0 *Init*
359 0 893916024 285573784 296081400 0 0 BGP Router
579 0 34427376 51784 34751592 0 0 SBC main process
0 0 23568134288 17045126880 27932728 978852307 42612 *Dead*
76 0 3647343352 3209769912 8335216 2412 10452 IOSD ipc task
157 0 8722512 134760 6460552 0 0 CWAN OIR Handler
462 0 3904128 5160 3940832 849828 0 EEM ED Syslog
1 0 2503480 71736 2487240 0 0 Chunk Manager
442 0 2018688 273872 1736264 4824 0 Crypto IKEv2
482 0 1444024 27688 1446200 0 0 EEM Server
485 0 1259130320 1257068784 1394688 0 0 Archive Config
0 0 0 0 1314200 0 0 *MallocLite*
604 0 713794808 682674720 1255232 0 0 OSPF-1 Router
275 0 806548152 2213504208 1110920 9648 0 IP RIB Update
424 0 631096 0 672960 0 0 QOS_MODULE_MAIN
I've tried searching the release notes and caveats for this software version, but didn't see anything that would seem to indicate there was a problem related to the issue I am seeing, but I could've also missed something. Has anyone else ever seen this behavior or have any suggestions for troubleshooting the issue?
