Slow Telnet.

[peter.hoegh]

Hello all

Recently we had a unintended reboot of our maine router (Cisco 6506) It looked fine and looked like the reboot went smooth. We are now experiencing extremely slow Telnet access, slow login and "rubber keys" when we try to enter a command in the router. Serial connection is fine and the CLI is very responsive when you log in via. the console. 

The router is not high in memory or cpu usage. It's working fine on the network, not slow in ordinary work.    

The configuration on the router looks ok, this includes STP settings. We have no STP blocked ports. I have tried to reboot the unit. No result.

I am quit desperate so if you have any advice, please share.

Thanks in advance.

Peter      

[Leo Laohoo]

---

@peter.hoegh wrote:

we had a unintended reboot of our maine router   

---

Post the complete output to the following commands: 

• sh version
• sh module
• sh proc cpu sort | ex 0.00

[peter.hoegh]

SW_Raadhus-S4.1_SK_new#sh ver
Cisco IOS Software, IOS-XE Software, Catalyst 4500 L3 Switch Software (cat4500es8-UNIVERSALK9-M), Version 03.06.08.E RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2018 by Cisco Systems, Inc.
Compiled Mon 22-Jan-18 09:19 by prod_rel_team



Cisco IOS-XE software, Copyright (c) 2005-2015 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.



ROM: 15.1(1r)SG5
SW_Raadhus-S4.1_SK_new uptime is 1 week, 3 hours, 15 minutes
Uptime for this control processor is 4 hours, 21 minutes
System returned to ROM by SSO Switchover
System restarted at 07:23:33 CEDT Tue Jul 5 2022
System image file is "bootflash:/cat4500es8-universalk9.SPA.03.06.08.E.152-2.E8.bin"
Jawa Revision 3, RadTrooper Revision 0x0.0x41, Conan Revision 0x1449

Last reload reason: Reload command



This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.


License Information for 'WS-X45-SUP8-E'
License Level: entservices Type: Permanent
Next reboot license Level: entservices

cisco WS-C4510R+E (P5040) processor (revision 2) with 4194304K bytes of physical memory.
Processor board ID FXS1851Q18J
P5040 CPU at 2.2GHz, Supervisor 8-E
Last reset from Reload
68 Virtual Ethernet interfaces
126 Gigabit Ethernet interfaces
16 Ten Gigabit Ethernet interfaces
511K bytes of non-volatile configuration memory.

Configuration register is 0x2

SW_Raadhus-S4.1_SK_new#sh module
Chassis Type : WS-C4510R+E

Power consumed by backplane : 40 Watts

Mod Ports Card Type Model Serial No.
---+-----+--------------------------------------+------------------+-----------
1 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1902L5BR
2 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1902L566
3 12 1000BaseX SFP WS-X4712-SFP-E CAT1851L3Z5
4 12 1000BaseX SFP WS-X4712-SFP-E CAT1851L3Y3
5 8 Sup 8-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP8-E CAT1839L7E6
6 8 Sup 8-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP8-E CAT1818L17K
8 6 1000BaseX (GBIC) WS-X4306-GB JAE064500WK

M MAC addresses Hw Fw Sw Status
--+--------------------------------+---+------------+----------------+---------
1 64f6.9d51.2f80 to 64f6.9d51.2faf 1.1 Ok
2 64f6.9d51.3190 to 64f6.9d51.31bf 1.1 Ok
3 64f6.9db6.9280 to 64f6.9db6.928b 1.0 Ok
4 64f6.9db6.94e4 to 64f6.9db6.94ef 1.0 Ok
5 64f6.9d1e.9440 to 64f6.9d1e.9447 1.2 15.1(1r)SG5 03.06.08.E Ok
6 64f6.9d1e.9448 to 64f6.9d1e.944f 1.2 15.1(1r)SG5 03.06.08.E Ok
8 0007.eb97.f7d6 to 0007.eb97.f7db 2.2 Ok

Mod Redundancy role Operating mode Redundancy status
----+-------------------+-------------------+----------------------------------
5 Active Supervisor SSO Active
6 Standby Supervisor SSO Standby hot

SW_Raadhus-S4.1_SK_new#sh proc cpu sort | ex 0.00
Core 0: CPU utilization for five seconds: 24%; one minute: 25%; five minutes: 24%
Core 1: CPU utilization for five seconds: 41%; one minute: 36%; five minutes: 37%
Core 2: CPU utilization for five seconds: 4%; one minute: 3%; five minutes: 3%
Core 3: CPU utilization for five seconds: 5%; one minute: 6%; five minutes: 6%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
5216 4171506 50867453 250 18.86 17.81 18.09 0 iosd
5159 86407 531318 162 0.19 0.13 0.13 0 ffm
5163 10218 41003 249 0.02 0.02 0.02 0 cpumemd

SW_Raadhus-S4.1_SK_new#

[Leo Laohoo]

---

@peter.hoegh wrote:
SW_Raadhus-S4.1_SK_new uptime is 1 week, 3 hours, 15 minutes
Uptime for this control processor is 4 hours, 21 minutes
System returned to ROM by SSO Switchover

---

This does not bode well.

Is this a VSS pair?

[peter.hoegh]

Hi Leo
No, we only have one Cisco 4500, so no VSS. I booted the switch this morning, it fits the 4 hours on the control processor’s uptime.
I thought this meant that my other Supervisor module took over and retained redundancy on my router.
I have one active Supervisor and one standby Supervisor. I was told that this was like it was supposed to be = Redundancy Status: Standby Hot
I probably don’t understand the finer parts

[balaji.bandi]

is this live network ?

you can pull out one of the supervisor and test it.

other side you can try console see is that better ?

looks for me any debug running constantly ?

what IP you trying to connect ? what is your IP address ?

can you post below output :

show run

show ip interface brief

[MHM Cisco World]

first indication of high CPU utilize is slow telnet, 

I see that IOSd run around 18% from CPU 
can I see the config of SW

[MHM Cisco World]

I think this high CPU utilize due to BGP scanner in SW,

check the CPU history are it periodic with BGP update ??

[Leo Laohoo]

Please post the complete output to the command "dir slavecrashinfo:".

[peter.hoegh]

Hi Leo
As requested output from “Dir slavechrashinfo:”

SW_Raadhus-S4.1_SK_new#dir slavecrashinfo:
Directory of slavecrashinfo:/

4034 -rw- 0 Dec 16 2015 16:38:17 +01:00 koops.dat
4035 -rw- 0 Jul 5 2022 07:33:08 +02:00 cilogs
4036 -rw- 0 Jan 21 2018 12:59:40 +01:00 deleted_crash_files
4037 -rwx 1593016 Jan 21 2018 12:59:40 +01:00 crashinfo_iosd_20180121-115800-UTC
4038 -rw- 44 Jan 21 2018 12:59:40 +01:00 last_crashinfo
4039 -rwx 77704273 Jan 21 2018 12:59:45 +01:00 fullcore_iosd_20180121-115800-UTC
4040 -rw- 272 Jul 5 2022 07:31:12 +02:00 last_systemreport_log

167252992 bytes total (83939328 bytes free)

[Leo Laohoo]

If the Active supervisor card is in slot 5 (and slot 6) is the standby, then post the complete output to the command "sh log onboard module 6 uptime detail ". 

[marce1000]

          - Do you have same experience with other protocol such as SSH ?
 M.

[peter.hoegh]

Hello
Yes, we have the same problem with SSH.

[marce1000]

 - Is there any chance that some part of configuration was not saved before reboot. on some platforms you have separate vrf for management access which needs dedicated ip for full and fast managerial access, probably not applicable here but verify if the management access is still as intended per current configuration such as shortest routing to device and no other routing conflicts which could now induce delays (e.g).

 M.

[peter.hoegh]

Hi
Our network is separated with vlans. We have a administrative vlan (vlan 900) This interface has ip address 172.20.27.1 which is the address we use when we try to access the router. I have checked the interfaces and all the configuration seems to be ok. All so we don’t get any errors, we can log in, we can execute commands but I takes light-years compared too before. We don’t use any no vrf schemes to administration.
I have looked at our routing tables and compared them to our latest backup, looks fine.