catalyst 9200 stack issue

[M Mohammed]

Hi All,

i have cisco catalyst 9200, a stack of 5, which reboots every often. please see below in the output for the changes to the link OK its 8 and 6. could that be a possibility for the reboot and why is that different any advise please? 

Sw#/Port# Port Status Neighbor Cable Length Link OK Link Active Sync OK #Changes to LinkOK In Loopback
-------------------------------------------------------------------------------------------------------------------
1/1 OK 5 100cm Yes Yes Yes 1 No
1/2 OK 2 50cm Yes Yes Yes 8 No
2/1 OK 1 50cm Yes Yes Yes 6 No
2/2 OK 3 50cm Yes Yes Yes 1 No
3/1 OK 2 50cm Yes Yes Yes 1 No
3/2 OK 4 50cm Yes Yes Yes 1 No
4/1 OK 3 50cm Yes Yes Yes 1 No
4/2 OK 5 50cm Yes Yes Yes 1 No
5/1 OK 4 50cm Yes Yes Yes 1 No
5/2 OK 1 100cm Yes Yes Yes 1 No

Kind regards

Mohammed

[Leo Laohoo]

What firmware is the stack running on?

[M Mohammed]

hi @Leo Laohoo 

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 52 C9200L-48P-4G 16.11.1 CAT9K_LITE_IOSXE INSTALL
2 52 C9200L-48P-4G 16.11.1 CAT9K_LITE_IOSXE INSTALL
3 52 C9200L-48P-4G 16.11.1 CAT9K_LITE_IOSXE INSTALL
4 52 C9200L-48P-4G 16.11.1 CAT9K_LITE_IOSXE INSTALL
5 52 C9200L-48P-4G 16.11.1 CAT9K_LITE_IOSXE INSTALL

[balaji.bandi]

i would advise to cross check all the stackcables and they are intact.

i may see some kind of bug reported like this, not able to find one. once they re-fix the stack cable, it was stable later.

[Leo Laohoo]

16.11.1?
I seriously recommend upgrading the firmware of the stack.
16.11.1 has several serious issues with stacking.
Post the complete output to the command "sh platform soft mount switch <SWITCH NUMBER> r0 | i ^tmpfs.*tmp | e /tmp/".
Normal value is <5%. If any of the switch has a value of >70% then this is a well-known bug: CSCvp73396
And now here's the kicker: If any member of the stack has crashed and recovered, they might be hitting a 2nd bug - CSCvp03655.

[M Mohammed]

@Leo Laohoo 

did run the command but found not output

WIN-3FL-SW1#sh platform software mount switch 1 R0 | i ^tmpfs.*tmp | e /tmp/
WIN-3FL-SW1#

any other output you think 

@balaji.bandi  thanks

[balaji.bandi]

since you did not find any output as expected.

i still stand on same point, check the Physical cables, and upgrade to latest suggest code .

[M Mohammed]

@balaji.bandi 

sure, i will try as you recommend, will find the downtime and once done let you know the outcome

regards

Mohammed

[Leo Laohoo]

Pick one of the switches that "rebooted" recently.
Post the complete output to the command "dir crashinfo-<SWITCH NUMBER>:".

[M Mohammed]

Hi @Leo Laohoo Please see below

WIN-3FL-SW1#dir crashinfo-1
%Error opening flash:/crashinfo-1 (No such file or directory)
WIN-3FL-SW1#dir crashinfo-2
%Error opening flash:/crashinfo-2 (No such file or directory)

[Leo Laohoo]

Syntax is incorrect.
It should be "dir crashinfo-X:" (notice the ":" at the end).

[M Mohammed]

Hi, @Leo Laohoo 

Please see below the information for CRASHINFO

SW1#dir crashinfo-1:
Directory of crashinfo:/

11 -rw- 0 Jan 1 1970 00:00:56 +00:00 koops.dat
36641 drwx 32768 Mar 11 2020 15:24:13 +00:00 tracelogs
30 -rw- 25699930 Nov 6 2019 23:15:57 +00:00 system-report_1_20191106-231545-UTC.tar.gz
12 -rw- 1197256 Aug 5 2019 11:23:38 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20190805-112335.tar.gz
13 -rw- 1211417 Aug 5 2019 11:23:41 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_2-20190805-112339.tar.gz
15 -rw- 372311 Aug 26 2019 05:01:49 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_7-20190826-050148.tar.gz
14 -rw- 1257070 Aug 6 2019 00:58:38 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_5-20190806-005832.tar.gz
16 -rw- 385857 Aug 26 2019 05:01:51 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_9-20190826-050150.tar.gz
17 -rw- 392361 Sep 4 2019 05:56:24 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_14-20190904-055622.tar.gz
18 -rw- 412411 Sep 12 2019 22:54:03 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_16-20190912-225401.tar.gz
19 -rw- 421996 Sep 12 2019 22:54:03 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_18-20190912-225401.tar.gz
20 -rw- 355484 Sep 14 2019 22:54:06 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_20-20190914-225405.tar.gz
21 -rw- 408611 Sep 14 2019 22:54:22 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_23-20190914-225421.tar.gz
22 -rw- 474900 Sep 14 2019 22:58:26 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_24-20190914-225825.tar.gz
23 -rw- 400298 Sep 21 2019 20:58:36 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_26-20190921-205832.tar.gz
24 -rw- 1526472 Sep 22 2019 09:02:59 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_1-20190922-090252.tar.gz
25 -rw- 297003 Sep 28 2019 00:18:35 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_4-20190928-001833.tar.gz
26 -rw- 319362 Oct 1 2019 13:26:54 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_2-20191001-132651.tar.gz
27 -rw- 1339770 Oct 3 2019 04:42:32 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20191003-044227.tar.gz
28 -rw- 295564 Oct 5 2019 08:11:33 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_1-20191005-081130.tar.gz
29 -rw- 354224 Oct 29 2019 21:23:52 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20191029-212348.tar.gz
31 -rw- 309854 Nov 6 2019 23:14:51 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20191106-231448.tar.gz
32 -rw- 385508 Nov 14 2019 01:32:49 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_1-20191114-013246.tar.gz
33 -rw- 386882 Nov 14 2019 01:32:49 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_3-20191114-013246.tar.gz
34 -rw- 594134 Nov 14 2019 10:49:23 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_6-20191114-104919.tar.gz
35 -rw- 1588260 Nov 14 2019 22:27:31 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20191114-222726.tar.gz
36 -rw- 391763 Nov 28 2019 08:19:50 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20191128-081947.tar.gz
37 -rw- 346899 Dec 9 2019 09:49:49 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_0-20191209-094947.tar.gz
38 -rw- 330034 Dec 28 2019 23:39:32 +00:00 WIN-3FL-SW1_1_RP_0_trace_archive_1-20191228-233931.tar.gz

825638912 bytes total (738447360 bytes free)

[Leo Laohoo]

Whao. That is a A LOT OF files!
Are you sure you still want to use 16.11.0?

[M Mohammed]

ok, will upgrade soon and monitor before i share my results, many thanks!!

[Leo Laohoo]

CSCvq56135