Re: AP-C9115AXI-E disconnect from c9800 controller and rejoin

sumithliyanage · ‎10-27-2023

We have a c9800-48 controller connect with 93 of c9115AXI, controller monitor by solar wind, AP goes down randomly and rejoin again without disconnecting the session.

Alert is similar to below:

The following Access Point went DOWN on 27 October 2023 at 15:33: -

Access Point: TDL-Ground-Front-Office

IP Address: 10.16.23.3

This alert was triggered by the alert named 'PO3 Access Point Changed Status'

=================================================================

after exactly the 10 min

The following Access Point came UP on 27 October 2023 at 15:43: -

Access Point: TDL-Ground-Front-Office

IP Address: 10.16.23.3

===========================================

checking the logs on AP similar to below

Oct 27 11:42:44 kernel: [*10/27/2023 11:42:44.3087] CLSM[AE:2B:C2:8A:72:D7]: Failed adding dms client config
Oct 27 12:02:46 kernel: [*10/27/2023 12:02:46.2741] CLSM[C6:60:F8:E8:DD:F3]: Failed adding dms client config
Oct 27 12:38:16 kernel: [*10/27/2023 12:38:16.7105] DOT11_DRV[0]: off_chhannel RX timeout, ch = 1
Oct 27 13:07:15 kernel: [*10/27/2023 13:07:15.3090] wl: Bad Address
Oct 27 13:42:16 kernel: [*10/27/2023 13:42:16.3467] Re-Tx Count=1, Max Re-Tx Value=8, SendSeqNum=8, NumofPendingMsgs=2
Oct 27 13:42:16 kernel: [*10/27/2023 13:42:16.3467]
Oct 27 13:42:21 kernel: [*10/27/2023 13:42:21.0976] Re-Tx Count=2, Max Re-Tx Value=8, SendSeqNum=9, NumofPendingMsgs=3
Oct 27 13:42:21 kernel: [*10/27/2023 13:42:21.0976]
Oct 27 13:42:25 kernel: [*10/27/2023 13:42:25.8496] Re-Tx Count=3, Max Re-Tx Value=8, SendSeqNum=11, NumofPendingMsgs=5
Oct 27 13:42:25 kernel: [*10/27/2023 13:42:25.8496]
Oct 27 13:42:30 kernel: [*10/27/2023 13:42:30.6016] Re-Tx Count=4, Max Re-Tx Value=8, SendSeqNum=11, NumofPendingMsgs=5
Oct 27 13:42:30 kernel: [*10/27/2023 13:42:30.6016]
Oct 27 13:42:35 kernel: [*10/27/2023 13:42:35.3526] Re-Tx Count=5, Max Re-Tx Value=8, SendSeqNum=16, NumofPendingMsgs=10
Oct 27 13:42:35 kernel: [*10/27/2023 13:42:35.3526]
Oct 27 13:42:40 kernel: [*10/27/2023 13:42:40.1046] Re-Tx Count=6, Max Re-Tx Value=8, SendSeqNum=19, NumofPendingMsgs=13
Oct 27 13:42:40 kernel: [*10/27/2023 13:42:40.1046]
Oct 27 13:42:44 kernel: [*10/27/2023 13:42:44.8566] Re-Tx Count=7, Max Re-Tx Value=8, SendSeqNum=20, NumofPendingMsgs=14
Oct 27 13:42:44 kernel: [*10/27/2023 13:42:44.8566]
Oct 27 13:42:49 kernel: [*10/27/2023 13:42:49.6076] Re-Tx Count=8, Max Re-Tx Value=8, SendSeqNum=21, NumofPendingMsgs=15
Oct 27 13:42:49 kernel: [*10/27/2023 13:42:49.6076]
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Max retransmission count exceeded, going back to DISCOVER mode.
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 464, eleLen = 472, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ...Vendor SubType: AP_CDP_CACHE_PAYLOAD(24) len: 460 vendId 409600
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 3083, eleLen = 3091, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 3083, eleLen = 3091, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 14, eleLen = 22, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 14, eleLen = 22, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 35, eleLen = 43, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ....TLV: TLV_AP_THROUGHPUT_PAYLOAD(745), level: 0, seq: 0, nested: true
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 35, eleLen = 43, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ....TLV: TLV_AP_THROUGHPUT_PAYLOAD(745), level: 0, seq: 0, nested: true
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 104, eleLen = 112, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ....TLV: TLV_INACTIVE_CLIENT_DATA_PAYLOAD(2213), level: 0, seq: 0, nested: true
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 124, eleLen = 132, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ....TLV: TLV_INACTIVE_CLIENT_DATA_PAYLOAD(2213), level: 0, seq: 0, nested: true
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_ECHO_REQUEST, type = 1, len = 0, eleLen = 8, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 464, eleLen = 472, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ...Vendor SubType: AP_CDP_CACHE_PAYLOAD(24) len: 460 vendId 409600
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 863, eleLen = 871, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 38, eleLen = 46, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ....TLV: TLV_ECHOPAYLOAD(42), level: 0, seq: 0, nested: true
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 64, eleLen = 72, sendSeqNum = 24
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3596] ....TLV: TLV_APETHERINTFPAYLOAD(46), level: 0, seq: 0, nested: true
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.3606] Flexconnect Switching to Standalone Mode!
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.6206]
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.6206] CAPWAP State: DTLS Teardown
Oct 27 13:42:54 upgrade: Script called with args:[CANCEL]
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.7076] status 'upgrade.sh: Script called with args:[CANCEL]'
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.7606] do CANCEL, part2 is active part
Oct 27 13:42:54 upgrade: Cleanup tmp files ...
Oct 27 13:42:54 kernel: [*10/27/2023 13:42:54.7806] status 'upgrade.sh: Cleanup tmp files ...'

Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.3295] systemd[1]: Starting dhcpv6 client watcher...
Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.3605] systemd[1]: Stopping DHCPv6 client...
Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.3615] Discovery Response from 10.16.4.250
Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.3625] Discovery Response from 10.16.4.250
Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.3695] systemd[1]: Starting DHCPv6 client...
Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.4105] systemd[1]: Started DHCPv6 client.
Oct 27 13:43:09 kernel: [*10/27/2023 13:43:09.4415] systemd[1]: Started dhcpv6 client watcher.
Oct 27 13:43:18 kernel: [*10/27/2023 13:43:18.6935]
Oct 27 13:43:18 kernel: [*10/27/2023 13:43:18.6935] CAPWAP State: DTLS Setup
Oct 27 13:43:18 kernel: [*10/27/2023 13:43:18.8275] dtls_verify_server_cert: Controller certificate verification successful
Oct 27 13:43:19 kernel: [*10/27/2023 13:43:19.1565]
Oct 27 13:43:19 kernel: [*10/27/2023 13:43:19.1565] CAPWAP State: Join
Oct 27 13:43:19 kernel: [*10/27/2023 13:43:19.1795] Sending Join request to 10.16.4.250 through port 5249
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.4635] systemd[1]: Starting Lighttpd Watcher...
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.4935] systemd[1]: Started Lighttpd Watcher.
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.8215] Sending Join request to 10.16.4.250 through port 5249
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.8255] Join Response from 10.16.4.250
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.8255] AC accepted join request with result code: 0
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.8605] Received wlcType 0, timer 30
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.9025]
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.9025] CAPWAP State: Image Data
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.9025] AP image version 17.6.4.56 backup 8.10.130.0, Controller 17.6.4.56
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.9025] Version is the same, do not need update.
Oct 27 13:43:23 upgrade: Script called with args:[NO_UPGRADE]
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.9475] status 'upgrade.sh: Script called with args:[NO_UPGRADE]'
Oct 27 13:43:23 kernel: [*10/27/2023 13:43:23.9975] do NO_UPGRADE, part2 is active part
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.0035]
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.0035] CAPWAP State: Configure
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.0055] capwapd: reading file /click/nss_lag_control/lacp_state: No such file or directory
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.0055] capwapd: reading file /click/nss_lag_control/capwap_state: No such file or directory
Oct 27 13:43:24 capwapd[5616]: Check lagloadbalance setting flex_mode 1 cfg 0 linkstate 1 ap_type 83
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.7795]
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.7795] CAPWAP State: Run
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.8375] AP has joined controller TDL-WIFICTRL-01
Oct 27 13:43:24 kernel: [*10/27/2023 13:43:24.9835] Flexconnect Switching to Connected Mode!
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.6005] Previous AP mode is 2, change to 2
Oct 27 13:43:25 capwapd[5616]: Check lagloadbalance setting flex_mode 1 cfg 0 linkstate 1 ap_type 83
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.6325] capwapd: reading file /click/nss_lag_control/lacp_state: No such file or directory
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.6325] capwapd: reading file /click/nss_lag_control/capwap_state: No such file or directory
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.8365] Current session mode: ssh, Configured: Telnet-No, SSH-Yes, Console-Yes
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.8365]
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.8365] Current session mode: telnet, Configured: Telnet-No, SSH-Yes, Console-Yes
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.8365]
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.9085] Current session mode: console, Configured: Telnet-No, SSH-Yes, Console-Yes
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.9085]
Oct 27 13:43:25 chpasswd: password for user changed
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.9405] chpasswd: password for user changed
Oct 27 13:43:25 chpasswd: password for user changed
Oct 27 13:43:25 kernel: [*10/27/2023 13:43:25.9595] chpasswd: password for user changed
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1085] systemd[1]: Starting Cisco syslogd watcher...
Oct 27 13:43:26 syslogd exiting
Oct 27 13:43:26 syslogd started: BusyBox v1.32.1
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1735] systemd[1]: Started Cisco syslog service.
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1875] systemd[1]: Started Cisco syslogd watcher.
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1905]
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1905] Same LSC mode, no action needed
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1935] CLSM[00:00:00:00:00:00]: U3 Client RSSI Stats feature is deprecated; can no longer be enabled
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.1975] systemd[1]: Starting ntp file watcher...
Oct 27 13:43:26 ntp_update: NTP: Fri Oct 27 13:43:26 2023 :Can not create ntp process log file.
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.2115] Update NTP source to WLC
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.2245] systemd[1]: Started ntp file watcher.
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5335] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5335] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5355] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5355] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5375] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5375] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5385] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5385] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5405] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5405] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5425] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5425] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5645] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5645] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5655] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5655] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5675] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5675] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5695] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5695] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5715] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5715] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5725] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5725] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5745] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5745] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5765] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.5765] not a valid cache_type in add_cache_param_via_tlv
Oct 27 13:43:26 kernel: [*10/27/2023 13:43:26.9285] Got WSA Server config TLVs
Oct 27 13:43:28 kernel: [*10/27/2023 13:43:28.5245] systemd[1]: Starting Lighttpd Watcher...
Oct 27 13:43:28 kernel: [*10/27/2023 13:43:28.5645] systemd[1]: Started Lighttpd Watcher.
Oct 27 13:43:28 kernel: [*10/27/2023 13:43:28.9725] systemd[1]: Starting nbar watcher...
Oct 27 13:43:29 kernel: [*10/27/2023 13:43:29.0185] systemd[1]: Started nbar watcher.
Oct 27 13:43:29 kernel: [*10/27/2023 13:43:29.4365] systemd[1]: Starting nbar watcher...
Oct 27 13:43:29 kernel: [*10/27/2023 13:43:29.4645] systemd[1]: Started nbar watcher.
Oct 27 13:43:31 kernel: [*10/27/2023 13:43:31.5175] systemd[1]: Starting nbar watcher...
Oct 27 13:43:31 kernel: [*10/27/2023 13:43:31.5465] systemd[1]: Started nbar watcher.
Oct 27 13:43:31 kernel: [*10/27/2023 13:43:31.9585] systemd[1]: Starting nbar watcher...
Oct 27 13:43:31 kernel: [*10/27/2023 13:43:31.9855] systemd[1]: Started nbar watcher.
Oct 27 13:43:32 kernel: [*10/27/2023 13:43:32.4685] AP tag change to TAG-POL-TDH
Oct 27 13:43:32 kernel: [*10/27/2023 13:43:32.5365] flags value is 1 process iot_radio
Oct 27 13:43:32 root: BLE reset lock acquired
Oct 27 13:43:32 kernel: [*10/27/2023 13:43:32.7895] Powering down BLE radio
Oct 27 13:43:34 root: released BLE reset lock
Oct 27 13:43:34 kernel: [*10/27/2023 13:43:34.4715] systemd[1]: Starting nbar watcher...
Oct 27 13:43:34 kernel: [*10/27/2023 13:43:34.5055] systemd[1]: Started nbar watcher.
Oct 27 13:43:53 NCI: CLEANAIR: Slot 0 admin disabled
Oct 27 13:43:53 kernel: [*10/27/2023 13:43:53.0625] set cleanair [slot0][band0] disable
Oct 27 13:43:53 kernel: [*10/27/2023 13:43:53.0645] set cleanair [slot1][band1] disable
Oct 27 13:43:55 NCI: CLEANAIR: Slot 1 admin disabled
Oct 27 13:45:11 sshd[14133]: error: Could not get shadow information for admin
Oct 27 13:45:11 sshd[14133]: Accepted password for admin from 10.16.33.139 port 52761 ssh2
Oct 27 13:45:12 FIPS[14265]: *** shell: FIPS Mode = disabled ***

sumithliyanage · ‎10-27-2023

another AP--TDL-Ground-Front-Office

The following Access Point went DOWN on 27 October 2023 at 15:33: -

Access Point: TDL-Ground-Front-Office

IP Address: 10.16.23.3

==============================

after exactly the 10 min

The following Access Point came UP on 27 October 2023 at 15:43: -

Access Point: TDL-Ground-Front-Office

IP Address: 10.16.23.3

=======================================

Oct 27 14:31:51 kernel: [*10/27/2023 14:31:51.3396] Re-Tx Count=1, Max Re-Tx Value=8, SendSeqNum=172, NumofPendingMsgs=1
Oct 27 14:31:51 kernel: [*10/27/2023 14:31:51.3396]
Oct 27 14:31:56 kernel: [*10/27/2023 14:31:56.0916] Re-Tx Count=2, Max Re-Tx Value=8, SendSeqNum=174, NumofPendingMsgs=3
Oct 27 14:31:56 kernel: [*10/27/2023 14:31:56.0916]
Oct 27 14:32:00 kernel: [*10/27/2023 14:32:00.8435] Re-Tx Count=3, Max Re-Tx Value=8, SendSeqNum=175, NumofPendingMsgs=4
Oct 27 14:32:00 kernel: [*10/27/2023 14:32:00.8435]
Oct 27 14:32:05 kernel: [*10/27/2023 14:32:05.5955] Re-Tx Count=4, Max Re-Tx Value=8, SendSeqNum=176, NumofPendingMsgs=5
Oct 27 14:32:05 kernel: [*10/27/2023 14:32:05.5955]
Oct 27 14:32:10 kernel: [*10/27/2023 14:32:10.3474] Re-Tx Count=5, Max Re-Tx Value=8, SendSeqNum=179, NumofPendingMsgs=8
Oct 27 14:32:10 kernel: [*10/27/2023 14:32:10.3474]
Oct 27 14:32:15 kernel: [*10/27/2023 14:32:15.0984] Re-Tx Count=6, Max Re-Tx Value=8, SendSeqNum=182, NumofPendingMsgs=11
Oct 27 14:32:15 kernel: [*10/27/2023 14:32:15.0984]
Oct 27 14:32:19 kernel: [*10/27/2023 14:32:19.8504] Re-Tx Count=7, Max Re-Tx Value=8, SendSeqNum=186, NumofPendingMsgs=15
Oct 27 14:32:19 kernel: [*10/27/2023 14:32:19.8504]
Oct 27 14:32:24 kernel: [*10/27/2023 14:32:24.6013] Re-Tx Count=8, Max Re-Tx Value=8, SendSeqNum=186, NumofPendingMsgs=15
Oct 27 14:32:24 kernel: [*10/27/2023 14:32:24.6013]
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Max retransmission count exceeded, going back to DISCOVER mode.
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 568, eleLen = 576, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] ...Vendor SubType: AP_CDP_CACHE_PAYLOAD(24) len: 564 vendId 409600
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 464, eleLen = 472, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] ...Vendor SubType: AP_CDP_CACHE_PAYLOAD(24) len: 460 vendId 409600
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 34, eleLen = 42, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] ....TLV: TLV_INACTIVE_CLIENT_DATA_PAYLOAD(2213), level: 0, seq: 0, nested: true
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 334, eleLen = 342, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 334, eleLen = 342, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 158, eleLen = 166, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_ECHO_REQUEST, type = 1, len = 0, eleLen = 8, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 14, eleLen = 22, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 14, eleLen = 22, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 20, eleLen = 28, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 464, eleLen = 472, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] ...Vendor SubType: AP_CDP_CACHE_PAYLOAD(24) len: 460 vendId 409600
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] Dropping msg CAPWAP_WTP_EVENT_REQUEST, type = 34, len = 34, eleLen = 42, sendSeqNum = 188
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3533] ....TLV: TLV_INACTIVE_CLIENT_DATA_PAYLOAD(2213), level: 0, seq: 0, nested: true
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.3543] Flexconnect Switching to Standalone Mode!
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.6113]
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.6113] CAPWAP State: DTLS Teardown
Oct 27 14:32:29 upgrade: Script called with args:[CANCEL]
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.6913] status 'upgrade.sh: Script called with args:[CANCEL]'
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.7363] do CANCEL, part2 is active part
Oct 27 14:32:29 upgrade: Cleanup tmp files ...
Oct 27 14:32:29 kernel: [*10/27/2023 14:32:29.7573] status 'upgrade.sh: Cleanup tmp files ...'
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.3222] systemd[1]: Starting dhcpv6 client watcher...
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.3472] Discovery Response from 10.16.4.250
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.3472] Discovery Response from 10.16.4.250
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.3552] systemd[1]: Stopping DHCPv6 client...
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.3652] systemd[1]: Starting DHCPv6 client...
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.3982] systemd[1]: Started DHCPv6 client.
Oct 27 14:32:44 kernel: [*10/27/2023 14:32:44.4192] systemd[1]: Started dhcpv6 client watcher.
Oct 27 14:32:53 kernel: [*10/27/2023 14:32:53.6871]
Oct 27 14:32:53 kernel: [*10/27/2023 14:32:53.6871] CAPWAP State: DTLS Setup
Oct 27 14:32:53 kernel: [*10/27/2023 14:32:53.8161] dtls_verify_server_cert: Controller certificate verification successful
Oct 27 14:32:54 kernel: [*10/27/2023 14:32:54.1611]
Oct 27 14:32:54 kernel: [*10/27/2023 14:32:54.1611] CAPWAP State: Join
Oct 27 14:32:54 kernel: [*10/27/2023 14:32:54.1851] Sending Join request to 10.16.4.250 through port 5278
Oct 27 14:32:55 kernel: [*10/27/2023 14:32:55.9781] systemd[1]: Starting Lighttpd Watcher...
Oct 27 14:32:56 kernel: [*10/27/2023 14:32:56.0051] systemd[1]: Started Lighttpd Watcher.
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.8150] Sending Join request to 10.16.4.250 through port 5278
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.8190] Join Response from 10.16.4.250
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.8190] AC accepted join request with result code: 0
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.8540] Received wlcType 0, timer 30
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.8930]
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.8930] CAPWAP State: Image Data
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.9460] AP image version 17.6.4.56 backup 8.10.130.0, Controller 17.6.4.56
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.9460] Version is the same, do not need update.
Oct 27 14:32:58 upgrade: Script called with args:[NO_UPGRADE]
Oct 27 14:32:58 kernel: [*10/27/2023 14:32:58.9860] status 'upgrade.sh: Script called with args:[NO_UPGRADE]'
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.0310] do NO_UPGRADE, part2 is active part
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.0370]
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.0370] CAPWAP State: Configure
Oct 27 14:32:59 capwapd[5482]: Check lagloadbalance setting flex_mode 1 cfg 0 linkstate 1 ap_type 83
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.0400] capwapd: reading file /click/nss_lag_control/lacp_state: No such file or directory
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.0400] capwapd: reading file /click/nss_lag_control/capwap_state: No such file or directory
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.8890]
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.8890] CAPWAP State: Run
Oct 27 14:32:59 kernel: [*10/27/2023 14:32:59.9400] AP has joined controller TDL-WIFICTRL-01
Oct 27 14:33:00 kernel: [*10/27/2023 14:33:00.1000] Flexconnect Switching to Connected Mode!
Oct 27 14:33:00 kernel: [*10/27/2023 14:33:00.8200] Previous AP mode is 2, change to 2
Oct 27 14:33:00 capwapd[5482]: Check lagloadbalance setting flex_mode 1 cfg 0 linkstate 1 ap_type 83
Oct 27 14:33:00 kernel: [*10/27/2023 14:33:00.8220] capwapd: reading file /click/nss_lag_control/lacp_state: No such file or directory
Oct 27 14:33:00 kernel: [*10/27/2023 14:33:00.8220] capwapd: reading file /click/nss_lag_control/capwap_state: No such file or directory
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.0310] systemd[1]: Starting Lighttpd Watcher...
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.0340] Current session mode: ssh, Configured: Telnet-No, SSH-Yes, Console-Yes
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.0340]
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.0340] Current session mode: telnet, Configured: Telnet-No, SSH-Yes, Console-Yes
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.0340]
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.0650] systemd[1]: Started Lighttpd Watcher.
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.1110] Current session mode: console, Configured: Telnet-No, SSH-Yes, Console-Yes
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.1110]
Oct 27 14:33:01 chpasswd: password for user changed
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.1410] chpasswd: password for user changed
Oct 27 14:33:01 chpasswd: password for user changed
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.1580] chpasswd: password for user changed
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.3100] systemd[1]: Starting Cisco syslogd watcher...
Oct 27 14:33:01 syslogd exiting
Oct 27 14:33:01 syslogd started: BusyBox v1.32.1
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.3840] systemd[1]: Started Cisco syslogd watcher.
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.3940]
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.3940] Same LSC mode, no action needed
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.3990] systemd[1]: Starting ntp file watcher...
Oct 27 14:33:01 ntp_update: NTP: Fri Oct 27 14:33:01 2023 :Can not create ntp process log file.
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.4130] Update NTP source to WLC
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.4280] systemd[1]: Started ntp file watcher.
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.4460] CLSM[00:00:00:00:00:00]: U3 Client RSSI Stats feature is deprecated; can no longer be enabled
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8190] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8190] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8200] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8200] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8210] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8210] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8230] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8230] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8250] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8250] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8260] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8260] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8500] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8500] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8510] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8510] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8530] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8530] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8540] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8550] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8560] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8560] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8580] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8580] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8590] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8590] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8610] In write handler 'add_cache_param_via_tlv' for 'pmk_tracker :: PMKTracker':
Oct 27 14:33:01 kernel: [*10/27/2023 14:33:01.8610] not a valid cache_type in add_cache_param_via_tlv
Oct 27 14:33:02 kernel: [*10/27/2023 14:33:02.2330] Got WSA Server config TLVs
Oct 27 14:33:04 kernel: [*10/27/2023 14:33:04.3510] systemd[1]: Starting nbar watcher...
Oct 27 14:33:04 kernel: [*10/27/2023 14:33:04.4030] systemd[1]: Started nbar watcher.
Oct 27 14:33:04 kernel: [*10/27/2023 14:33:04.8000] systemd[1]: Starting nbar watcher...
Oct 27 14:33:04 kernel: [*10/27/2023 14:33:04.8290] systemd[1]: Started nbar watcher.
Oct 27 14:33:06 kernel: [*10/27/2023 14:33:06.0970] systemd[1]: Starting Lighttpd Watcher...
Oct 27 14:33:06 kernel: [*10/27/2023 14:33:06.1250] systemd[1]: Started Lighttpd Watcher.
Oct 27 14:33:06 kernel: [*10/27/2023 14:33:06.4490] systemd[1]: Starting nbar watcher...
Oct 27 14:33:06 kernel: [*10/27/2023 14:33:06.4780] systemd[1]: Started nbar watcher.
Oct 27 14:33:06 kernel: [*10/27/2023 14:33:06.9080] systemd[1]: Starting nbar watcher...
Oct 27 14:33:06 kernel: [*10/27/2023 14:33:06.9340] systemd[1]: Started nbar watcher.
Oct 27 14:33:08 kernel: [*10/27/2023 14:33:08.1070] AP tag change to TAG-POL-TDH
Oct 27 14:33:08 kernel: [*10/27/2023 14:33:08.1980] flags value is 1 process iot_radio
Oct 27 14:33:08 root: BLE reset lock acquired
Oct 27 14:33:08 kernel: [*10/27/2023 14:33:08.4360] Powering down BLE radio
Oct 27 14:33:09 kernel: [*10/27/2023 14:33:09.6600] systemd[1]: Starting nbar watcher...
Oct 27 14:33:09 kernel: [*10/27/2023 14:33:09.6930] systemd[1]: Started nbar watcher.
Oct 27 14:33:09 root: released BLE reset lock
Oct 27 14:33:27 NCI: CLEANAIR: Slot 0 admin disabled
Oct 27 14:33:27 kernel: [*10/27/2023 14:33:27.2598] set cleanair [slot0][band0] disable
Oct 27 14:33:27 kernel: [*10/27/2023 14:33:27.2628] set cleanair [slot1][band1] disable
Oct 27 14:33:29 NCI: CLEANAIR: Slot 1 admin disabled
Oct 27 14:42:27 kernel: [*10/27/2023 14:42:27.2624] DOT11_DRV[1]: off_chhannel RX timeout, ch = 116
Oct 27 14:54:40 sshd[8437]: error: Could not get shadow information for admin
Oct 27 14:54:40 sshd[8437]: Accepted password for admin from 10.16.33.139 port 56215 ssh2
Oct 27 14:54:40 FIPS[8539]: *** shell: FIPS Mode = disabled ***

Leo Laohoo · ‎10-27-2023

I recommend upgrading to either 17.6.6a or 17.9.4a.

marce1000 · ‎10-27-2023

- If you speak about things happening every 10 mins , I more think about external network events (surge) e.g :

Anyway : 1) What software version is the c9800-48 running ?

2) Have a checkup of the c9800-48 configuration with the CLI command show tech wireless ; feed the output into
Wireless Config Analyzer

3) Troubleshoot AP joining issues according to : https://logadvisor.cisco.com/logadvisor/wireless/9800/9800APJoin

4) Check APs with : https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/217738-monitor-catalyst-9800-kpis-key-performa.html#anc4

M.

-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

sumithliyanage · ‎10-29-2023

this isn't happening every 10 min, the alert reset time is 10 min. we have 93 APs connected with the C9800-40 controller and randomly APs are going down and rejoining without breaking the session. this means the uptime of AP and rejoin time will be different. Please attached excel file

sumithliyanage · ‎10-29-2023

ROM: 16.10(2r)

Cisco IOS XE Software, Version 17.06.04
Cisco IOS Software [Bengaluru], C9800 Software (C9800_IOSXE-K9), Version 17.6.4, RELEASE SOFTWARE (fc1)

marce1000 · ‎10-29-2023

>...Cisco IOS XE Software, Version 17.06.04 (ROM: 16.10(2r))
- Upgrade to latest advisory release : https://software.cisco.com/download/home/286316412/type/282046477/release/Cupertino-17.9.4a
Take care -> From https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-9/release-notes/rn-17-9-9800.html
>...Before upgrading the Cisco Catalyst 9800-40 Series Wireless Controller image to Cisco IOS XE Cupertino 17.9.x using bundle mode, you must ensure that the ROMMON version is 17.7(3r) or later.
For other platforms, such as Cisco Catalyst 9800-80 Series Wireless Controllers, you should upgrade to 17.3(3r) and for Cisco Catalyst 9800-L Series Wireless Controllers, we recommend that you upgrade the ROMMON version to 16.12(3r) or later.
After the upgrade, you cannot downgrade to older ROMMON versions.

M.

-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

sumithliyanage · ‎10-29-2023

is possible to upgrade Bengaluru to Cupertino 17.9.x? what is the different between these versions?

marce1000 · ‎10-29-2023

- 17.9.4a ; also contains the fix for https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-9/release-notes/rn-17-9-9800.html#whatsnew1794

M.

-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

Rich R · ‎10-27-2023

Get a radioactive trace (with internal logs) of the AP ethernet MAC address and packet captures at the AP switch port and on the WLC and then open a TAC case. Also provide "show tech wireless" from WLC and "show tech" and "show log" from the AP. They'll likely ask for more debugs but that's a start.

Meanwhile I'd say you're probably hitting https://bst.cisco.com/bugsearch/bug/CSCwh61011 so I would try installing APSP2 which might help: https://software.cisco.com/download/home/286316412/type/286325254/release/17.9.4

Just noticed you're still on 17.6.4 so you'd have to upgrade to 17.9.4 for a start then install the APSP.

------------------------------
Please click Helpful if this post helped you and Select as Solution (drop down menu at top right of this reply) if this answered your query.
------------------------------
TAC recommended codes for AireOS WLC's and TAC recommended codes for 9800 WLC's
Best Practices for AireOS WLC's, Best Practices for 9800 WLC's and Cisco Wireless compatibility matrix
Check your 9800 WLC config with Wireless Config Analyzer using "show tech wireless" output or "config paging disable" then "show run-config" output on AireOS and use Wireless Debug Analyzer to analyze your WLC client debugs
Field Notice: FN63942 APs and WLCs Fail to Create CAPWAP Connections Due to Certificate Expiration
Field Notice: FN72424 Later Versions of WiFi 6 APs Fail to Join WLC - Software Upgrade Required
Field Notice: FN72524 IOS APs stuck in downloading state after 4 Dec 2022 due to Certificate Expired
- Fixed in 8.10.196.0, latest 9800 releases, 8.5.182.12 (8.5.182.13 for 3504) and 8.5.182.109 (IRCM, 8.5.182.111 for 3504)
Field Notice: FN70479 AP Fails to Join or Joins with 1 Radio due to Country Mismatch, RMA needed
How to avoid boot loop due to corrupted image on Wave 2 and Catalyst 11ax Access Points (CSCvx32806)
Field Notice: FN74035 - Wave2 APs DFS May Not Detect Radar After Channel Availability Check Time
Leo's list of bugs affecting 2800/3800/4800/1560 APs

sumithliyanage · ‎10-29-2023

we have 93 APs associated with the c9800- controller and 2 models of AP

1- C9105AXW-E

2- C9115AXI-E-****only this model is having the issue, I have tested one in the IT office, and it goes less than 10 sec searching and rejoin, then we received the alert on SolarWinds for AP down, then after 10 min receiving the alert for AP is up, this is happening randomly for all AP - C9115AXI-E, actually ap isn't go down for 10 min alert receiving gap is 10 min for when this happen.

sumithliyanage · ‎10-29-2023

logs

TDL-WIFICTRL-01#sh log
Syslog logging: enabled (0 messages dropped, 3 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.

No Inactive Message Discriminator.

Console logging: level errors, 13748 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 427716 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
Persistent logging: disabled

No active filter modules.

Trap logging: level errors, 13748 message lines logged
Logging to 10.16.0.30 (udp port 514, audit disabled,
link up),
13747 message lines logged,
0 message lines rate-limited,
0 message lines dropped-by-MD,
xml disabled, sequence number disabled
filtering disabled
Logging Source-Interface: VRF Name:
TLS Profiles:

Log Buffer (4096 bytes):
4-MAC_CONFLICT: Dynamic mac 7EE0.CAED.F6B1 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:37:52.898: %SYS-6-TTY_EXPIRE_TIMER: (exec timer expired, tty 2 (10.16.33.139)), user admin
Oct 29 09:37:52.898: %SYS-6-LOGOUT: User admin has exited tty session 2(10.16.33.139)
Oct 29 09:38:46.303: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:029 TS:00015036206296201920 %SWPORT-4-MAC_CONFLICT: Dynamic mac AEAD.9F46.978C from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:39:46.334: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:161 TS:00015036266330381472 %SWPORT-4-MAC_CONFLICT: Dynamic mac 7EE0.CAED.F6B1 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:40:46.374: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:107 TS:00015036326372472307 %SWPORT-4-MAC_CONFLICT: Dynamic mac 7EE0.CAED.F6B1 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:41:46.403: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:077 TS:00015036386404969559 %SWPORT-4-MAC_CONFLICT: Dynamic mac 7EE0.CAED.F6B1 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:42:46.426: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:068 TS:00015036446430280612 %SWPORT-4-MAC_CONFLICT: Dynamic mac 7EE0.CAED.F6B1 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:43:46.544: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:199 TS:00015036506551018505 %SWPORT-4-MAC_CONFLICT: Dynamic mac CE8C.BD73.4CB5 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:44:46.586: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:020 TS:00015036566595748588 %SWPORT-4-MAC_CONFLICT: Dynamic mac 96FE.95CA.41A5 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:45:46.668: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:197 TS:00015036626681002134 %SWPORT-4-MAC_CONFLICT: Dynamic mac 8233.7927.AD85 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:46:47.734: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:203 TS:00015036687749444220 %SWPORT-4-MAC_CONFLICT: Dynamic mac 921E.2AD7.97EF from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:47:47.840: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:179 TS:00015036747858088534 %SWPORT-4-MAC_CONFLICT: Dynamic mac 8233.7927.AD85 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:48:11.675: %IOSXE-3-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:005 TS:00015036771694667524 %DTLS-3-REPLAY_ERROR_DTLS: DTLS anti-replay error, src_addr: 10.16.23.33, src_port: 5259, dest_addr: 10.16.4.250, dst_port: 5247
Oct 29 09:48:29.277: %WEBSERVER-5-SESS_LOGOUT: Chassis 1 Successfully logged out from host 10.16.33.139 by user 'admin' using crypto cipher 'ECDHE-RSA-AES128-GCM-SHA256'
Oct 29 09:48:47.874: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:065 TS:00015036807894939278 %SWPORT-4-MAC_CONFLICT: Dynamic mac CE28.FF0A.A493 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:49:48.000: %IOSXE-4-PLATFORM: Chassis 1 R0/0: cpp_cp: QFP:0.0 Thread:152 TS:00015036868023874035 %SWPORT-4-MAC_CONFLICT: Dynamic mac B6E2.C861.CA05 from Port-channel1 conflict with WlClient, please check the network topology and make sure there is no loop.
Oct 29 09:50:36.846: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: admin] [Source: 10.16.33.139] [localport: 22] at 09:50:36 UTC Sun Oct 29 2023

sumithliyanage · ‎10-29-2023

last 24 hr following Aps affected

AP Name Ethernet MAC Radio MAC AP Up Time Association Up Time
---------------------------------------------------------------------------------------------------------------------------------------------------
TDL-Ground-Bar-Snug a00f.3713.ba04 a00f.373b.cfe0 228 days 19 hours 8 minutes 14 seconds 22 hours 2 minutes 33 seconds
1st-corridor-30s-1 a49b.cd2c.88f4 e44e.2dca.47a0 66 days 22 hours 25 minutes 2 seconds 21 hours 58 minutes 35 seconds
TDL-Ground-PR a49b.cd2c.985c e44e.2dca.c2e0 191 days 19 hours 53 minutes 32 seconds 20 hours 53 minutes 46 seconds
TDL-2nd-Corridor-R227 a00f.3713.69c4 a00f.3739.4de0 163 days 0 hour 48 minutes 5 seconds 18 hours 15 minutes 13 seconds
1st-corridor a49b.cd2c.77fc e44e.2dc9.bfe0 113 days 20 hours 30 minutes 4 seconds 17 hours 42 minutes 54 seconds
TDL-2nd-Corridor-R239 a00f.3713.8d5c a00f.373a.6aa0 30 days 1 hour 8 minutes 39 seconds 16 hours 49 minutes 56 seconds
TDL-Ground-Promenade-Station-3 a49b.cd2c.99f0 e44e.2dca.cf80 206 days 3 hours 24 minutes 9 seconds 15 hours 17 minutes 19 seconds
APA00F.3713.9FB8 a00f.3713.9fb8 a00f.373a.fd80 87 days 19 hours 16 minutes 3 seconds 14 hours 35 minutes 5 seconds
TDL-Ground-Bar a00f.3713.be58 a00f.373b.f280 57 days 18 hours 0 minute 35 seconds 14 hours 33 minutes 2 seconds
TDL-Ground-Reception-Desk a49b.cd2c.9190 e44e.2dca.8c80 177 days 1 hour 52 minutes 43 seconds 12 hours 39 minutes 46 seconds
TDL-Promonad-GR a49b.cd2c.8b98 e44e.2dca.5cc0 44 days 23 hours 30 minutes 5 seconds 12 hours 12 minutes 20 seconds
TDL-Ground-Concierge-Desk a49b.cd2c.943c e44e.2dca.a1c0 177 days 18 hours 47 minutes 32 seconds 7 hours 53 minutes 21 seconds
1st-corridor-2 a49b.cd2c.72bc e44e.2dc9.95e0 113 days 20 hours 30 minutes 18 seconds 6 hours 46 minutes 13 seconds
TDL-Ground-Bar-Washrooms a00f.3713.74a0 a00f.3739.a4c0 138 days 17 hours 38 minutes 2 seconds 1 hour 48 minutes 34 seconds
TDL-2nd-Corridor-Lift-R205 a00f.3713.a16c a00f.373b.0b20 163 days 0 hour 50 minutes 52 seconds 41 minutes 45 seconds

sumithliyanage · ‎10-29-2023

TDL-WIFICTRL-01#show wireless stats ap session termination
Event Previous State Occurance Count
------------------------------------------------------------------------------------
NO_EVENT Unknown 5372
DTLS session closed INIT 15
Heartbeat timer expiry JOIN_WAIT 1
Reset by API JOIN_PROCESS 2
DTLS session closed JOINED 2
Image download status IMAGE_DOWNLOAD 12
Reset by API RUN 69
DTLS session closed RUN 94
Heartbeat timer expiry RUN 226
Message timer expiry RUN 191

sumithliyanage · ‎10-29-2023

TDL-WIFICTRL-01#
TDL-WIFICTRL-01#show wireless stats ap history | i Disjoined
AP00B0.E186.48F8 00b0.e187.8a80 Disjoined 06/16/23 13:36:32 06/16/23 13:37:54 Unsupported ap 2
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 10/27/23 17:55:50 NA App send req failed 1
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 10/13/23 15:49:29 NA Tag modified 1
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 10/13/23 15:11:59 NA Data DTLS state change 1
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 10/13/23 15:07:34 NA Wtp Clear config 1
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 09/13/23 14:59:32 NA DTLS close alert from peer 1
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 09/13/23 14:43:48 NA App send req failed 1
TDL-Ground-Promenade-Station-1 a00f.3737.5440 Disjoined 09/13/23 14:04:04 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/28/23 16:51:33 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/27/23 09:12:24 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/26/23 14:20:18 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 19:05:15 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 17:28:50 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 13:58:22 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 13:50:14 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 12:20:27 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 11:48:10 NA App send req failed 1
APA00F.3713.69C4 a00f.3739.4de0 Disjoined 10/25/23 11:34:39 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/24/23 15:56:25 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/22/23 08:51:32 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/21/23 13:28:17 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/20/23 12:40:43 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/18/23 17:25:51 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/18/23 17:12:56 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/05/23 11:42:38 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/04/23 13:56:51 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/03/23 16:42:38 NA App send req failed 1
1st-corridor-3 a00f.3739.6540 Disjoined 10/03/23 09:19:10 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/25/23 10:29:49 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/19/23 10:12:24 NA Wtp reset config cmd sent 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/19/23 00:44:33 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 23:05:44 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 22:59:47 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 22:34:07 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 22:29:20 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 22:12:12 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 22:02:11 NA App send req failed 1
TDL-Ground-Bar-Upper-Behind-Bar a00f.3739.9440 Disjoined 10/18/23 21:33:34 01/01/70 00:00:00 App send req failed 10
TDL-Ground-Garden a00f.3739.9c60 Disjoined 09/13/23 14:59:35 NA DTLS close alert from peer 1
TDL-Ground-Garden a00f.3739.9c60 Disjoined 08/02/23 15:48:31 NA Tag modified 1
TDL-Ground-Garden a00f.3739.9c60 Disjoined 08/02/23 13:08:43 NA Data DTLS state change 1
TDL-Ground-Garden a00f.3739.9c60 Disjoined 08/02/23 13:05:14 NA DTLS close alert from peer 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/29/23 08:18:07 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/29/23 00:57:24 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 22:09:21 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 21:37:44 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 19:06:00 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 19:02:36 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 18:27:55 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 16:31:06 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 16:10:35 NA App send req failed 1
TDL-Ground-Bar-Washrooms a00f.3739.a4c0 Disjoined 10/28/23 13:25:51 01/01/70 00:00:00 App send req failed 10
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/28/23 18:16:53 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/28/23 14:47:12 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/28/23 08:51:37 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/27/23 20:04:07 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/27/23 08:58:31 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/25/23 18:58:04 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/25/23 13:37:11 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/25/23 08:34:30 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/24/23 15:39:26 NA App send req failed 1
APA00F.3713.8D5C a00f.373a.6aa0 Disjoined 10/24/23 09:33:49 NA App send req failed 1
APA00F.3713.904C a00f.373a.8220 Disjoined 10/09/23 11:43:43 NA App send req failed 1
APA00F.3713.904C a00f.373a.8220 Disjoined 09/25/23 14:47:03 NA App send req failed 1
APA00F.3713.904C a00f.373a.8220 Disjoined 09/14/23 09:29:37 NA App send req failed 1
APA00F.3713.904C a00f.373a.8220 Disjoined 09/13/23 14:59:47 NA DTLS close alert from peer 1
APA00F.3713.904C a00f.373a.8220 Disjoined 09/08/23 14:46:15 NA App send req failed 1
APA00F.3713.904C a00f.373a.8220 Disjoined 09/06/23 12:57:59 NA App send req failed 1
APA00F.3713.904C a00f.373a.8220 Disjoined 07/03/23 14:43:47 NA Heart beat timer expiry 1
APA00F.3713.904C a00f.373a.8220 Disjoined 05/19/23 10:17:38 NA Data DTLS state change 1
APA00F.3713.904C a00f.373a.8220 Disjoined 05/19/23 10:16:49 NA Tag modified 1
APA00F.3713.904C a00f.373a.8220 Disjoined 05/17/23 12:21:45 NA Image Download Success 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 10/24/23 17:57:50 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 10/13/23 15:07:06 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 10/11/23 08:13:21 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 10/09/23 14:00:37 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 10/05/23 20:25:24 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 09/30/23 10:29:30 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 09/29/23 14:45:18 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 09/21/23 13:05:03 NA App send req failed 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 09/13/23 14:59:31 NA DTLS close alert from peer 1
APA00F.3713.91D0 a00f.373a.8e40 Disjoined 09/08/23 11:11:17 NA App send req failed 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 10/12/23 11:25:34 NA Heart beat timer expiry 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 10/04/23 10:54:05 NA Tag modified 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 09/29/23 12:04:23 NA Heart beat timer expiry 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 09/23/23 08:59:34 NA Heart beat timer expiry 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 09/14/23 10:55:59 NA Tag modified 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 09/14/23 10:30:24 NA Site tag config modified 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 09/14/23 10:18:16 NA Tag modified 1
TDL-Systems-Test-9ED0 a00f.373a.f640 Disjoined 09/14/23 10:13:42 NA Site tag config modified 1