AP1852 migration from mobility express to 9800-CL

[lalitkumar88551]

Hi All,

please help and suggest. i have create a test lab before execute live envirment.  iam faceing  issue client system disconnect from internet within 1-2 min. below find log from AP & wlc. can you suggest what was the reason. as i checked log from wlc9800-cl showing AP join & disjoin. 

Log file attached below in attachement. 

WLC

Aug 18 11:30:53.788: CAPWAP-IDB : CREATE STATUS: Successfully sent IPC Message
Aug 18 11:30:53.765: %CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 1 R0/0: wncd: AP Event: AP Name: APDCCE-C130-6200, MAC: dcce.c130.6200 Joined
Aug 18 11:32:47.023: CAPWAP-IDB : Received message from wncd
Aug 18 11:32:47.024: CAPWAP-IDB : DELETE: Handler called. Msg : 56375E397B38
Aug 18 11:32:47.024: CAPWAP-IDB : DELETE: Processing message : 56375E397B38
Aug 18 11:32:47.026: CAPWAP-IDB : 90000007 - DELETE : Capwap tunnel
Aug 18 11:32:47.026: CAPWAP-IDB : Deleting unicast port 33
Aug 18 11:32:47.028: CAPWAP-IDB : 90000007 - Port link change. Port link state : 0, IDB state : 4
Aug 18 11:32:47.028: CAPWAP-IDB : HWIDB is down for if id 90000007
Aug 18 11:32:47.033: CAPWAP-IDB : IDB SHUTDOWN: Capwap1, State : 6
Aug 18 11:32:47.041: CAPWAP-IDB : 90000007 - DELETE : capwap tunnel delete success
Aug 18 11:32:47.011: %CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 1 R0/0: wncd: AP Event: Session-IP: 192.168.1.185[5256] Mac: dcce.c131.ee80 CAPWAP DTLS session closed for AP, cause: DTLS server session shutdown

Access Point

Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
Aug 18 11:32:47 kernel: [*08/18/2022 11:32:47.0731]

Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).

I am not much familiar about catalysts C9800-CL

[ammahend]

Sounds buggy, what’s the version you are on ? 

[lalitkumar88551]

Wlc running version 17.3.5.42

[ammahend]

do a test, configure the controller IP manually, instead of DNS based discovery and test again

ap name <AP_NAME> controller primary <WLC name> <Controller IP address>

[lalitkumar88551]

I tried to  configure the controller IP manually in AP . but same issue facing, Access point auto join and disjoin automatically & client received Request time out after 40-50 successfully reply.

[Scott Fella]

I think you need to be more specific on how you are testing. You are moving from ME to a 9800, so the ap would drop all client while upgrading and converting the system from ME to CAPWAP.  The ME guide states that a ME ap when powered up or rebooted will check the local subnet or any dhcp/dns discovery you have and will join a controller if one is found. If not, the ap will transition back to ME. 
I would expect clients to be impacted while migrating AP’s to the 9800 and keep in mind that you would need to migrate all the AP’s at that location because you would not have mobility between the ME and the 9800. 

[lalitkumar88551]

AP convert successfully ME to Capwap & AP also join wirless controller 9800-CL.but ap not stable. Its continue join &disjoin process 1-2 min later.client are also access SSID and internet. But when AP disjoin client also lost connectivity internet and gateway also not reachable. Lost connection at any time after an interval. As I checked  log found DTLS session shutdown & AP join and disjoin logs in controller side. But I don't  know why its session  shutdown.

[Rich R]

1. Is there still a ME controller present?  If so that is not supported at the same time as WLC.

2. Look at possible causes for the disconnect, for example packet loss, link latency, MTU etc. As per https://www.ietf.org/rfc/rfc5415.txt

   Run to DTLS Teardown (p):  This state transition occurs when an error
      has occurred in the DTLS stack, causing the DTLS session to be
      torn down.
      WTP:  The WTP enters this state when it receives one of the
            following DTLS notifications: DTLSAborted,
            DTLSReassemblyFailure, or DTLSPeerDisconnect (see
            Section 2.3.2.2).  The WTP MAY tear down the DTLS session if
            it receives frequent DTLSDecapFailure notifications.  The
            WTP also transitions to this state if the underlying
            reliable transport's RetransmitCount counter has reached the
            MaxRetransmit variable (see Section 4.7).  The WTP starts
            the DTLSSessionDelete timer (see Section 4.7.6).
      AC:   The AC enters this state when it receives one of the
            following DTLS notifications: DTLSAborted,
            DTLSReassemblyFailure, or DTLSPeerDisconnect (see
            Section 2.3.2.2).  The AC MAY tear down the DTLS session if
            it receives frequent DTLSDecapFailure notifications.  The AC
            transitions to this state if the underlying reliable
            transport's RetransmitCount counter has reached the
            MaxRetransmit variable (see Section 4.7).  This state
            transition also occurs when the AC's EchoInterval timer (see
            Section 4.7.7) expires.  The AC starts the DTLSSessionDelete
            timer (see Section 4.7.6).

You should also try a factory default reset on the AP - sometimes that can clear odd problems.

[lalitkumar88551]

There is no  ME controller present, only 9800-Cl WLC & one AP connect this WLC . but issue is why AP join & disjoin continuously within 2-3 minutes & this  happening continue i checked  almost 6 hours still AP join & disjoin.  

[Rich R]

So is the 9800-CL present locally on the same vlan/subnet?

If so that rules out link latency and hopefully MTU.  But you should still check switch ports and VM ports for errors or duplex mismatch for example.  

And you should still try factory default reset on the AP.

Next do a packet capture and see what's happening at packet level between the WLC and AP.

[thecowsquishmallow]

I think you need to be greater precise on how you're trying out. You are transferring from ME to a 9800, so the ap could drop all client whilst upgrading and converting the system from ME to CAPWAP. The ME manual states that a ME ap whilst powered up or rebooted will check the local subnet or any dhcp/dns discovery you have and could be part of a controller if one is located. If no longer, the ap will transition lower back to ME.

[lalitkumar88551]

AP already converted ME to CAPWAP, i just facing issue AP join & disjoin with 9800 WLC & client disconnected internet  when AP join & disjoin process happening . 

[lalitkumar88551]

what's the important of BLE radio.  just capture AP debuging log & found BLE radion Powering down.

[*08/22/2022 07:00:44.4883] dtls_connection_find_using_link_info: Searching connection [192.168.1.185]:5256--[192.168.1.51]:5246, result 0x2bc47800
[*08/22/2022 07:00:44.4883] dtls_cssl_msg_cb: Received >>> DTLS Header [Length 000d]
[*08/22/2022 07:00:44.4983] local_in_addr_comp: Client and server addresses/port/version of 2 nodes are [192.168.1.185]:5256(65277)--[192.168.1.51]:5246(65277) [192.168.1.185]:5256--[192.168.1.51]:5246
[*08/22/2022 07:00:44.4983] dtls_connection_find_using_link_info: Searching connection [192.168.1.185]:5256--[192.168.1.51]:5246, result 0x2bc47800
[*08/22/2022 07:00:44.4983] dtls_cssl_msg_cb: Sent <<< DTLS Header [Length 000d]
[*08/22/2022 07:00:44.5083] flags value is 1
[*08/22/2022 07:00:45.3980] Powering down BLE radio
[*08/22/2022 07:00:47.0075] local_in_addr_comp: Client and server addresses/port/version of 2 nodes are [192.168.1.185]:5256(65277)--[192.168.1.51]:5246(65277) [192.168.1.185]:5256--[192.168.1.51]:5246
[*08/22/2022 07:00:47.0075] dtls_connection_find_using_link_info: Searching connection [192.168.1.185]:5256--[192.168.1.51]:5246, result 0x2bc47800
[*08/22/2022 07:00:47.0075] dtls_cssl_msg_cb: Received >>> DTLS Header [Length 000d]
[*08/22/2022 07:00:47.0075] local_in_addr_comp: Client and server addresses/port/version of 2 nodes are [192.168.1.185]:5256(65277)--[192.168.1.51]:5246(65277) [192.168.1.185]:5256--[192.168.1.51]:5246
[*08/22/2022 07:00:47.0075] dtls_connection_find_using_link_info: Searching connection [192.168.1.185]:5256--[192.168.1.51]:5246, result 0x2bc47800
[*08/22/2022 07:00:47.0075] dtls_cssl_msg_cb: Sent <<< DTLS Header [Length 000d]

[Rich R]

If you're not using BLE it doesn't matter - concentrate on the advice already provided.