Re: AC SMC stopped suddenly to work

justacsmcinuse · ‎12-08-2023

AnyConnect Secure Mobility Client for Linux in use here.

4.10.06079 is the only version I get from remote network administrators as of time being.

This version of AC SMC for Linux is installed and in use. It worked very well till couple of days ago. From the day on it stopped to work one gets only error message "Connection attempt failed". Two authentication factor is set for affected vpn connection, however due to error mentioned the vpn connection establishment doesn't reach the step where authentication starts. Authenticator app as second factor is used.

Windows 10 and iOS devices in same LAN have no problems vpn to connect to same remote network.

Also remote network IT-team checked it on their side with their Linux machine and they have seen no problem as for their test.

How to proceed further?

Logs as attached below were found, whereby following few lines are eventually of special interest:

Dec 08 07:36:43 machi acvpnui[14342]: Function: PeerCertVerifyCB File: ../../vpn/Api/CTransportCurlStatic.cpp Line: 1114 Return success from VerifyServerCertificate
Dec 08 07:36:43 machi acvpnui[14342]: Function: errorCB File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 119 xml errorCB: Document is empty
Dec 08 07:36:43 machi acvpnui[14342]: Function: startParser File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 206 Invoked Function: xmlParseDocument Return Code: -1 (0xFFFFFFFF) Description: MTUADJUSTMENTCACHE_ERROR_UNKNOWN
Dec 08 07:36:43 machi acvpnui[14342]: Function: processXML File: ../../vpn/Api/xml/AggAuth.cpp Line: 71 Invoked Function: XmlParser::parseXml Return Code: -33554423 (0xFE000009) Description: Unable to process response from Gateway.
Dec 08 07:36:43 machi acvpnui[14342]: Function: processResponseStringFromSG File: ../../vpn/Api/ConnectMgr.cpp Line: 11894 Invoked Function: AggrAuth::processXML Return Code: -27590645 (0xFE5B000B) Description: AGGAUTH_ERROR_FAILED_TO_PARSE_XML

context log

An SSL VPN connection to vpn.unihamm.org has been requested by the user.
Dec 08 07:36:42 machi acvpnui[14342]: Loading preferences for the current user from profile /opt/cisco/anyconnect/profile/vpn.unihamm.org.xml
Dec 08 07:36:42 machi acvpnui[14342]: Function: errorCB File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 119 xml errorCB: Document is empty
Dec 08 07:36:42 machi acvpnui[14342]: Function: startParser File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 206 Invoked Function: xmlParseDocument Return Code: -1 (0xFFFFFFFF) Description: MTUADJUSTMENTCACHE_ERROR_UNKNOWN
Dec 08 07:36:42 machi acvpnui[14342]: [282B blob data]
Dec 08 07:36:42 machi acvpnui[14342]: Function: errorCB File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 119 xml errorCB: Document is empty
Dec 08 07:36:42 machi acvpnui[14342]: Function: startParser File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 206 Invoked Function: xmlParseDocument Return Code: -1 (0xFFFFFFFF) Description: MTUADJUSTMENTCACHE_ERROR_UNKNOWN
Dec 08 07:36:42 machi acvpnui[14342]: [261B blob data]
Dec 08 07:36:42 machi acvpnui[14342]: Current Preference Settings: ServiceDisable: false CertificateStoreLinux: All ShowPreConnectMessage: false AutoConnectOnStart: false MinimizeOnConnect: true LocalLanAccess: true DisableCaptivePortalDetection: false AutoReconnect: true AutoUpdate: true LinuxLogonEnforcement: SingleLocalLogon LinuxVPNEstablishment: LocalUsersOnly ProxySettings: Native AllowLocalProxyConnections: true PPPExclusion: Disable PPPExclusionServerIP:  AutomaticVPNPolicy: false TrustedNetworkPolicy: Disconnect UntrustedNetworkPolicy: Connect TrustedDNSDomains:  TrustedDNSServers:  TrustedHttpsServerList:  EnableScripting: false TerminateScriptOnNextEvent: false EnableAutomaticServerSelection: false AuthenticationTimeout: 12 IPProtocolSupport: IPv4 AllowManualHostInput: true BlockUntrustedServers: true PublicProxyServerAddress:  CertificatePinning: false
Dec 08 07:36:42 machi acvpnui[14342]: Function: setConnectionData File: ../../vpn/Api/ConnectMgr.cpp Line: 2080 Resetting client certificate list.
Dec 08 07:36:42 machi acvpnui[14342]: Function: getCertList File: ../../vpn/Api/ApiCert.cpp Line: 500 Number of certificates found: 0
Dec 08 07:36:42 machi acvpnui[14342]: Message type information sent to the user: Contacting vpn.unihamm.org.
Dec 08 07:36:42 machi acvpnui[14342]: Initiating VPN connection to the secure gateway https://vpn.unihamm.org
Dec 08 07:36:42 machi acvpnagent[995]: Function: processConnectNotification File: ../../vpn/Agent/MainThread.cpp Line: 15113 Received connect notification (host vpn.unihamm.org, profile vpn.unihamm.org.xml)
Dec 08 07:36:43 machi acvpnagent[995]: Function: resolveHostName File: ../../vpn/Common/Utility/HostLocator.cpp Line: 721 Invoked Function: CHostLocator::resolveHostNameAlt Return Code: -29229035 (0xFE420015) Description: DNSREQUEST_ERROR_EMPTY_RESPONSE
Dec 08 07:36:43 machi acvpnagent[995]: Function: getHostIPAddrByName File: ../../vpn/Common/IPC/SocketSupport.cpp Line: 345 Invoked Function: ::getaddrinfo Return Code: -5 (0xFFFFFFFB) Description: No address associated with hostname
Dec 08 07:36:43 machi acvpnagent[995]: Function: resolveHostName File: ../../vpn/Common/Utility/HostLocator.cpp Line: 733 Invoked Function: CSocketSupport::getHostIPAddrByName Return Code: -31129588 (0xFE25000C) Description: SOCKETSUPPORT_ERROR_GETADDRINFO
Dec 08 07:36:43 machi acvpnagent[995]: Function: ResolveHostname File: ../../vpn/Common/Utility/HostLocator.cpp Line: 843 Invoked Function: CHostLocator::resolveHostName Return Code: -31129588 (0xFE25000C) Description: SOCKETSUPPORT_ERROR_GETADDRINFO failed to resolve host name vpn.unihamm.org to IPv6 address
Dec 08 07:36:43 machi acvpnagent[995]: Function: logResolutionResult File: ../../vpn/Common/Utility/HostLocator.cpp Line: 927 Host vpn.unihamm.org has been resolved to IP address 157.1.242.166
Dec 08 07:36:43 machi acvpnagent[995]: Writing to hosts file:  157.1.242.166        vpn.unihamm.org ###Cisco AnyConnect VPN client modified this file. Please do not modify contents until this comment is removed.
Dec 08 07:36:43 machi acvpnagent[995]: Function: respondToConnectNotification File: ../../vpn/Agent/MainThread.cpp Line: 6319 The requested VPN connection to vpn.unihamm.org will target the following IP protocols and addresses: primary - IPv4 (address 157.1.242.166), secondary - N/A.
Dec 08 07:36:43 machi acvpnagent[995]: Function: determineAcidexMacAddrMapForTlv File: ../../vpn/Agent/MainThread.cpp Line: 6694 [ACIDEX] Determined public interface MAC address 08-00-33-xx-xx-xx (interface IPv4 address: 10.0.xx.xx)
Dec 08 07:36:43 machi acvpnui[14342]: Function: getUserName File: ../../vpn/Api/CTransportCurlStatic.cpp Line: 2843 PasswordEntry username is root
Dec 08 07:36:43 machi acvpnui[14342]: Function: PeerCertVerifyCB File: ../../vpn/Api/CTransportCurlStatic.cpp Line: 1114 Return success from VerifyServerCertificate
Dec 08 07:36:43 machi acvpnui[14342]: Function: errorCB File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 119 xml errorCB: Document is empty
Dec 08 07:36:43 machi acvpnui[14342]: Function: startParser File: ../../vpn/Common/Xml/CVCSaxParser.cpp Line: 206 Invoked Function: xmlParseDocument Return Code: -1 (0xFFFFFFFF) Description: MTUADJUSTMENTCACHE_ERROR_UNKNOWN
Dec 08 07:36:43 machi acvpnui[14342]: Function: processXML File: ../../vpn/Api/xml/AggAuth.cpp Line: 71 Invoked Function: XmlParser::parseXml Return Code: -33554423 (0xFE000009) Description: Unable to process response from Gateway.
Dec 08 07:36:43 machi acvpnui[14342]: Function: processResponseStringFromSG File: ../../vpn/Api/ConnectMgr.cpp Line: 11894 Invoked Function: AggrAuth::processXML Return Code: -27590645 (0xFE5B000B) Description: AGGAUTH_ERROR_FAILED_TO_PARSE_XML
Dec 08 07:36:43 machi acvpnui[14342]: Message type warning sent to the user: Connection attempt has failed.
Dec 08 07:36:43 machi acvpnui[14342]: Function: processIfcData File: ../../vpn/Api/ConnectMgr.cpp Line: 3390 ConnectMgr::processResponseStringFromSG failed
Dec 08 07:36:43 machi acvpnui[14342]: Message type error sent to the user: Connection attempt has failed.
Dec 08 07:36:43 machi acvpnui[14342]: Function: connect File: ../../vpn/Api/ConnectMgr.cpp Line: 2597 ConnectMgr::processIfcData failed
Dec 08 07:36:43 machi acvpnui[14342]: Function: initiateConnect File: ../../vpn/Api/ConnectMgr.cpp Line: 1493 Connection failed.
Dec 08 07:36:43 machi acvpnui[14342]: VPN state: Disconnected Network state: Network Accessible Network control state: Network Access: Available Network type: Undefined
Dec 08 07:36:43 machi acvpnagent[995]: Function: determinePublicAddrCandidateFromDefRoute File: ../../vpn/AgentUtilities/HostConfigMgr.cpp Line: 3057 Invoked Function: CHostConfigMgr::FindDefaultRouteInterface Return Code: -22806495 (0xFEA40021) Description: ROUTETABLE_ERROR_GETBESTROUTE_FAILED IPv6
Dec 08 07:36:43 machi acvpnagent[995]: Function: updatePotentialPublicAddresses File: ../../vpn/AgentUtilities/HostConfigMgr.cpp Line: 3190 Invoked Function: CHostConfigMgr::determinePublicAddrCandidateFromDefRoute Return Code: -22806495 (0xFEA40021) Description: ROUTETABLE_ERROR_GETBESTROUTE_FAILED IPv6
Dec 08 07:36:43 machi acvpnui[14342]: Function: run File: ../../vpn/Api/ConnectMgr.cpp Line: 897 Invoked Function: ConnectMgr::initiateConnect Return Code: -29556727 (0xFE3D0009) Description: CONNECTMGR_ERROR_UNEXPECTED

justacsmcinuse · ‎12-08-2023

Error occurs while AnyConnect client app tries to parse some XML, it looks like it was parsing xml-formatted data received from remote network vpn gateway (SAML dialog?). XML-parsing is made using lib which comes from Gnome. It is libxml2 in version 2.12.1. GNOME maintainer can't investigate further as caller code is closed source and they need xml code going to be parsed at the time of error occurrence which they also don't have.

xml-parsing library which other source can I install for Cisco AnyConnect Secure Mobility Client version 4.10.06079 in order to see if it is parsing lib or caller function which fails?

It is a fully open question if remote network IT-team will one day distribute newer version of Cisco AnyConnect app for use.

$ ldd /opt/cisco/anyconnect/bin/vpn  \
| grep xml \
| grep so \
| sed -e '/^[^\t]/ d' \
| sed -e 's/\t//' \
| sed -e 's/.*=..//' \
| sed -e 's/ (0.*)//' \
| uniq -c \
| sort -n \
| awk '{print $2}' \
| xargs file -L
/usr/lib/libxml2.so.2: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=e5a30c3718622026948a7063b21c639d15ec10a6, stripped

stsargen · ‎12-08-2023

You might want to look at this thread and verify the version of Webkit you have is not affected. https://community.cisco.com/t5/vpn/anyconnect-vpn-login-screen-freezes-on-red-hat-enterprise-linux/td-p/4967635

justacsmcinuse · ‎12-08-2023

Thanks for hint!

Well, test done!

It was my hope for webkit to house the reason. 2.42.2 already installed. All its reverse dependencies I could classify as redundant - not ordered by user/administrator neither by higher-level reverse dependencies. Hence myself removed webkit and all its 3 reverse dependencies. Regarding problem with AnyConnect client unfortunately no improvement.

stsargen · ‎12-08-2023

Have you tried rolling back the version of libxml? I see reports from one other customer seeing an issue with version 2.12 and roll back resolves the issue.

justacsmcinuse · ‎12-08-2023

Myself encountered report that kind too one time. It was however archlinux, Manjaro (this case) doesn't work like archlinux regarding package management - far less freedom in Manjaro this matter.

I removed webkit according to source you were referring to. I didn't touch libxml2 as the web of hot dependencies doesn't allow it, in case of webkit the situation was much much simpler. In case of this particular Manjaro system here which present tons of reverse dependencies concerning libxml2. 2.42.2 is webkit version, not libxml2 one.

By the way is webkit involved in xml-parsing process the Cisco client initiates when vpn connection build starts?

justacsmcinuse · ‎12-08-2023

bumped

justacsmcinuse · ‎12-08-2023

content obsolete

justacsmcinuse · ‎12-08-2023

content obsolete

justacsmcinuse · ‎12-10-2023

Upstream team waits, Linux distro hands off, Cisco shows its back, user suffers libxml2 client Linux distribution doesn't want to make pressure on lib maintainer.

https://bbs.archlinux.org/viewtopic.php?pid=2136112#p2136112

GNOME team needs information if they should analyse

justacsmcinuse · ‎12-11-2023

One further point. Not later as today Cisco AC App 4.10 downloads and installs new version: 4.10.07073. This was observed on Windows platform. Manjaro libxml2 current bug prevents this type operating system users from receiving new version - lock-out - bad situation gets worse for folks affected. It is unclear what problems 07073 fixes. Eventually parsing xml works well with it.

stsargen · ‎12-15-2023

Can you confirm that this issue is now resolved in the 4.10.07073 release? Otherwise we will investigate internally.

justacsmcinuse · ‎12-15-2023

Thanks for your ping. Answer send back to you in private message few minutes ago.

What fixes does 4.10.07073 include?

justacsmcinuse · ‎02-02-2024

Hi @stsargen,

How far is this issue from get fixed in client version 4.10?

My Linux Manjaro still can't connect to remote network. Picture today identical to status as my first report in community forum early December. Yet I need working VPN (based on Cisco solution) for progressing in my tasks.

As far as Manjaro Linux this failure blocks AC client from conducting any further actions, among others automatic update to higher version (upon establishment of VPN connection).

Witch 4.10.08025 still reproducible on Manjaro. Latest one still not adopted libxml2 2.12.4.

stsargen · ‎02-02-2024

This issue is not yet resolved. You can track issue using Cisco bug toolkit. https://bst.cisco.com/bugsearch/bug/CSCwi70170