Re: Collector upload fails with error 404 Not Found

cmortimer1 · ‎05-02-2019

I have a collector that was working fine and now for the past 2 weeks when trying to upload it comes back with a 404 not Found Error. We are running 2.8.1.3 currently. I have ran packet captures and see the bidirectional traffic between my collector and the Cloud and do not see anything getting blocked. I have verified connectivity to the Upload URL and get the message back "It Works" from the website. I have rebuilt the collector from the ground up and re-certified it multiple times in the Portal. Yet no matter what I still get 404 and cannot find anything to indicate why.

brawall · ‎05-06-2019

Can you share a screenshot of the error as well as the appliance ID of the collector?

cmortimer1 · ‎05-07-2019

CSP0009040738: A connectivity exception occured while processing the request. The Exception is :: HTTPs TransportMode has Failed 404:Not Found

cmortimer1 · ‎05-21-2019

Bump

brawall · ‎05-21-2019

Can you share output of curl --tlsv1.2 https://concsoweb-prd.cisco.com/HeadEndWebAppServlet/ -v -k from the cli?

Thanks,

Brandon

cmortimer1 · ‎05-21-2019

* About to connect() to concsoweb-prd.cisco.com port 443 (#0)
* Trying 72.163.7.113... connected
* Connected to concsoweb-prd.cisco.com (72.163.7.113) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject: CN=concsoweb-prd.cisco.com,O="Cisco Systems, Inc.",L=San Jose,ST=CA,C=US
* start date: Feb 26 18:55:03 2019 GMT
* expire date: Feb 26 19:05:00 2021 GMT
* common name: concsoweb-prd.cisco.com
* issuer: CN=HydrantID SSL ICA G2,O=HydrantID (Avalanche Cloud Corporation),C=US
> GET /HeadEndWebAppServlet/ HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: concsoweb-prd.cisco.com
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Tue, 21 May 2019 16:31:43 GMT
< Server: Apache
< Set-Cookie: ObSSOCookie=loggedoutcontinue; path=/; domain=.cisco.com
< Location: https://sso.cisco.com/obrareq.cgi?wh%3DConn-Prod%20wu%3D%2FHeadEndWebAppServlet%2F%20wo%3D1%20rh%3Dhttp%3A%2F%2Fconcsoweb-prd.cisco.com%20ru%3D%252FHeadEndWebAppServlet%252F
< Content-Length: 357
< Cache-Control: max-age=0
< Expires: Tue, 21 May 2019 16:31:43 GMT
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://sso.cisco.com/obrareq.cgi?wh%3DConn-Prod%20wu%3D%2FHeadEndWebAppServlet%2F%20wo%3D1%20rh%3Dhttp%3A%2F%2Fconcsoweb-prd.cisco.com%20ru%3D%252FHeadEndWebAppServlet%252F">here</a>.</p>
</body></html>
* Connection #0 to host concsoweb-prd.cisco.com left intact
* Closing connection #0

brawall · ‎05-21-2019

Can you share the same but without the "-k" flag?

cmortimer1 · ‎05-21-2019

* About to connect() to concsoweb-prd.cisco.com port 443 (#0)
* Trying 72.163.7.113... connected
* Connected to concsoweb-prd.cisco.com (72.163.7.113) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject: CN=concsoweb-prd.cisco.com,O="Cisco Systems, Inc.",L=San Jose,ST=CA,C=US
* start date: Feb 26 18:55:03 2019 GMT
* expire date: Feb 26 19:05:00 2021 GMT
* common name: concsoweb-prd.cisco.com
* issuer: CN=HydrantID SSL ICA G2,O=HydrantID (Avalanche Cloud Corporation),C=US
> GET /HeadEndWebAppServlet/ HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: concsoweb-prd.cisco.com
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Tue, 21 May 2019 16:38:03 GMT
< Server: Apache
< Set-Cookie: ObSSOCookie=loggedoutcontinue; path=/; domain=.cisco.com
< Location: https://sso.cisco.com/obrareq.cgi?wh%3DConn-Prod%20wu%3D%2FHeadEndWebAppServlet%2F%20wo%3D1%20rh%3Dhttp%3A%2F%2Fconcsoweb-prd.cisco.com%20ru%3D%252FHeadEndWebAppServlet%252F
< Content-Length: 357
< Cache-Control: max-age=0
< Expires: Tue, 21 May 2019 16:38:03 GMT
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://sso.cisco.com/obrareq.cgi?wh%3DConn-Prod%20wu%3D%2FHeadEndWebAppServlet%2F%20wo%3D1%20rh%3Dhttp%3A%2F%2Fconcsoweb-prd.cisco.com%20ru%3D%252FHeadEndWebAppServlet%252F">here</a>.</p>
</body></html>
* Connection #0 to host concsoweb-prd.cisco.com left intact
* Closing connection #0

brawall · ‎05-21-2019

Can you share the appliance ID and version of collector? Has it been upgraded or is it on the same version that it was deployed on?

cmortimer1 · ‎05-21-2019

It has been upgraded, rebuilt fresh, and so on. I originally built a brand new Collector to replace the old one and the new one worked fine for a couple weeks. Then this started. I upgraded it after the fact to see if that would help, but it didn't. I rebuilt it fresh again and still was not working.

Applicane ID: CSP0009040738

Version: 2.8.1.3

brawall · ‎05-21-2019

Can you PM the contents of the file located at /opt/ConcsoTgw/tail-end-gateway-decoupled/conf/csof_config.xml

Also do you require a proxy server for uploads/internet access?

cmortimer1 · ‎05-21-2019

File Contents Sent. Yes we have a Proxy for Internet Access. I do not have it configured on the CSPC itself. We have WCCP in place to redirect HTTP(s) traffic to the Proxy. I have put in exceptions in WCCP to bypass the Proxy and it has not made a difference.

brawall · ‎05-21-2019

Can we try enabling the proxy settings on the collector?

"conf proxy <ip address> <port> <user> <pass>" where user and pass are optional. After this reboot and share with me again the output of "service concsotgw restart"

Thanks,

Brandon

cmortimer1 · ‎05-23-2019

For Some reason I cannot login using Admin, I get the following error:

*****PID file exists. But Adminshell service is not completely up.*****
*****If adminshell is restarted manually, please wait for 15-20 mins .*****

*****If adminshell is not restarted manually Please check nohup.out log at /opt/cisco/ss/adminshell/logs for any errors .*****
*****Try restarting adminshell once using 'service adminshell restart' command .*****

I can login with root and collectorlogin, but then I do not get the adminshell to run the commands. When Trying to configure the Proxy Settings in the GUI, it fails to configure the Proxy Settings.

brawall · ‎05-23-2019

Is admin accessible after restarting it via root?

service adminshell restart