Re: UPLOAD FAILED Internal Server Error

jackwong@grandandtoy.com · ‎12-23-2019

Hello - hoping someone can help with this Internal Server Error.

We are running CSPC 2.8.1.6. It uploaded successfully the first 3 times then it failed every time after that with "Internal Server Error". Manually exporting the zipped collection data then uploading it to SNTC works.

We tried the suggestion posted here but it still didn't work:

https://community.cisco.com/t5/smart-net-total-care-portal-and/unable-to-upload-amp-unable-to-update-from-csp-c/m-p/3936554/highlight/true#M12231

Thanks in advance!

adias · ‎12-23-2019

In deploying the image you would have created the cli admin password? (java based shell)

If so have you also created/activated the users collectorlogin and root ? (real linux shell)

If not do the following for the cli admin shell

admin# pwdreset root 90

And

admin# pwdreset collectorlogin 90

it will activate those accounts and generate their passwords ( keep them where you can refer to them in the future)

after that

Via SSH Login to collector as user collectorlogin and then use command below to connect as root (cannot connect directly)

# su root

and then the following commands and send me in Private message:

# tail -100 /opt/ConcsoTgw/tail-end-gateway-decoupled/bin/CONN_TEG_LOGS/tail-end-gateway.log

# openssl s_client -showcerts -connect concsoweb-prd.cisco.com:443

# tail -100 /opt/ConcsoTgw/tail-end-gateway-decoupled/bin/nohup.out

# wget https://72.163.7.113/HeadEndWebAppServlet/ --no-check-certificate

jackwong@grandandtoy.com · ‎12-24-2019

Thank you for the reply.

Everytime I tried to PM you with the info I'm getting:

The private message you were trying to access does not exist.

Please click the Back button on your browser.

jackwong@grandandtoy.com · ‎01-10-2020

Same error - still can't upload.
Every time I tried to PM you with the info gathered, I get this error on: The private message you were trying to access does not exist.

adias · ‎01-10-2020

ok just reply to the post with the outcome of the following command

cat /opt/ConcsoTgw/tail-end-gateway-decoupled/bin/CONN_TEG_LOGS/tail-end-gateway.log | grep 'Send File failed,http'

jackwong@grandandtoy.com · ‎01-10-2020

cat /opt/ConcsoTgw/tail-end-gateway-decoupled/bin/CONN_TEG_LOGS/tail-end-gateway.log | grep 'Send File failed,http'
189: 0.0.0.0: Nov 19 2019 06:32:18.978 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp383649426-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
190: 0.0.0.0: Nov 19 2019 06:32:18.984 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp383649426-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
159: 0.0.0.0: Nov 24 2019 06:32:27.539 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2012295507-22][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
160: 0.0.0.0: Nov 24 2019 06:32:27.540 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2012295507-22][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
179: 0.0.0.0: Dec 04 2019 10:37:44.772 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp1851229726-22][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:404
180: 0.0.0.0: Dec 04 2019 10:37:44.779 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp1851229726-22][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Not Found
259: 0.0.0.0: Dec 04 2019 10:49:59.799 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp1851229726-16][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:404
260: 0.0.0.0: Dec 04 2019 10:49:59.799 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp1851229726-16][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Not Found
169: 0.0.0.0: Dec 08 2019 06:33:01.574 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp682657622-21][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:404
170: 0.0.0.0: Dec 08 2019 06:33:01.578 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp682657622-21][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Not Found
105: 0.0.0.0: Dec 22 2019 06:32:25.189 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2129844823-21][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:404
106: 0.0.0.0: Dec 22 2019 06:32:25.190 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2129844823-21][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Not Found
105: 0.0.0.0: Dec 23 2019 08:58:03.772 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp201230571-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
106: 0.0.0.0: Dec 23 2019 08:58:03.772 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp201230571-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
89: 0.0.0.0: Dec 23 2019 09:05:30.887 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
90: 0.0.0.0: Dec 23 2019 09:05:30.888 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
126: 0.0.0.0: Dec 23 2019 09:43:26.402 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-16][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
127: 0.0.0.0: Dec 23 2019 09:43:26.409 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-16][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
258: 0.0.0.0: Dec 23 2019 10:41:09.017 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
259: 0.0.0.0: Dec 23 2019 10:41:09.023 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-20][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
280: 0.0.0.0: Dec 23 2019 10:42:37.901 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-18][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
281: 0.0.0.0: Dec 23 2019 10:42:37.902 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-18][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable
372: 0.0.0.0: Jan 05 2020 06:32:01.117 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-21][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=625][isRemote=false]: message: Send File failed,http status code:503
373: 0.0.0.0: Jan 05 2020 06:32:01.119 -0500: %CONN_TEG-6-CONN_STR_INFO: %[ch=connectivityteg][mid=10003][requestTranId=99999999][applianceId=CSP0009043760][customerId=61380][source=Connectivity][component=CONN_TEG][threadName=qtp2081703606-21][className=com.cisco.ca.csp.conn.gateway.tailend.core.command.SendFileCommandHTTPS][methodName=uploadFileILB][lineNumber=628][isRemote=false]: message: Send File failed,http status:Service Unavailable

adias · ‎01-10-2020

ok you now getting timesync errors that mean that the time it is taking from your server and the end server are not synchronized or taking too long.

please correct, by syncing with an NTP server: CONNECT TO CLI with user admin ( not the same as GUI )

admin# timesync

If successful, your upload profile execution should not show an HTTP 503 error. if generic appear again send me the same as last command for log