Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
1088
Views
3
Helpful
4
Replies

LMS 4.1 -- WebUI Service Temporarily Unavailable -- lots of CLOSE_WAIT Sockets

lo.mueller
Level 1
Level 1

‎11-14-2012 11:36 PM

Hi all,

environment: LMS 4.1 on Solaris 10

from time to time Web-UI Login is not possilbe showing a message

"Service Temporarily Unavailable"

Troubleshooting the problem we discovered that lots of TCP-Connections are in CLOSE_WAIT state when WEBUI-Unavailabilty

comes up.

Are there any know issue or any recommended OS/Application-Settings overcoming this issue?

Best Regards

Lothar

######################################

>>> About 73 Sockets are in CLOSE_WAIT status

# netstat -a | grep apch | grep CLOSE_WAIT | sort -k 1,1

enbw-198.34485       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.35318       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.35848       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.42339       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.42893       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.45778       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.45945       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46062       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46204       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46233       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46306       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46324       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46355       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46515       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46573       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46717       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46817       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.46962       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47209       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47216       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47258       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47283       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47289       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47351       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47540       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47560       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.47835       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48045       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48128       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48282       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48377       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48394       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48396       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48441       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.48592       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.62470       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.62558       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.62657       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.62747       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.62875       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.62962       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.63123       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.63317       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.63369       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.63466       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.64141       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.64197       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

enbw-198.65445       enbw-198.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.33997      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.35025      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.36753      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44102      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44104      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44108      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44124      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44130      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44147      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44656      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44690      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44693      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44701      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.44732      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.45251      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.45755      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.45787      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.45875      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.46300      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.46317      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.46353      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.46789      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.48253      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.48451      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

localhost.63179      localhost.cscoapch_ssl 49152      0 49152      0 CLOSE_WAIT

Labels:
0 Helpful
4 Replies 4

Vinod Arya
Cisco Employee
Cisco Employee

‎11-15-2012 01:26 AM

You should stop the daemon manager completely and see no casuser initiated process is there.

If there is any other Software which is taking all the connections and details, please remove/disable it and try restarting the daemon manager.

-Thanks Vinod **Rating Encourages contributors, and its really free. **

lo.mueller
Level 1
Level 1
In response to Vinod Arya

‎11-15-2012 02:41 AM

Hi Vinod,

thanks for your answer.

Restarting the daemons is what I just did.

As this was not the first time having this issue, I'm not quite sure if we are facing a more general

problem.

It seems that for certain reasons TCP-Sockets won't be closed correctly from the processes

participating in communication (all local traffic hostname=enbw-198)

Could there be a more general problem with Tomcat and/or ActiveMQ?

Thanks for your help

Lothar

0 Helpful

Vinod Arya
Cisco Employee
Cisco Employee
In response to lo.mueller

‎11-20-2012 05:09 AM

If there is no casuser initiated process there is no known issues with Tomcat or ActiveMQ. Only if the server is co-installed with other NMS application which is using some similar ports it may be an issue.

You may want to check prstat -u casuser to check if any processes is stil working when deamon is completely down.

-Thanks

-Thanks Vinod **Rating Encourages contributors, and its really free. **
0 Helpful

lo.mueller
Level 1
Level 1

‎01-28-2013 10:44 PM

Hi there,

I opened a TAC SR.

Seems that source of the problem is with HTTPS and ACS (TACACS).

Keep you informed

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents