I worked with cisco on this

psnyder1985 · ‎06-04-2015

Every morning when all of our users get to work and start logging in our web traffic goes to a halt through our IronPort S170. It is so bad that it takes 5-10 minutes to get to the CLI through SSH after authenticating.

I am newer to the Cisco IronPort product and know about 8 months ago the hard drives were replaced with higher performing harddrives recommended by Cisco. From the looks of it I am imagining all of our RAM is being used up or the disk performance is that bad and is causing the problems. I noticed that there are a high number of idle connections where it looks like the Iron Port is not tearing them down and our Response Time (ms) is through the roof.

Could I get some insight from some others on this situation. After finally getting into the GUI I took a screen shot of the Status Page?

Here was the first Status Run that I was able to do:

-----------------------------------

Status as of: Thu Jun 04 08:39:54 2015 CDT
Up since: Thu Nov 13 07:00:12 2014 CST (203d 39m 41s)
System Resource Utilization:
CPU 1.2%
RAM 78.0%
Reporting/Logging Disk 65.6%
Transactions per Second:
Average in last minute 71
Bandwidth (Mbps):
Average in last minute 2.533
Response Time (ms):
Average in last minute 5235
Connections:
Total connections 2143

--------------------------------------

Here was the Status Detail Run I was able to do a little while after this

Status as of: Thu Jun 04 08:52:00 2015 CDT
Up since: Thu Nov 13 07:00:12 2014 CST (203d 51m 46s)
System Resource Utilization:
CPU 33.0%
RAM 74.1%
Reporting/Logging Disk 63.7%
Transactions per Second:
Average in last minute 38
Maximum in last hour 79
Average in last hour 50
Maximum since proxy restart 79
Average since proxy restart 53
Bandwidth (Mbps):
Average in last minute 2.160
Maximum in last hour 10.942
Average in last hour 5.008
Maximum since proxy restart 10.942
Average since proxy restart 5.600
Response Time (ms):
Average in last minute 2635
Maximum in last hour 4325
Average in last hour 1690
Maximum since proxy restart 4325
Average since proxy restart 1606
Cache Hit Rate:
Average in last minute 1
Maximum in last hour 4
Average in last hour 1
Maximum since proxy restart 4
Average since proxy restart 1
Connections:
Idle client connections 994
Idle server connections 414
Total client connections 1244
Total server connections 899

psnyder1985 · ‎07-07-2015

I worked with cisco on this and found there was an old AD entry that had to be updated and a DNS entry internal to our network that needed updated and these two failures were causing all of our problems.

Vance Kwan · ‎07-12-2015

If you are on version 8.5.X, that version of AsyncOS was overhauled, and a new version of FreeBSD was used (AsyncOS is built on FreeBSD). 8.5.X is using FreeBSD 9.2 for the very first time. This is an open bug where S160's and S170's took a hit on disk performance (a major one) presumably due to the FreeBSD upgrade.

You can either go back to version 8.0.X, or take measures to I/O utilization on the S170 until the bug is fixed. Increasing auto updates to a higher time value is common (default is 5m. I recommend 4h or so).

It all depends on what you can get away with depending on your traffic load.

-Vance

Vitaliy.Drobyazko1 · ‎03-17-2016

The same !
Very Very slow works Web interface (click .. wait 5-10 minutes)
When Fixes ??
---------------------------------------------------------------
UDI: S170 V05 *
Name: S170
Product: Cisco S170 Web Security Appliance
Model: S170
Version: 9.0.1-162
Build Date: 2016-02-18
Install Date: 2016-03-17 12:09:08
Serial #: *
BIOS: 9B1C118A
RAID: 02
RAID Status: OPTIMAL
RAID Type: 1
BMC: 2.03
---------------------------------------------------------------

Mohammed Idris · ‎04-10-2016

Hi Vitaly,

I am also facing the same issue. Have you found any workaround. Please let me know.

An application fault occurred: ('poller.pyx coro._coro.kqueue_poller.set_wait_for (coro/_coro.c:14771)|271', "<class 'coro._coro.SimultaneousError'>", "<SimultaneousError co=<coro #454 name='<bound method Updates.check_for_upgrade_start_or_not of <shared.Updates.Updates instance at 0x81b5ddfc8>>' dead=0 started=1 scheduled=0 at 0x819750ea0> other=<coro #451 name='finish_request_threaded(*(<coro_ssl.ssl_sock object at 0x81b279090>, ('192.168.111.216', 55836)), **{})' dead=0 started=1 scheduled=0 at 0x815fec3e0> event=<kevent_key filter=-1 ident=75>>", '[_coro.pyx coro._coro._wrap1 (coro/_coro.c:8502)|757] [shared/Updates.py check_for_upgrade_start_or_not|385] [shared/Updates.py check_for_upgrade_start|445] [shared/UI.py immediate_changevar|108] [shared/UI.py _with_trans_varstore|64] [egg/variable_store.py start_trans|1299] [egg/command_client.py send_start_trans|454] [egg/command_client.py send_message|593] [egg/CommandMessage.py read_message|118] [egg/CommandMessage.py _read|97] [socket.pyx coro._coro.sock.recv (coro/_coro.c:19744)|501] [socket.pyx coro._coro.sock._wait_for_read (coro/_coro.c:23099)|1031] [poller.pyx coro._coro.kqueue_poller._wait_for_read (coro/_coro.c:15001)|293] [poller.pyx coro._coro.kqueue_poller._wait_for_with_eof (coro/_coro.c:14917)|285] [poller.pyx coro._coro.kqueue_poller._wait_for (coro/_coro.c:15211)|309] [poller.pyx coro._coro.kqueue_poller.set_wait_for (coro/_coro.c:14771)|271]')

Product: Cisco S170 Web Security Appliance
Model: S170
Version: 9.0.1-162
Serial Number: ***
Timestamp: 10 Apr 2016 14:49:10 +0400

Vitaliy.Drobyazko1 · ‎04-10-2016

Week later
S170 WSA ceased to operate normally.
I migrated to a virtual version -- S300V WSAV

Mohammed Idris · ‎04-10-2016

Thanks for your reply.

Virtual version would be difficult in my case due to hardware and license requirement. Lets me wait for one week and monitor the status. By the way was it the same error message you got as I posted in my previous post ??

Vitaliy.Drobyazko1 · ‎04-10-2016

S170 there were many errors !

If You have Subscription Licenses for users, You can under this License migrated to a virtual version.
Contact Cisco Support, They :
Serial Number S300V will be attached to PAK (where Subscription Licenses),
and send to You new XML license file.

Cisco S170 VERY Slow Performance