cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1446
Views
0
Helpful
12
Replies

Cisco Ironport S360 stopped capturing traffic

Joel Fox
Level 1
Level 1

Good afternoon - I have an Ironport S360 (EOL in ~30 days....) that has stopped capturing traffic. From what I can tell, both P1 and P2 are operational (int the Up/Up status on our core switch). There are also 2 L4 interfaces connected to our switch to capture "the rest" of the suspect traffic. The weird thing is that nothing configuration-wise has changed on the appliance, switch, or firewall. (I know this because we have an application to take daily backups and compare changes). In the past when the device seems to have gone out to lunch, a simple reboot worked. I have done that, but no joy. We are running AsyncOS for Web v8.5.3-069. For the P1 and P2 interfaces, p1 is on one vlan, and p2 is on another. All vlans can talk to each other, so communication isn't the issue. for the L4 interface, it is monitoring a Vlan (let's say 999). and the firewall is also monitoring vlan 999.
Internet / IPS / IDS are working, but Ironport is not. Any suggestions? I'm working on straightening out our Smartnet issue, but until then I can't engage TAC so I'm hoping someone else has had this issue as well.

Thanks in advance!

12 Replies 12

How is the traffic getting to the WSA? WCCP or explicit proxy?  What firewall are you using?

If its WCCP, check the WCCP logs, you might have to set them to debug level.

Also, I've removed/readded the WCCP config on the firewall side, or kicked the proxy, instead of a reboot...

To kick the proxy, in the cli: 

wsav1> diagnostic


Choose the operation you want to perform:
- NET - Network Diagnostic Utility.
- PROXY - Proxy Debugging Utility.
- REPORTING - Reporting Utilities.
[]> proxy

- SNAP    - Take a snapshot of the proxy
- OFFLINE - Take the proxy offline (via WCCP)
- RESUME  - Resume proxy traffic via (via WCCP)
- CACHE   - Clear proxy cache
[]> kick

The firewall is an ASA5525-X,  and it's getting there via WCCP. I have a syslogger application, I'll send the wccp logs to that.  I'll try removing and re-applying WCCP config, I haven't tried that yet. If all else fails, I'll try the proxy route. On that note, I'm assuming that since it's a hidden command, it's basically ensuring a hard reset of the WCCP service....

While typing this, we turned WCCP debugging on for the firewall, and saw the Here I Am packet with a bad receive id...

Now I've changed direction and I think I'm going to issue the Kick command as that solved a couple other issues I saw in the forums. Since this is an undocumented hidden command, is there anything else I need to do besides issue the command to re-initalize the service? Like get my resume in order, box up my things, etc? haha.

Still no luck; here is what I have done so far (in order):

On the ASA - removed the wccp web-cache redirect and interface lines

On the WSA - diagnostic - proxy - kick

reboot the WSA

While the WSA is booting, added the wccp commands back in

Using the "debug  wccp events" command, I still get the following message: WCCP-EVNT:S00: Here_I_Am packet from 192.168.1.254 w/bad rcv_id 00000000

When I added the wccp commands back in to the ASA I got the following messages:

WCCP-EVNT: Creating WCCP process

WCCP-EVNT: Adding WCCP-UDP & GRE-WCCP NP rules

What I don't quite understand is the GRE-WCCP rules - we don't use GRE; at least not on our core switch.

GRE is used in the WCCP between the wsa and Asa.  It's an option in the wsa config 

What Asa code version are you running?

Ok, that makes sense if it's "behind the scenes" so to speak. We are running 9.7(1)

Hmmm.... that rules out a bug we hit in the 9.1.x range...

Are the interfaces for the WSA and the inside of the firewall on the same subnet?

Might make sense to change try changing to L2 on the WCCP config.

You will want to get to 9.7.1-4 or higher on the ASA before you get to 213 days uptime... there's a counter that kills the box when it overflows CSCvd78303

No, they are on different subnets. I can surely try that though. Thank you for the heads up on the FW bug! It has only been up 58 days, but I will certainly have it upgraded before then.

I am trying to read through the manuals now; this is a crash course of Ironport configuration for me as our admin is out.  Do they need to be on the same interface to make the suggested change?

for you to use L2, they have to be layer 2 adjacent, eg, no routers...

http://www.cisco.com/web/services/news/ts_newsletter/tech/chalktalk/archives/200806.html

Ken - first, thank your for all of your help! just this week alone I've learned much more about Ironport. Second, I am going to abandon the s360 as I have a s170 that I'm going to use. I do have one question though (or maybe a couple, depending on how the first question goes). We are using all 3 interfaces of our 360 (technically 4 counting the management interface). Naturally two are for P1 and P2, and the other 1 is for L4. Our current configuration is as follows:

All interfaces are on our core L3 switch:

M1 - vlan 300 (192.168.3.x subnet) gig port 0/1

P1 - vlan 100 (192.168.1.x subnet) gig port 0/2

P2 - vlan 200 (192.168.2.x subnet) gig port 0/3

L4 - vlan 1000 (Internet vlan) gig port 0/4

Our inside firewall interface is plugged into vlan 1000 as well on gig port 0/5.

We also have a monitor session created for this:

session 1 source is gi0/5

session 1 destination is gi0/4

It seems that Ironport does not collect reporting very well.. Is there a better or more efficent way to set this up since I'm setting it up brand new?

Thank you for all your help again!

Joel

So, your config looks like it was based on either doing an "inline" config, or an explicit config where you tell machines on each subnet what IP to hid... all of which is probably moot since you're using WCCP... 

We do the following:

M1 - wherever you want management/login/etc traffic.  The web management interface/ssh/ftp to the box to get logs off/ActiveDirectory/Ldap/etc.

P1 - on vlan 1000  eg the web cache address in the firewall

L1 - on vlan 1000 - L4TM stuff.


Web traffic in this scenario goes to the firewall, if it sees the WSA up (because WCCP is happy) it wraps the traffic and sends it over to the WSA, the WSA surfs for the user, and sends the data back to the user...

If you don't have to push the WCCP traffic through a router hop, you're saving some load and headache...

So am I correct in thinking that the monitoring session is not doing anything but creating processor overhead on my switch for no reason? This was all set up well before I got involved, and my goal is to use this "down" time as effectively as possible and make things more efficient if I can do so.

The only router hop we have is the Firewall sending traffic to the Internet Gateway. I should note that we are an MPLS network, and handle all Internet traffic for the company through our corporate office.

Also - I have no idea why P2 is on a different vlan. Aside from the management IP on the core switch, nothing else is on that vlan. Just seems odd to me!

The monitor session is doing something different, you can leave that on, and plug in the L1 on the new box into ge/04

There are 2 things happening.

WCCP pulls port 80/443/and whatever other ports out of outbound traffic and sends it to the proxy process listening on P1.

Monitor session sends everything to L1, and that's is looking for other stuff that ISN'T web traffic.  eg command and control info for botnets, and host of other baddies... Think of it as sort of a "poor man's IPS".   Google "WSA L4TM"

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: