VACL Capture Woes

KURT HILLIG · ‎12-18-2012

We've got Cat6500s in our data centers, each with both ACE and FWSMs installed.

Our security folks want to have an IDS attached to each 6500 monitoring all traffic on selected networks; we (the network folks) want to have a sniffer available on each box for troubleshooting. The 6500 only allows for two simultaneous SPAN sessions, and the FWSM uses one of them, leaving only one left for the IDS and sniffer to share; but these are independent boxes and one SPAN isn't sufficient, hence we're trying to set up VACL capture instead, to direct "interesting" traffic to the IDS.

Conceptually this is pretty simple:

ip access-list extended ALL_TRAFFIC

permit ip any any

vlan access-map CAPTURE_TRAFFIC 10

match ip address ALL_TRAFFIC

action forward capture

vlan filter CAPTURE_TRAFFIC vlan-list 782

interface TenGigabitEthernet1/15

description IDS Box

switchport

switchport mode access

switchport capture

switchport capture allowed vlan 782

mtu 9216

load-interval 30

There's plenty of traffic on this network:

r-ASBDC-A#show interface Vlan782 | include rate

Queueing strategy: fifo

5 minute input rate 917000 bits/sec, 885 packets/sec

5 minute output rate 19492000 bits/sec, 2729 packets/sec

But the IDS box isn't receiving any traffic, and we're seeing lots of output drops:

r-ASBDC-A#show interface TenGigabitEthernet1/15

TenGigabitEthernet1/15 is down, line protocol is down (monitoring)

Hardware is C6k 10000Mb 802.3, address is 0021.a0ef.6d56 (bia 0021.a0ef.6d56)

Description: IDS Box

MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 10Gb/s

Transport mode LAN (10GBASE-R, 10.3125Gb/s), media type is 10Gbase-SR

input flow-control is off, output flow-control is off

Clock mode is auto

ARP type: ARPA, ARP Timeout 04:00:00

Last input never, output 00:14:48, output hang never

Last clearing of "show interface" counters 00:15:17

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 5008317

Queueing strategy: fifo

Output queue: 0/40 (size/max)

30 second input rate 0 bits/sec, 0 packets/sec

30 second output rate 0 bits/sec, 0 packets/sec

0 packets input, 0 bytes, 0 no buffer

Received 0 broadcasts (0 multicasts)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

0 input packets with dribble condition detected

0 packets output, 0 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

This router is a 6509E/Sup720-PFC3CXL running 12.2(33)SXJ; the 10GE line card is a WS-F6700-DFC3C.

We've got this working successfully on outher 6500s where the outbound interface is GE, but haven't had any

luck doing this out 10GE ports. What are we missing?

miguel.desantiago · ‎12-18-2012

Why don't you setup a monitor session to monitor the vlan or interface you want and send it to two separate interfaces, the 10Gi and Sniffer? On the Sniffer box, if you are using WireShark, you can setup capture filters to limit what you want to see.

monitor session <1-80> source vlan 782 both

monitor session <1-80> destination interface Te1/15, GiX/X "Sniffer interface"

Hope this helps!

KURT HILLIG · ‎12-18-2012

As I said, we only have one SPAN available - since the FWSM sucks up one - and we need two independent monitor sessions. We use the one available SPAN for the sniffer - since with this we can sniff an interface, or a VLAN, or receive a remote SPAN from another switch in the data center. The IDS only needs to look at VLAN traffic, so VACL capture (if we can make it work) is appropriate for this.

miguel.desantiago · ‎12-19-2012

Run these two commands to verify your config. It should list something similiar to below.

show vlan access-map CAPTURE_TRAFFIC

!
Vlan access-map "CAPTURE_TRAFFIC"  10
        match: ip address ALL_TRAFFIC
        action: forward capture

!

show vlan filter

LAN Map CAPTURE_FILTER:
        Configured on VLANs:  782
            Active on VLANs:  782

Your config looks correct and I dont know your setup but it may be an issue with the port. You may want to setup a laptop with WireShark or some other packet capture software, change the port to a Gi and see if its working correctly.

KURT HILLIG · ‎12-19-2012

r-ASBDC-A#show vlan access-map CAPTURE_TRAFFIC

Vlan access-map "CAPTURE_TRAFFIC" 10

match: ip address ALL_TRAFFIC

action: forward capture

r-ASBDC-A#show vlan filter

VLAN Map CAPTURE_TRAFFIC:

Configured on VLANs: 782

Active on VLANs: 782

r-ASBDC-A#sho run int Te1/15

Building configuration...

Current configuration : 177 bytes

!

interface TenGigabitEthernet1/15

description IDS Box

switchport

switchport capture

switchport capture allowed vlan 782

mtu 9216

load-interval 30

end

r-ASBDC-A#sho int TenGigabitEthernet1/15 | inc put_

input flow-control is off, output flow-control is off

Last input never, output 1d04h, output hang never

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 92321601

Output queue: 0/40 (size/max)

30 second input rate 0 bits/sec, 0 packets/sec

30 second output rate 0 bits/sec, 0 packets/sec

0 packets input, 0 bytes, 0 no buffer

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

0 input packets with dribble condition detected

0 packets output, 0 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

I've tried adding "switchport capture allowed vlan all" to the interface config; this made no difference.

miguel.desantiago · ‎12-20-2012

As I said your config and setup all looks correct so there is no problem in that area. You should now move on to testing if its the Te1/15 port that is causing the problem. Do you have any GigEthernet ports on your 6500? If so do as I suggested... change the switchport capture to the GigEthernet port and connect a packet capture machine. See if the packet capture is able to see any of the traffic.. you may want to try looking in the log, turn on terminal monitor if connecting through ssh or telnet, and try the debug vacl command to see any errors.