FWSM/Cat 6500 - Traffic from fwsm context hitting RP -> high interrupts

Malette40 · ‎04-25-2015

Hello,

We have at work 1 Catalyst 6509 with 2 SUP720-3B, running 12.2(33)SXJ9 and a FWSM card with a 4.1(15) firmware version.

My problem is that the cpu of our 6509 Route Processor is at ~65% with like 30-35% of interrupts for quite some time.
Using this documentation (http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/116475-technote-product-00.html),
I launched a "debug netdr capture" to view the trafic punted to the RP.

Most of the trafic I saw is trafic from or to the fwsm. Here is an example:

------- dump of incoming inband packet -------
interface Vl660, routine mistral_process_rx_packet_inlin, timestamp
15:43:16.862
dbus info: src_vlan 0x294(660), src_indx 0x340(832), len 0x46(70)
   bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x380(896)
   F8020000 02940000 03400300 46080000 00060000 00000000 00000000 03800000
mistral hdr: req_token 0x0(0), src_index 0x340(832), rx_offset 0x76(118)
   requeue 0, obl_pkt 0, vlan 0x294(660)
destmac 00.D0.02.B0.C0.00, srcmac 00.26.0B.A9.19.00, protocol 0800
protocol ip: version 0x04, hlen 0x05, tos 0x00, totlen 52, identifier 62978
   df 1, mf 0, fo 0, ttl 64, src 172.29.52.81, dst 172.29.234.252
     tcp src 54424, dst 445, seq 2571338139, ack 3250842871, win 2720
off 8 checksum 0x97BD ack

------- dump of outgoing inband packet -------
interface Vl801, routine draco2_fastsend, timestamp 15:43:16.862
dbus info: src_vlan 0x321(801), src_indx 0x340(832), len 0x46(70)
   bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x380(896)
   00020000 03212800 03400300 46080000 00060000 00000000 00000000 03800000
mistral hdr: req_token 0x0(0), src_index 0x340(832), rx_offset 0x76(118)
   requeue 0, obl_pkt 0, vlan 0x294(660)
destmac 00.50.56.94.0C.DB, srcmac 00.D0.02.B0.C0.00, protocol 0800
protocol ip: version 0x04, hlen 0x05, tos 0x00, totlen 52, identifier 62978
   df 1, mf 0, fo 0, ttl 63, src 172.29.52.81, dst 172.29.234.252
     tcp src 54424, dst 445, seq 2571338139, ack 3250842871, win 2720
off 8 checksum 0x97BD ack

Here's some some help for understanding the output:
Vlan 660 = /30 vlan interconnecting the cat 6509 and the FWSM outside iface
Vlan801 = vlan whose SVI is directly on the cat 6509
00.D0.02.B0.C0.00 = mac addr corresponding to the mac addr of the cat 6k in vlan 660
00.26.0B.A9.19.00 = mac addr corresponding to the FWSM outside if

Here is the configuration of the Vlan660 SVI:

interface Vlan801
 description market
 ip address 172.29.234.254 255.255.255.0
 ip access-group market-in in
 ip access-group market-out out
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip flow ingress
end

I see really lots of these in the netdr output, and I don't really knows if this should happen, to my understanding this traffic should be hardware switched and should not go to the RP...
If anyone has ideas, this would be greatly appreciated.

Thank you,
Regards,

Malette40 · ‎04-28-2015

After some more testing, I found why.

My vlan interconnecting the MSFC and the FWSM was configured like below:

interface Vlan660
 description FWSM
 ip address 172.29.30.114 255.255.255.252
 no ip redirects
 ip flow ingress
 ip verify unicast source reachable-via rx 199
end

The "ip verify unicast source..." was the culprit. Access-list 199 was just a one line with "deny ip any any log-input". Based on this doc (http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SX/configuration/guide/book/secure.html), I found that traffic denied by the urpf acl was sent directly to the RP for an uRPF check.

Anyway, Thank you for looking at this problem.

Regards