High CPU due to IP Input- Cisco 7600-SUP7203BXL

yogesh rajguru · ‎12-19-2013

Hi All,

We are facing High CPU issue due to IP Input process.

As per NETDr capture & we observed that Broadcast generated by majorily 2 MAC-Addresses.

00.E0.ED.1A.2D.DB Source Vlan 226

00.25.90.20.DB.9E Source Vlan 6

Apart from this able to see other repetative entries for

destmac 00.19.07.AA.45.00, srcmac 00.18.74.1F.8A.C0 Source via either Gig4/2 or Gig 5/2 and

destmac 00.00.0C.07.AC.06, srcmac 00.1A.E2.3A.73.00 Source via Vlan 6.

, which unable to understand.

------- dump of outgoing inband packet -------

interface Gi4/2, routine draco2_ibc_soutput, timestamp 17:18:21.882

dbus info: src_vlan 0x400(1024), src_indx 0x380(896), len 0xA3(163)

bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x0(0)

00020000 04002800 03800000 A3000000 00000000 00000000 00000000 00000000

mistral hdr: req_token 0x0(0), src_index 0x380(896), rx_offset 0x30(48)

requeue 0, obl_pkt 0, vlan 0x0(0)

destmac 00.19.07.AA.45.00, srcmac 00.18.74.1F.8A.C0, protocol 0800

layer 3 data: 45000091 00004000 3F1103DB 3BA0F019 77FC93CB 0035634E

007DA0A8 0F0E8100 00010001 00000000 0B6D6174 74697061

6E646579 04617574 00000400 00000345 00000380 081E

------- dump of outgoing inband packet -------

interface Gi5/2, routine draco2_ibc_soutput, timestamp 17:18:21.882

dbus info: src_vlan 0x408(1032), src_indx 0x380(896), len 0x42(66)

bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x0(0)

00020000 04082800 03800000 42000000 00000000 00000000 00000000 00000000

mistral hdr: req_token 0x0(0), src_index 0x380(896), rx_offset 0x30(48)

requeue 0, obl_pkt 0, vlan 0x0(0)

destmac 00.19.07.AA.45.00, srcmac 00.18.74.1F.8A.C0, protocol 0800

layer 3 data: 45000030 6D284000 3E06983E 3BA0F0A8 77FC931C 62D91FCD

9FB361A8 00000000 700216D0 B23B0000 02040564 01030300

6C730461 00000000 00000408 00000345 0000

------- dump of incoming inband packet -------

interface Vl6, routine mistral_process_rx_packet_inlin, timestamp 17:18:21.894

dbus info: src_vlan 0x6(6), src_indx 0x345(837), len 0xAC(172)

bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x380(896)

E8020400 00060000 03450300 AC080000 00110000 00000000 00000000 03800000

mistral hdr: req_token 0x0(0), src_index 0x345(837), rx_offset 0x76(118)

requeue 0, obl_pkt 0, vlan 0x6(6)

destmac 00.00.0C.07.AC.06, srcmac 00.1A.E2.3A.73.00, protocol 0800

protocol ip: version 0x04, hlen 0x05, tos 0x00, totlen 154, identifier 0

df 1, mf 0, fo 0, ttl 64, src 59.160.240.25, dst 119.252.147.213

udp src 53, dst 10118 len 134 checksum 0x6685

------- dump of incoming inband packet -------

interface Vl6, routine mistral_process_rx_packet_inlin, timestamp 17:18:21.894

dbus info: src_vlan 0x6(6), src_indx 0x345(837), len 0x40(64)

bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x380(896)

40020400 00060000 03450300 40080000 00060000 00000000 00000000 03800000

mistral hdr: req_token 0x0(0), src_index 0x345(837), rx_offset 0x76(118)

requeue 0, obl_pkt 0, vlan 0x6(6)

destmac 00.00.0C.07.AC.06, srcmac 00.1A.E2.3A.73.00, protocol 0800

protocol ip: version 0x04, hlen 0x05, tos 0x00, totlen 40, identifier 5057

df 1, mf 0, fo 0, ttl 64, src 202.54.124.191, dst 202.137.236.183

tcp src 29498, dst 12987, seq 3514083490, ack 3875320802, win 32767 off 5 checksum 0x76B1 ack

Attaching the NETDR capture as a reference.

How to troubleshoot it further.

Regards,

YSR.

lmediavilla · ‎12-19-2013

on some cisco gear you can do rate-limit command on the port, you can try to stop the broadcast traffic with that command.

cheers

vikram singh · ‎12-20-2013

If this happen again please check the below things.

Check the NAT translation table if device is NAT enabled.

Check your security device in the network ( for eg- Fortinet, Sliverpeak, Alertlogic).

if so then disable the port for that device for a while and check again.

I came accross the issues like this and in my case one of the security device was creating mess in my network.